After updating to latest version 1.2.16, team routing logic is not working properly

I’ve updated to latest version 1.2.16. After updating it, I’m noticing team is making tool call transfer_task_to_member. It is passing few member_ids which don’t even exist in my code like 5001, LoanDetailsFetcher. It seem it is hallucinating.

@Dirk @monalisha @mishramonalisha76 @Monali please take a look at this.

Hey @MahorShekhar Thank you for raising this and reaching out to us.
I’ve shared this with the team, we’re working through all requests one by one and will get back to you soon.
If it’s urgent, please let us know. We appreciate your patience!

Just adding some more context here. Please see the below response from LLM

Member with ID loan-offer-analysis not found in the team or any subteams. Please choose the correct member from the list of members

why is the team leader looking for member with ID loan-offer-analysis, even if I’m specifying name and id of all participating agents. from where it is generating these random member ids ?
I tried running team with route and coordinate mode both. Facing same issue with both configuration. this issue persists in v1.2.15 and higher.

@Dirk @Monali @monalisha @kausmos @ayush

Edit:

Additionally, please see the mention of tools. It is just tool name with no description of it. Previously, description retrieved from docstring was being added in previous versions of SDK. I can’t see it now.

@Dirk I’m facing multiple issues in latest version. Previosly my conversation flow and delegation flow were working fine, I was just facing challenges in maintaining session states during conversation at each step | tool call.

Now in latest version, everything seems broken :worried:. Even conversation flow response is not right. Please take a look at this:


class SalesToolkit(Toolkit):
    name = "sales_toolkit"
    instructions = """
    1. `send_email_for_online_application_form_link` will update the session state to `pending_on_user`.
    """

    def __init__(self):
        super().__init__(name=self.name, instructions=self.instructions, add_instructions=True)
        self.register(self.get_customer_data)
        self.register(self.calculate_emi)
        self.register(self.get_historical_credit_history)
        self.register(self.get_recommendation_similar_customers)
        self.register(self.send_email_for_online_application_form_link)

    def calculate_emi(self, amount: int, rate: float, tenure_in_months: int) -> str:
        """
            Calculate the Equated Monthly Installment (EMI) for a loan
        Args:
            amount (int): customer id
            rate (float): rate of interest
            tenure_in_months (int): number of months loan is borrowed for
        Returns:
            str: string containing monthly EMI information for given amount, rate and tenure.
        """
        rate_per_month = rate / 12 / 100
        emi = (amount * rate_per_month * (1 + rate_per_month) ** tenure_in_months) / ((1 + rate_per_month) ** tenure_in_months - 1)
        return f"EMI for ₹{amount} at {rate}% for {tenure_in_months} months is approximately ₹{round(emi)}."

I’ve calculate_emi tool call registered under tool kit, but still sales-agent cannot find it.

@Dirk @Monali @kausmos @ayush @monalisha Something is not right after v1.2.15. :pensive_face:

Hey @MahorShekhar,
We’re so excited to see your recent interest in Agno and how active you’ve been in the community—it’s awesome to have you here!

Apologies for the delays from our side, and thank you for your patience.

Just a small request: please avoid tagging the team members on every doubt—rest assured, we’re monitoring messages and will get back to you as soon as we can.

Also, I’m super curious to hear more about what you’re building with Agno—do share when you get a chance!

Hi @MahorShekhar

You said you upgraded recently. Which version of Agno were you running before that it did indeed work on? Mind sharing a snippet also of how you are setting up the team so we can cross out the obvious things first? Then we can start diving deeper into your issue. We have not seen this before so I am curious to understand why, so we can fix it.

@WillemdeJongh I was using v1.2.15 earlier. I see after v1.2.16, some improvement in teams got rolled out.

team.py

def get_loan_upselling_team(session_id:str=None, user_id:str=None) -> Team:
    loan_upselling_team: Team = Team(
        name="Loan Upselling team.",
        user_id=user_id,
        mode="coordinate",
        session_id=session_id,
        session_state={"state": "running"},
        storage=session_db,
        memory=common_memory,
        model=azure_openai_model,
        members=[sales_agent, verification_agent, decision_making_agent],
        description="A team of agents handling customer queries for personal loan processes at Horizon Financial.",
        instructions=
        """
        General Instructions:

        1. Please read the session_summary if provided in team context to know about past interactions with the user.
        2. You can only handle queries for Personal loan. Other types of loan are out of your area of expertise.
        3. Participating agents can ask you to gather customer input on their behalf that they need to provide response to customer query.
        4. After taking user input, please provide that user input back to that agent which has asked you to gather that from the user.
        5. Note down important user data like customer_id, name etc so that you can recall these data later on and provide these as input when required by other tool calls. You should persist this data in long term memory as well.
        6. If you feel user input is important and could be reused in future tool calls, please save those data in session_data so that you don't need to ask same details twice with user and every participating agent has access to this data.
        7. Tool call response of participating agent might contain information about next session to be updated. If this data exists, please update session state accordingly.
        8. Along with deciding which tool to make, also please decide the session state after tool call would have been executed. Use `update_session_state` to update session state along with each tool call based on the following rules:
          - if goal is not achieved and conversation is still going on, then session state should be `running`.
          - if goal is achieved, then session state should be `completed`.
          - if async tool call is made or you are waiting for response from some third party, then session state should be `waiting_async_tool`
          - if action is pending on the user, then session state should be `pending_on_user`
          - if any exception happens, then session state should be `disconnected`.
        """
        ,
        success_criteria="""
        Following is the success criteria:
        - customer is satisfied with the response and his query is resolved.
        - customer has no further query to ask.
        - customer no longer wishes to continue the conversation..
        - customer is no longer interested in the loan offers.
        - customer loan application process is either approved or declined.
        """,
        add_context=True,
        context={"session_id":session_id},
        add_state_in_messages=True,
        # add_datetime_to_instructions=True,
        enable_agentic_context=True,
        share_member_interactions=True,
        read_team_history=True,
        enable_team_history=True,
        show_tool_calls=True,
        show_members_responses=True,
        debug_mode=True,
        num_of_interactions_from_history=10,

        # new params support in v1.3
        enable_agentic_memory=True,
        enable_user_memories=True,
        enable_session_summaries=True
    )
    loan_upselling_team.tools = [TeamToolkit(loan_upselling_team)]

    return loan_upselling_team

team_toolkit:

class TeamToolkit(Toolkit):
    name = "team_toolkit"
    instructions = """
      Please use `update_session_state` after each team run to update current session state.
    """

    def __init__(self, team: Team|None):

        super().__init__(name="team_toolkit", instructions=self.instructions, add_instructions=True)
        self.team = team
        self.register(self.update_session_state)

    def update_session_state(self, current_session_state:str, new_session_state:str) -> str:
        """updates the team's session state based on following rules:
        - if goal is not achieved and conversation is still going on, then session state should be `running`.
        - if goal is achieved, then session state should be `completed`.
        - if async tool call is made, then session state should be `waiting_async_tool`.
        - if action is pending on the user, then session state should be `pending_on_user`.
        - if any exception happens, then session state should be `disconnected`.

        Args:
            current_session_state: Current session state
            new_session_state: session state to be updated

        Returns:
            session_state: New team session state
        """
        self.team.session_state = {"state": new_session_state}
        return f"Session state: {new_session_state} saved."

sales_agent.py:

sales_agent = Agent(
    agent_id="sales-agent",
    name="Sales Agent",
    model=azure_openai_model,
    description="You are a sales assistant for Horizon Financial that engages customers with loan offers.",
    goal="Present pre-approved loan offers and assist with general loan-related queries.",
    instructions=[
        """
        Workflow:

        1. First gather basic details like customer_id from the customer using tool `get_customer_data`.
        2. If customer is interested in the loans, present loan offers for the customer using the data received from `get_customer_data`.
        3. You can use `calculate_emi` tool to handle queries from customer related to monthly EMIs.
        4. You can use `get_recommendation_similar_customers` tool for providing offers based on similar customer preferences.
        5. If customer is interested in applying for loan, first verify this eligibility and then identity via Verification agent.
        6. After successful verification, ask customer's PAN card for verification. Submit the request to CRM and wait for response.
        7. After successful pan verification, send online application link to customer via email. Once user fills the form, we will move forward with the conversation.
        8. After successful form submission, use decision making agent to take a decision on customer's loan application.


        General instructions:

        1. You are a friendly sales assistant for Horizon Financial.
        2. You will greet the customer and take customer details before initiating a conversation with the user.
        3. If you need any input from user to run tool calls, please ask team leader to gather that information for you.
        4. Use the provided tool calls to retrieve customer data, calculate EMIs or find loan offer recommendations from similar customers.

        """
    ],
    tools=[SalesToolkit()],
    read_chat_history=True,
    read_tool_call_history=True,
    add_history_to_messages=True,
    num_history_responses=10,
    add_state_in_messages=True,
    memory=common_memory,
    add_context=True,
    add_memory_references=True,
    add_session_summary_references=True,
    # show_tool_calls=True,
    # markdown=True,
    # add_name_to_instructions=True,
    # debug_mode=True,
)

Here are the few snippets from my code. Let me know if you need more input from me. I would request you to go through all issues that I’ve spoke of in this thread. Please refer my screenshots for the more context.

CC @Dirk

EDIT:

@WillemdeJongh @Dirk as I asked in previous thread as well, why don’t tool calls have a description in system prompt. It was there previosuly, right ? If I’m not mistaken. Could this be the reason for the drop in response quality, not calling tools properly to be specific.

@MahorShekhar Sorry it is hard to replicate without all of your code. I assume your model is AzureOpenAI(id=gpt-4o)?

Some comments from me:

  • The team leader sometimes attempts to call the functions of the members. I will make minor prompt adjustments to avoid this.
  • We removed the tool_descriptions from the <team_members> because that made the team leader want to call these functions even more. It shouldn’t need these descriptions.
  • In general we had to improve the way that agent-IDs are used because the team often got the team ID wrong. If we can get your full snippet (or atleast a larger sample) we can test better.

@Dirk I strongly believe there is some bug in deciding which tool calls to make and also in method forward_task_to_member. It is forwarding request to some non-existent member id. From where it is deciding that member id is team_leader_id.

Please see my debug logs.

DEBUG *************************************** Agent Run End: e24e4909-ff4a-46ad-8283-2011dc450af5 ****************************************              
DEBUG Updated team context with member name: Sales Agent                                                                                                
DEBUG =============================================================== tool ===============================================================              
DEBUG Tool call Id: call_iVdBv2HLgyF0xYCj1I9sckJn                                                                                                       
DEBUG Member with ID team_leader_id not found in the team or any subteams. Please choose the correct member from the list of members:                   
                                                                                                                                                        
       - Agent 1:                                                                                                                                       
         - ID: sales-agent                                                                                                                              
         - Name: Sales Agent                                                                                                                            
         - Role: Handle loan related queries and fetching customer data.                                                                                
         - Available tools:                                                                                                                             
          - get_customer_data                                                                                                                           
          - calculate_emi                                                                                                                               
          - get_historical_credit_history                                                                                                               
          - get_recommendation_similar_customers                                                                                                        
          - send_email_for_online_application_form_link                                                                                                 
       - Agent 2:                                                                                                                                       
         - ID: verification-agent                                                                                                                       
         - Name: Verification Agent                                                                                                                     
         - Role: Handle loan eligibility related queries.                                                                                               
         - Available tools:                                                                                                                             
          - verify_pan_card                                                                                                                             
          - check_loan_eligibility                                                                                                                      
          - get_bank_balance                                                                                                                            
          - get_average_monthly_balance                                                                                                                 
          - check_payment_irregularities                                                                                                                
       - Agent 3:                                                                                                                                       
         - ID: decision-making-agent                                                                                                                    
         - Name: DecisionMaking Agent                                                                                                                   
         - Role: Handle loan eligibility related queries.                                                                                               
         - Available tools:                                                                                                                             
          - check_credit_history                                                                                                                        
          - make_decision                                                                                                                               
                                                                                                                                                        
DEBUG **********************************************************  TOOL METRICS  **********************************************************              
DEBUG * Time:                        0.0011s                                                                                                            
DEBUG **********************************************************  TOOL METRICS  **********************************************************              
DEBUG -------------------------------------------------------- Azure Response End --------------------------------------------------------              
DEBUG Added RunResponse to Memory                                                                                                                       
┏━ Message ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                                                                                                                                                      ┃
┃ sure, my customer is 123                                                                                                                             ┃
┃                                                                                                                                                      ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
┏━ Sales Agent Tool Calls ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                                                                                                                                                      ┃
┃ • set_shared_context(state=Fetching pre-approved loan offers and recommendations for customer ID 123.)                                               ┃
┃                                                                                                                                                      ┃
┃ • forward_task_to_member(member_id=team_leader_id, expected_output=Fetch and provide pre-approved loan offers                                        ┃
┃   and recommendations for customer ID 123.)                                                                                                          ┃
┃                                                                                                                                                      ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
┏━ Sales Agent Response ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                                                                                                                                                      ┃
┃ Member with ID team_leader_id not found in the team or any subteams. Please choose the correct member from the list of members:                      ┃
┃                                                                                                                                                      ┃
┃  - Agent 1:                                                                                                                                          ┃
┃    - ID: sales-agent                                                                                                                                 ┃
┃    - Name: Sales Agent                                                                                                                               ┃
┃    - Role: Handle loan related queries and fetching customer data.                                                                                   ┃
┃    - Available tools:                                                                                                                                ┃
┃     - get_customer_data                                                                                                                              ┃
┃     - calculate_emi                                                                                                                                  ┃
┃     - get_historical_credit_history                                                                                                                  ┃
┃     - get_recommendation_similar_customers                                                                                                           ┃
┃     - send_email_for_online_application_form_link                                                                                                    ┃
┃  - Agent 2:                                                                                                                                          ┃
┃    - ID: verification-agent                                                                                                                          ┃
┃    - Name: Verification Agent                                                                                                                        ┃
┃    - Role: Handle loan eligibility related queries.                                                                                                  ┃
┃    - Available tools:                                                                                                                                ┃
┃     - verify_pan_card                                                                                                                                ┃
┃     - check_loan_eligibility                                                                                                                         ┃
┃     - get_bank_balance                                                                                                                               ┃
┃     - get_average_monthly_balance                                                                                                                    ┃
┃     - check_payment_irregularities                                                                                                                   ┃
┃  - Agent 3:                                                                                                                                          ┃
┃    - ID: decision-making-agent                                                                                                                       ┃
┃    - Name: DecisionMaking Agent                                                                                                                      ┃
┃    - Role: Handle loan eligibility related queries.                                                                                                  ┃
┃    - Available tools:                                                                                                                                ┃
┃     - check_credit_history                                                                                                                           ┃
┃     - make_decision                                                                                                                                  ┃
┃                                                                                                                                                      ┃
┃                                                                                                                                                      ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
┏━ Team Tool Calls ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                                                                                                                                                      ┃
┃ • set_shared_context(state={'customer_id': '123'})                                                                                                   ┃
┃                                                                                                                                                      ┃
┃ • forward_task_to_member(member_id=sales-agent, expected_output=Fetch and provide pre-approved loan offers and                                       ┃
┃   recommendations based on customer ID 123.)                                                                                                         ┃
┃                                                                                                                                                      ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
┏━ Response (10.4s) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                                                                                                                                                      ┃
┃ Member with ID team_leader_id not found in the team or any subteams. Please choose the correct member from the list of members:                      ┃
┃                                                                                                                                                      ┃
┃  - Agent 1:                                                                                                                                          ┃
┃    - ID: sales-agent                                                                                                                                 ┃
┃    - Name: Sales Agent                                                                                                                               ┃
┃    - Role: Handle loan related queries and fetching customer data.                                                                                   ┃
┃    - Available tools:                                                                                                                                ┃
┃     - get_customer_data                                                                                                                              ┃
┃     - calculate_emi                                                                                                                                  ┃
┃     - get_historical_credit_history                                                                                                                  ┃
┃     - get_recommendation_similar_customers                                                                                                           ┃
┃     - send_email_for_online_application_form_link                                                                                                    ┃
┃  - Agent 2:                                                                                                                                          ┃
┃    - ID: verification-agent                                                                                                                          ┃
┃    - Name: Verification Agent                                                                                                                        ┃
┃    - Role: Handle loan eligibility related queries.                                                                                                  ┃
┃    - Available tools:                                                                                                                                ┃
┃     - verify_pan_card                                                                                                                                ┃
┃     - check_loan_eligibility                                                                                                                         ┃
┃     - get_bank_balance                                                                                                                               ┃
┃     - get_average_monthly_balance                                                                                                                    ┃
┃     - check_payment_irregularities                                                                                                                   ┃
┃  - Agent 3:                                                                                                                                          ┃
┃    - ID: decision-making-agent                                                                                                                       ┃
┃    - Name: DecisionMaking Agent                                                                                                                      ┃
┃    - Role: Handle loan eligibility related queries.                                                                                                  ┃
┃    - Available tools:                                                                                                                                ┃
┃     - check_credit_history                                                                                                                           ┃
┃     - make_decision                                                                                                                                  ┃
┃                                                                                                                                                      ┃
┃                                                                                                                                                      ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
┏━ Session Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                                                                                                                                                      ┃
┃ Session summary updated                                                                                                                              ┃
┃                                                                                                                                                      ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

I’ll be DMing you my code so that you can debug. I request you to please do something as it is a complete blocker. v1.3 has some serious bugs.

@Monali @Dirk I know you guys have so much on your plate right now, but still I would request you guys if you could assist me here as this really is a burning issue. I’m stuck. :pensive_face:

Hey @MahorShekhar
I hope Dirk reachout to you and solved the issue.
If you are still stuck, pls let me know - we are here to help

yes @Monali @Dirk is helping me out on this one. He released v1.3.2 yesterday. But still I’m facing some issue. I’ve provided information for the same to him as well.

1 Like