Gemini not using knowledge base when other models do

General
,
1

Example project based on phi ws and using the playground API.
I’m creating a knowledgebase using pgvector and loading in a small set of test documents with unique fabricated content (not found on internet)

using the models openai:gpt-4o-mini or claude-3-opus-20240229 I can use the /v1/playground/agent/run endpoint and get the expected answer.

However using gemini-1.5-flash I get “i don’t know. my knowledge base does not contain information about {subject}”

  • Why might this be?
  • does it matter that the kb was constructed using the PDFKnowledgeBase with AgenticChunking?
2

Hi @avowkind
Thank you for reaching out and using Phidata! I’ve tagged the relevant engineers to assist you with your query. We aim to respond within 24 hours.
If this is urgent, please feel free to let us know, and we’ll do our best to prioritize it.
Thanks for your patience!

3

Hello @avowkind can you please share your Agent config with Gemini along with the debug logs when querying it regarding the documents?

Also, confirming if the same kb is being used for all three Agents

4

Thanks for responding.
I updated to latest version and having rebuilt my kb it is now working with gemini as well as openai
Yes it’s the same kb.

5

Hey @avowkind!

At the heart of it, it really comes down to the agent properly understanding the task and making the right function call. Sometimes, models can hallucinate and need a little extra guidance with explicit instructions to be more reliable, while other times, they handle things well with minimal input. I would love to have a look at your agent config if you can share and to help you guide better.

I know this can be tricky, but happy to help figure it out with you! Let me know how it’s going.