Background: I am exclusively using/testing your framework in a private network exclusively using Ollama and a number of LLMs on various servers.
Documentation Update: When you update your documentation in the next release, can you please include your recommendations, knowledge and best-practices on how/what to use when using Ollama. Both your framework and Ollama are rapidly evolving and having the associated growing pains. The comment I just read about Agent replacing Assistant, … would be helpful (and easy to do) in a Ollama Usage recommendation note.
I’ve been doing testing across your Cookbook examples, converting them to exclusively use Ollama. Things I’ve ‘discovered’ from my testing include: In most cases if an agent uses tools, it is better to use OllamaTools (but not necessarily in all cases). Control agents in a multi-agent environment should only use Ollama. Execution of various LLMs in Ollama can generate similar results, but there can be sufficient difference to adversely affect Agent execution. Suitable OpenAPI prompts in (some?) cases are unsuitable for Ollama LLMs.
My biggest request after more overview documentation is to fix all modes of MemoryAgent usage to work with Ollama. Without conversation and session memory, it’s difficult to create a robust virtual assistant.
Finally, it would be helpful to know your development priorities. The fastest, best and most financially lucrative focus is supporting the OpenAI models and ecosystem. But, there is an issue of privacy, which will inhibit major corporations from fully embracing the current environment. So what is your next?
Thanks and you are course to develop a truly great product.
Hi @bills
Thank you for reaching out and using Phidata! I’ve tagged the relevant engineers to assist you with your query. We aim to respond within 48 hours.
If this is urgent, please feel free to let us know, and we’ll do our best to prioritize it.
Thanks for your patience!
Hi @bills
Thanks for the insight! We work directly with some customers with a lot of the same requirements around using open source models via Ollama. I agree, we need to give you (the user) much better insight into using Ollama. I have added it to my list of documentation updates to make.
We have focussed on OpenAI since they are generally ahead of the curve in terms of features and are well known, but that doesn’t mean Ollama is of a lesser concern to us. At the moment “Memory Summarization” doesn’t work with Ollama, but other than that memory should work fine with OpenAI.
The truth is also that we are a small team and can’t get to everything. Feedback from the community greatly influences where we put our focus next, so Ollama will move up the list.
I just now noticed the information about the use of OllamaTools that @bills raised in his post - I’ll try this technique.
Overall, I also think that documenting and verifying the use of Ollama instead of OpenAI would make the library more robust and allow a better development experience, as well as support more Production deployment options.
As you can see from my above post, I also tried changing the examples to use Ollama, and I’m interested to know whether the issues that you encountered were also related to Pydantic validation(s) of what appears to be expected for Tool use, for example:
pydantic_core._pydantic_core.ValidationError: 1 validation error for Tool
function.parameters.properties.additional_information.type
Input should be a valid string [type=string_type, input_value=['string', 'null'], input_type=list]
Do you mind sharing an example (code) of how you solved it using OllamaTools, e.g. for Agent Team?
Unfortunately I am not a Pedantic expert… and am encountering similar uses that you are identifying. In some cases due to time constraints I actually removed the Pedantic dependencies… I too am encountering problems/challenges with multi-Agent usage. Depending on the LLM used, you can get significanlty different results and more/less Pedantic issues. I’m in the process of unwinding Agent results that create Pydantic issues and the generation of wrong/unexpected results. The challenge is working with multiple black boxes and trying to ‘fence in’ their output. I’m currently wishing for better LLMs… that I’m not forced to train. Let me know if you solve some of your issues. We Ollama users will be very appreciative.
Hi @Leonid
This was an issue with how we formatted tool definitions in the case of Ollama. They expect a slightly more strict format of JSON schema for tool definitions. We have since fixed this in version 1.1.5 and above.
@bills I am currently working on a revamp of teams that should be a lot more robust. The current implementation of teams does not handle structured output well. It also does not properly enable multimodal cases. We are making a better version of teams .
Cant agree more with @Dirk . I really like the Agno interface , and would like to use models locally with Ollama, as we are working of some Health-data usecases. Happy to assist in whatever shape or form to push this up the queue.
.