### Description
I have an agent built with access to a knowledge base. I'm serv…ing the LLM using vllm, and for this test, I set the temperature value to 0 to reduce variability in answers.
When I submit queries using the agno agent in the swagger interface, and submit the same query using the agui endpoint, the results tend to be quite different, including consistent hallucinations from the agui outputs that don't seem to happen through the standard interface.
The knowledge is regarding my docs website: <https://docs.hpc.oregonstate.edu> and the standard agent protocol does a decent job of providing information from the site.
However, the ag-ui result routinely mentions registering with SSH keys with your account (we do mention SSH keys in the docs but don't talk about registering them, and it also mentions connecting to a portal to register the keys (we don't have one), among incorrect abbreviations for our standard IDs (called ONID at Oregon State University, but it mentions OSU Net ID).
I can't seem to figure out if all the info is sent to the LLM when an ag-ui call is made, but I set certain instructions in the expected_output and additional_context attributes of my agent when I initialize, and the standard agent follows the instructions better than when its called though ag-ui.
I was considering building an integration using ag-ui but I may end up using the standard agent API instead.
### Steps to Reproduce
```
curl -X 'POST' \
'http://localhost:22222/agents/docs-assistant/runs' \
-H 'accept: application/json' \
-H 'Authorization: Bearer OSK_dev' \
-H 'Content-Type: multipart/form-data' \
-F 'message=How do I connect using SSH?' \
-F 'stream=false'
```
vs
```
curl -X 'POST' \
'http://localhost:22222/agui' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"threadId": "testing",
"runId": "testing",
"parentRunId": "testing",
"state": "",
"messages": [
{
"id": "test-ssh",
"role": "user",
"content": "How do I connect using SSH?",
"name": "string"
}
],
"tools": [],
"context": [],
"forwardedProps": ""
}'
```
### Agent Configuration (if applicable)
```python
def get_model() -> VLLM:
return VLLM(id=model, api_key=api_key, base_url=host, temperature=0)
def get_vector_db(db_path = Path(VECTOR_DB_DIR), hybrid: bool = True) -> VectorDb:
db_path.mkdir(mode=0o750, parents=True, exist_ok=True)
if hybrid:
table_name = "mkdocs_docs_hybrid"
search_type = SearchType.hybrid
else:
table_name = "mkdocs_docs"
search_type = SearchType.vector
return LanceDb(
table_name=table_name,
uri=db_path.as_posix(),
embedder=get_embedder(),
search_type=search_type,
)
def get_docs_agent(stream: bool = True) -> Agent:
logger.info("Agent model: %s", model)
logger.info("Agent host: %s", host)
return Agent(
name="Docs Assistant",
model=get_model(),
stream=stream,
introduction="You are an expert on the Wildwood HPC at Oregon State University Documentation Site. You were built by the Center for Quantitative Life Sciences (CQLS), including Ed Davis, Bioinformatics Manager.",
role="Helpful HPC Documentation Assistant",
expected_output=" ".join(
[
"Provide accurate answers based on the knowledge available from the website in the <references> section."
"If you cannot determine an adequate answer, include 'Contact [Support](mailto:support@cqls.oregonstate.edu) for help outside this guide.' as part of your response.",
"After providing an answer, include citations at the end of the response, when applicable. See the <citations> section for more details."
]
),
additional_context=" ".join(
[
"<citations>You can find the title and url in the metadata of each reference. For example, if a reference has the following metadata:",
"meta_data: {'url': 'https://docs.hpc.oregonstate.edu/cqls/connecting', 'title': 'Connecting', ...}",
"You would cite it as: [Connecting](https://docs.hpc.oregonstate.edu/cqls/connecting).",
"If multiple references are used, list them all at the end of your answer, e.g.:",
"**References:**\n- [Title1](url1)\n- [Title2](url2)</citations>"
]
),
knowledge=get_knowledge(),
add_knowledge_to_context=True,
search_knowledge=False,
markdown=True
)
agent_os = AgentOS(
id="cqls-hpc-website-kb-os",
description="CQLS Wildwood HPC Documentation Assistant OS",
agents=[get_docs_agent()],
interfaces=[AGUI(get_docs_agent())],
)
```
### Expected Behavior
I expect the output to be similar, or at least more consistently incorrect, between the ag-ui calls and the standard agent calls.
### Actual Behavior
The standard agent and ag-ui calls are both consistent within calls of the same type, but not between calls of different types. And the ag-ui responses are often filled with more hallucinations.
### Screenshots or Logs (if applicable)
_No response_
### Environment
```markdown
- OS: Rocky 9
- Agno version: 2.2.8 and 2.2.10 show the same behavior
```
### Possible Solutions (optional)
_No response_
### Additional Context
I tried to dig into the implementation details, and I couldn't figure out if any of the extra context is passed different if using the ag-ui interface. I did see the encoder logic but I can't imagine why encoding/decoding json would be causing these issues...
At any rate, Agno has been relatively great to use thus far. Thank you for all the hard work.
