Send pictures to agno

General
1

I’m using the interface: Post /agents/{agent_id}/runs to pass the files parameter. I want to know how the agno will process this picture and send it to the model configured in the agent. If I use the tool or the custom knowledge_retriever method, how should I receive it?

2

Hi @LBWhite55, thanks for reaching out and supporting Agno. I’ve shared this with the team, we’re working through all requests one by one and will get back to you soon.If it’s urgent, please let us know. We appreciate your patience!

3

Hi @LBWhite55 the files parameter will be used by agent and internally passed to the llm model to process it and then have the context to reply.

I assume you want to receive the file passed to agent in the tool right?
Read more about it here- What are Tools? - Agno

Here’s a demo example- File Input for Tools - Agno

4

Hi @kausmos ，When I use a large language model that can only handle text, the “images” printed by the tool are, Here is the printing result：
entered，images:,

now,I’m using a multimodal model in my code, but why are many of the printed “images” information showing as None? Here is the printing result.：
entered，images:[Image(url=None, filepath=‘image.png’, content=None, id=‘da9b7a15-d08e-46b5-9057-f2921ca59f84’, format=None, mime_type=None, detail=None, original_prompt=None, revised_prompt=None, alt_text=None)],
My question is whether the image parameters sent by the agent to the tool are processed internally by the agno framework or processed by a multimodal large - language model?

This is my code：
mcp_tools = MCPTools(
transport=“streamable-http”,
url=“http://localhost:8002/mcp”,
timeout_seconds= 60
)

mcp = FastMCP(name=“MyCustomTools”, host=“0.0.0.0”, port=8002, log_level=“INFO”)

@mcp.tool(description=“if_the_user_uploads_images_call_this_method，Receive the images”)
def get_pic_base64(images: Optional[Sequence[Image]] = None) → str:

logger.info(f"entered，images:{images}")
return "the_tool_obtained_the_specific_picture"

def get_jfr_agent():
jfr_product = Milvus(
uri = f"http://{config.MILVUS_HOST}:{config.MILVUS_PORT}",
collection = “jfr_goods”,
embedder = embedder
)
jfr_product_knowledge = Knowledge(vector_db=jfr_product)

assistant = Agent(
    name = "jfr_assistant",
    role="If the user uploads an image, the tool will be called for processing",
    model = OpenAILike(
        id = "qwen3-omni-flash",
        api_key = "sk-7ff2d2d2ee5b41a99c59e381c72aa75e",
        base_url = "https://dashscope.aliyuncs.com/compatible-mode/v1",
    ),
    tools=\[mcp_tools\],
    add_history_to_context = True,
    db = pg_db,
    add_knowledge_to_context = True,
    knowledge = jfr_product_knowledge,
    send_media_to_model=False,
    markdown = True
)
return assistant

jfr_agent = get_jfr_agent()
agent_os = AgentOS(
os_id = “my_agent”,
agents = [jfr_agent],
)

app = agent_os.get_app()

if _name_ == “_main_”:
agent_os.serve(app = “agentOS:app”, port = 8001, reload = False)

5

Hi @LBWhite55 the images send by the agent are available as is in the tools as a parameter as shown in- Image Input for Tools - Agno

we have some config variables like send_media_to_model which we False in the above example as we strictly want this image to be available in the tool to be processed and not sent to the model (the model attached with agent).

if the variable is True the image will be sent to the model attached to agent. and if that model is not multimodel we suggest to put it False otherwise it could throw error that it does not accept media.