Agent Playground - Agent Respond with Python Charts

General
1

I have been trying out prompts for my Agent to use python to generate simple charts using common python packages. But the issue is that the agent can create the image/rendering of the chart/graph. But it saves it into the folder with the agent playground.py file. Instead of actually responding to me in the playground with the image/rendering. How can I give the agent the ability to actually respond with its created/rendered image? You can see a created rendering below

image
image1017×731 42.7 KB

2

Hi @nathanmcwhorter , thanks for reaching out and supporting Agno. I’ve shared this with the team, we’re working through all requests one by one and will get back to you soon.
If it’s urgent, please let us know. We appreciate your patience!

3

Hi @nathanmcwhorter , Can you please agent config codebase, to help us replicate this issue?

4

@monalisha

Here is my agent configuration.

chart_agent = Agent(
name=“Chart Agent”,
model=OpenAIChat(id=“gpt-4o”),
tools=[PythonTools()],
instructions=[“you are a expert at data visaulizations in Python. You will take the users request and process charts/graphs using basic python libraries such as matlab. Return the completed visual to the user as an attachment”],
storage=SqliteStorage(table_name=“chart_agent”, db_file=agent_storage),
add_datetime_to_instructions=True,
reasoning=True,
add_history_to_messages=True,
num_history_responses=5,
markdown=True,
)

Im using the agent playground to interface with the agent. It looks to be generating the output .png but it wont send it back to me as the user (either as an in-line image) or as a attachment to download.

5

Hi @nathanmcwhorter , Currently this is not possible but this is something our team will be building soon and is in our next set of priorities. Thanks for raising this issue.

6

Hey,
I’ve faced same issue (when using VisualizationTools) and managed to find some workaround with setting up mounts for static files:

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.playground import Playground
from agno.tools.visualization import VisualizationTools
from pathlib import Path
from fastapi.staticfiles import StaticFiles

agent = Agent(
    model=OpenAIChat(id="gpt-4o"),
    tools=[VisualizationTools(output_dir="business_charts")],
    instructions=[
        "You are a data visualization expert and business analyst.",
        "When you create a chart, respond with a markdown image linking to the absolute URL: "
        "'http://localhost:7777/assets/<FILENAME>'. Example: ![chart](http://localhost:7777/assets/bar_chart_1.png)",
        "Always provide meaningful titles, axis labels, and context.",
        "Suggest insights based on the data visualized.",
        "Format data appropriately for each chart type.",
    ],
    show_tool_calls=True,
    markdown=True,
)
    
 
playground = Playground(agents=[agent])
app = playground.get_app()

# Mount the folder where VisualizationTools saves images
charts_dir = Path("business_charts")
charts_dir.mkdir(parents=True, exist_ok=True)
app.mount("/assets", StaticFiles(directory=str(charts_dir), html=False), name="assets")

if __name__ == "__main__":
    playground.serve("sample:app", reload=True)

and this is how it looks like on the playground:

image
image1383×812 46.6 KB

It’s is not something i feel comfortable using on a production grade agent,
@monalisha is there a better solution for this in latest versions?