> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Generate Inference Result
> Execute an agent and automatically capture traces, latency, and metadata
## Overview
The `generate()` method is the recommended way to execute your agent when you want automatic trace collection, timing, and inference result creation. It handles the complete lifecycle:
1. Initializes trace collection
2. Executes your agent with timing measurement
3. Creates/updates the inference result with all metadata
4. Saves collected traces to the platform
5. Cleans up trace context (prevents memory leaks)
This is the recommended method for production use as it handles all trace lifecycle management automatically.
## Agent Options
The quickest way to get started. Your function receives just the latest user message as a string.
```python theme={"system"} theme={"system"}
def my_agent(user_message: str) -> str:
# In a real scenario, call your model here
return f"Your model output to: {user_message}"
```
Use this when your agent needs the full conversation context. Your function receives the message list in the OpenAI format (`{"role": "...", "content": "..."}`).
```python theme={"system"} theme={"system"}
def my_agent(messages: list[dict]) -> str:
# messages follows the standard chat format:
# [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}, ...]
user_message = messages[-1]["content"]
return f"Your model output to: {user_message}"
```
For full control over input and output — including optional usage tracking, cost tracking, and retrieval context for RAG evaluations.
```python theme={"system"} theme={"system"}
def my_agent(input_data: AgentInput) -> AgentResponse:
user_message = input_data.last_user_message_str()
# Access structured input fields from the first user message's metadata
# (e.g. when test case input is {"user_message": "hello", "chat_type": "support"})
first_msg = input_data.messages[0] if input_data.messages else None
chat_type = first_msg.metadata.get("chat_type") if first_msg and first_msg.metadata else None
# In a real scenario, call your model here
model_output = f"Your model output to: {user_message}"
# Return AgentResponse with optional usage/cost tracking
return AgentResponse(
content=model_output,
usage_info={"input_tokens": 100, "output_tokens": 50},
)
```
All three signatures work with `evaluations.run()`, `inference_results.generate()`, and `simulator.simulate()`. Both sync and async functions are supported. The SDK auto-detects which signature you're using from the type hint on the first parameter.
For the full list of fields available on `AgentInput` (including structured input access via message metadata), see the [AgentInput reference](/sdk/api/agent/input).
## Parameters
The agent function to execute. Supported signatures:
* **Simple:** `(user_message: str) -> str` — receives last user message
* **Chat History:** `(messages: list[dict]) -> str` — receives full chat history (OpenAI format)
* **Structured:** `(input_data: AgentInput) -> AgentResponse` — structured input/output. See the [AgentInput reference](/sdk/api/agent/input) for all available fields including structured input access via message metadata.
Both sync and async functions are supported. Use the structured signature when you need `usage_info`, `cost_info`, or `retrieval_context`.
The session object to associate the inference result with. Obtain via `galtea.sessions.create()` or `galtea.sessions.get()`.
The input to send to the agent. Accepts a plain string or a dict with structured fields (e.g. `{"user_message": "hello", "chat_type": "support"}`). Stored directly in the inference result. If not provided and the session is test-based, the test case input is used automatically. Required for production/monitoring sessions. See [Input Schema](/concepts/product/endpoint-connection-input-schema) for how to define the expected field structure on your endpoint connection.
If `True`, raises a `ValueError` when the agent returns an empty response. Defaults to `False`.
`context_data` is automatically fetched from the session's test case and passed to the agent via `AgentInput.context_data`. No need to provide it explicitly.
## Returns
Returns an `InferenceResult` object with all captured data.
## Example
```python theme={"system"}
# Define your agent function
def my_agent(user_message: str) -> str:
return f"Response to: {user_message}"
inference_result = galtea.inference_results.generate(agent=my_agent, session=session, input="Generate something")
```
## What Gets Captured
The `generate()` method automatically captures and saves:
| Data | Source | Description |
| --------------------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| **Input** | `input` parameter | The user's input (string or structured dict) |
| **Output** | `AgentResponse.content` | The agent's response |
| **Retrieval Context** | `AgentResponse.retrieval_context` | Context used (for RAG). Stored as a `RETRIEVER` trace, not a readable field on the returned inference result. |
| **Latency** | Measured | End-to-end execution time in ms |
| **Usage Info** | `AgentResponse.usage_info` | Token counts (if provided) |
| **Cost Info** | `AgentResponse.cost_info` | Cost data (if provided) |
| **Traces** | `@trace` decorators | All traced operations |
## Providing Usage and Cost Information
Your agent can return usage and cost information in the `AgentResponse`:
```python theme={"system"}
@galtea.trace(name="main", type=TraceType.AGENT)
def my_agent_with_usage(input_data: AgentInput) -> AgentResponse:
# Your agent logic...
return AgentResponse(
content="Response content",
usage_info={
"input_tokens": 150,
"output_tokens": 75,
"cache_read_input_tokens": 50,
},
cost_info={
"cost_per_input_token": 0.00001,
"cost_per_output_token": 0.00003,
"cost_per_cache_read_input_token": 0.000001,
},
)
```
## Comparison with Manual Approach
| Feature | `generate()` | Manual with `set_context()` |
| ---------------------- | ------------ | ------------------------------ |
| Context initialization | Automatic | `set_context()` |
| Agent execution | Automatic | Manual call |
| Timing measurement | Automatic | Manual |
| Inference result | Auto-created | `inference_results.create()` |
| Trace export | Automatic | Automatic on `clear_context()` |
| Memory cleanup | Automatic | `clear_context()` |
| Error handling | Built-in | Manual try/finally |
Use `generate()` for most use cases. Use manual `set_context()`/`clear_context()` only when you need to manage inference result creation separately or have complex multi-step workflows.
## Error Handling
If the agent raises an exception, `generate()` ensures trace context is cleaned up:
```python theme={"system"}
def failing_agent(input_data: AgentInput) -> AgentResponse:
raise ValueError("Agent failed intentionally")
try:
result = galtea_client.inference_results.generate(agent=failing_agent, session=session_error, input="test")
except Exception as e:
# Trace context is automatically cleaned up
print(f"Agent failed: {e}")
```
## Related Methods
* [Create Inference Result](/sdk/api/inference-result/create) - Manual creation
* [Trace Service](/sdk/api/trace/service) - Manual trace management
* [Simulating Conversations](/sdk/tutorials/simulating-conversations) - Multi-turn simulations