- Galtea calls your agent. You tell Galtea where your agent lives. Galtea sends the test messages and records the answers. No code on your side.
- Your code calls your agent. You run a short Python script. The SDK sends the test messages to your agent function and sends the answers to Galtea.
- You send the traces your agent already produces. Galtea receives the real traffic of your agent and evaluates it, or adds the internal steps of your agent to a test run.
- Use any language or tool. The REST API, the CLI, and an Agent Skill for AI coding assistants.
Which option should I pick?
At a glance
“Spans only” means the option adds the internal steps of your agent (tool calls, retrieval, model calls) to the test runs of another option. It does not run the tests by itself.
1. Galtea calls your agent (no code)
Galtea sends each test message to your agent, reads the answer, and evaluates it. This is called Direct Inference. You can start a run from the Galtea Dashboard, the SDK (galtea.evaluations.run() with no agent function), or the CLI (galtea evaluations create-from-version), and watch it in the Dashboard. It also works for multi-turn conversations: Galtea simulates the user and talks to your agent turn by turn.
Endpoint Connection
For: any agent with an HTTP API, such as a chat endpoint, an OpenAI-compatible endpoint, or your own backend. You give Galtea the URL, the authentication, the shape of the request body, and where to find the answer in the response. The fastest path: paste acURL command of a real request and Galtea fills in the form for you. Optional initialization and finalization calls let Galtea open and close a session on your side.
Run tests from the Dashboard
Step-by-step guide: create the connection, attach it to a version, run an evaluation.
Endpoint Connection reference
All properties, the input template, and the response mapping.
Phone Connection
For: voice agents that answer a phone number. You give Galtea the phone number of your agent. Galtea dials it and holds a real voice call for each test case. It tests your full telephony stack, as a real caller does.Phone Connection
Set up the number and the call options.
WebRTC Connection
For: voice agents hosted on Pipecat Cloud. WebRTC is a standard for real-time audio over the internet. You give Galtea a Pipecat Cloud public API key and the agent name. Galtea opens a live voice session with your agent for each test case.WebRTC Connection
Set up the Pipecat Cloud key and the agent name.
SLNG.ai
For: voice agents hosted on SLNG.ai. Galtea reaches the text layer of your SLNG agent (its prompt and guardrails) through an Endpoint Connection. Nothing runs on your side.SLNG.ai integration
Set it up by hand or with a coding assistant.
2. Your code calls your agent (Python SDK)
Python SDK
For: agents that are not deployed yet, agents you can call only from inside your network, and tests that run in CI. You write one Python function that calls your agent. The SDK gives it each test message and sends the answers to Galtea for scoring. One call,galtea.evaluations.run(), runs a full evaluation. The SDK also simulates multi-turn conversations with your agent.
Quickstart
Install the SDK and run your first evaluation.
Evaluations from Specs
The recommended workflow with
evaluations.run().Simulate conversations
Test multi-turn behavior with simulated users.
GitHub Actions
Run the evaluation on every push or pull request.
@traced decorator. See Tracing Agent Operations.
Log production traffic with the SDK
For: sending real user conversations to Galtea from your own application code. Your application calls the SDK after each answer. You can keep your own conversation IDs. Galtea stores the conversation as a production session. If you create a Monitor for the product, it scores a sample of these sessions.Monitor Production Responses
Log and evaluate production conversations with the SDK.
3. Send the traces your agent already produces
OpenTelemetry
For: agents in any language that already emit OpenTelemetry (OTel) traces. OTel is the open standard for traces. Point your OTel exporter or Collector at the Galtea endpoint, add your Galtea API key, and name the product and version in resource attributes. Each span also needs a conversation ID, such assession.id. If your agent does not set one, add it in code. Galtea turns the real traffic into production sessions. If you create a Monitor for the product, it scores a sample of these sessions. The same endpoint also adds spans to Direct Inference test runs.
Send OTel traces to Galtea
Endpoint, authentication, and exporter setup.
Monitor real user traffic
Turn production traces into sessions that Monitors evaluate.
Langfuse
For: Python agents already instrumented with Langfuse. Change one import and pass the Galtea trace ID to your agent. During Direct Inference, Galtea sends this ID in theX-Galtea-Inference-Id header of each turn call to an Endpoint Connection. The SDK generate() and simulate() methods set it for you. The spans of those calls then go to both Langfuse and Galtea, and show in the Galtea test run. Spans with no Galtea trace ID go only to Langfuse. Nothing changes in Langfuse.
The live integration does not create production sessions. To monitor production traffic, use OpenTelemetry or log it with the SDK. To evaluate past conversations, import a trace export.
Langfuse integration
Install, swap the import, and link your traces.
Import a trace export
For: conversations you already traced in Langfuse or LangSmith. Upload the export file in the Dashboard: open a version, choose New Session, then Import traces. Each conversation becomes a session that you can evaluate at once. In the Dashboard, the Production or Development switch of the sidebar decides the session type. The same import is available asPOST /sessions/import and galtea sessions import, which make production sessions by default.
Import Sessions from a Trace Export
The supported formats and how each one maps to sessions.
4. Use any language or tool
REST API and CLI
For: teams that do not use Python, and scripts or pipelines. The Dashboard and the SDK use the Galtea REST API. The API reference lists every public operation, and thegaltea CLI (command-line tool) gives you each one as a terminal command.
REST API
Every endpoint, parameter, and response.
CLI
Install the
galtea command and log in.Agent Skill
For: teams that use an AI coding assistant such as Claude Code, Cursor, or Windsurf. Install the Galtea Agent Skill. Your assistant then knows how to use Galtea: it can create products, connect your agent, run evaluations, and read the results for you.Agent Skill
Install the skill in your coding assistant.
Other integrations
Claude Code Tracing
For: teams that want to evaluate their own Claude Code sessions. A Claude Code plugin sends each conversation to Galtea as a production session.Claude Code Tracing
Install the
galtea-tracing plugin.