Conversation Simulator

What is the Conversation Simulator?

The Conversation Simulator is a Galtea feature for testing any AI system that interacts with users through multi-turn conversations — from customer-facing chatbots to autonomous AI agents that execute tasks, call tools, and make decisions. It programmatically generates realistic user messages to simulate dialogues, helping you evaluate your AI’s ability to handle complex interactions, maintain context, and achieve user goals. Each simulation is guided by a Behavior Test Case containing a user persona, goal, scenario, and stopping criteria. You can run simulations directly via simulator.simulate() or as part of the specification-driven evaluations.run() workflow, which handles simulation and evaluation together.

Typical Uses

Dialogue flow testing — Verify your AI produces coherent, natural conversations across multiple turns
Role adherence — Confirm your AI stays in character and follows its assigned role
Task completion — Test whether your AI guides users to successfully complete their goals
Agent workflow validation — Evaluate whether your AI agent calls the right tools, follows the correct steps, and reaches the expected outcome
Robustness — Evaluate how your AI handles unexpected, off-topic, or challenging user inputs

Communication Strategies

The simulator supports two communication strategies that control how the simulated user messages are generated:

Written (default) — Concise, text-based messages mimicking real chat or messaging interactions
Spoken — Natural speech patterns with filler words, hesitations, and a conversational tone

The strategy is set on the Test Case (via CSV upload or SDK) and is passed to the simulator automatically.

Simulation Result Structure

The simulate() method returns a SimulationResult object containing the complete conversation history and metadata:

Field	Type	Description
`session_id`	`str`	The identifier for the simulation session.
`total_turns`	`int`	Total number of conversation turns in the simulation.
`messages`	`List[ConversationMessage]`	The full message history. Each message has `role`, `content`, and optional `retrieval_context` and `metadata`.
`finished`	`bool`	Whether the simulation ended naturally (`True`) or was stopped (`False`).
`stopping_reason`	`Optional[str]`	If stopped, the reason why the simulation ended.
`metadata`	`Optional[Dict]`	Additional simulation metadata (when `include_metadata=True`).

Example Output

{
  "session_id": "abc123",
  "total_turns": 5,
  "messages": [
    {"role": "user", "content": "I need to book a flight."},
    {"role": "assistant", "content": "Where are you flying to?"},
    {"role": "user", "content": "New York, next Tuesday."},
    {"role": "assistant", "content": "I found several options for SFO to JFK next Tuesday..."}
  ],
  "finished": true,
  "stopping_reason": "goal_achieved",
  "metadata": null
}

SDK Integration