Behavior Tests

Evaluate multi-turn dialogue interactions using conversation simulation with simulated users

What are Behavior Tests?

Behavior tests in Galtea are designed to evaluate your product through multi-turn dialogue interactions — whether it’s a conversational chatbot, a task-oriented assistant, or an autonomous AI agent. These tests use the Conversation Simulator to create realistic conversations with simulated users that have specific goals, personalities, and behaviors.

Unlike Accuracy Tests that focus on single-turn question-answer pairs, or Security & Safety Tests that test security vulnerabilities, Behavior tests evaluate how well your AI product can:

• Maintain context across multiple conversation turns
• Guide users toward successful task completion
• Handle unexpected but realistic user inputs
• Stay in character and follow conversation guidelines
• Execute multi-step workflows and manage complex dialogue flows

Creating Behavior Tests

You can create behavior tests in Galtea through two methods:

Behavior Generator

1. Define Your Product

   Start by defining your product in Galtea. These detailed product properties will be used as the foundation for generating realistic behavior scenarios.

2. Configure the test

   Select Behavior as the test type and Generated as the test origin

   The test creation process can be done via the SDK or the Galtea dashboard

3. Generate Behavior Scenarios

   Galtea will automatically analyze your product definition to create tailored conversation scenarios, complete with:

   • Realistic user personas based on your target audience
   • Goals aligned with your product’s use cases
   • Natural conversation flows that test key features

Upload Your Own Behavior Tests

1. Create your behavior test file

   Prepare a CSV file following the behavior test structure shown below

2. Configure the test

   Select “Behavior” as the test type and “Uploaded” as the test origin

   The test creation process can be done via the SDK or the Galtea dashboard

3. Upload your behavior test file

   Select the file you created and upload it to Galtea

Conversation Flow Categories

Behavior tests can cover various types of conversational interactions:

Task Completion

Scenarios where the simulated user has a specific goal to accomplish, such as booking a service, making a purchase, or getting support for an issue.

Information Seeking

Scenarios focused on retrieving specific information or answers, testing the AI’s knowledge and ability to provide helpful responses.

Customer Support

Scenarios that simulate customer service interactions, including complaints, troubleshooting, and problem resolution.

Sales and Consultation

Scenarios where the user is exploring options, comparing products, or seeking advice before making decisions.

Complex Multi-Step Processes

Scenarios that involve multiple stages or require gathering various pieces of information over several turns.

Edge Cases and Challenges

Scenarios that test how the AI handles difficult, unclear, or unexpected user behavior while maintaining conversation quality.

Example Behavior Tests and File Format

Here are examples of behavior test content and structure:

Travel Booking Scenarios

Table View

Goal	User Persona	Input	Stopping Criterias	Max Iterations	Scenario
Book a one-way flight from SFO to JFK for next Tuesday	A busy professional who is direct and values efficiency	I need to book a flight	The user has confirmed the flight booking|The chatbot indicates it cannot fulfill the request	10	Flight booking scenario
Find the cheapest round-trip flight to Europe in summer	A budget-conscious student who asks many questions	Hi, I’m looking for cheap flights to Europe	Flight is booked|User decides not to book|Maximum budget is exceeded	15	Budget travel scenario
Change an existing flight reservation	A frustrated customer whose plans have changed	I need to change my flight immediately	Reservation is successfully modified|Customer is transferred to agent	8	Flight modification scenario

Raw CSV

goal,user_persona,input,stopping_criterias,max_iterations,scenario
"Book a one-way flight from SFO to JFK for next Tuesday","A busy professional who is direct and values efficiency","I need to book a flight","The user has confirmed the flight booking|The chatbot indicates it cannot fulfill the request",10,"Flight booking scenario"
"Find the cheapest round-trip flight to Europe in summer","A budget-conscious student who asks many questions","Hi, I'm looking for cheap flights to Europe","Flight is booked|User decides not to book|Maximum budget is exceeded",15,"Budget travel scenario"
"Change an existing flight reservation","A frustrated customer whose plans have changed","I need to change my flight immediately","Reservation is successfully modified|Customer is transferred to agent",8,"Flight modification scenario"

Customer Support Scenarios

Table View

Goal	User Persona	Input	Stopping Criterias	Max Iterations	Scenario
Get help with a defective product	An upset customer who received a broken item	My product arrived broken and I’m very disappointed	Issue is resolved|Customer requests supervisor|Refund is processed	12	Product defect resolution
Understand how to use a new feature	A curious but non-technical user	I heard about a new feature but don’t know how to use it	User successfully uses the feature|User gives up|Technical support is escalated	10	Feature education scenario
Cancel a subscription	A polite customer who wants to downgrade service	I’d like to cancel my subscription please	Subscription is cancelled|Alternative plan is accepted|Retention offer is declined	8	Subscription cancellation

Raw CSV

goal,user_persona,input,stopping_criterias,max_iterations,scenario
"Get help with a defective product","An upset customer who received a broken item","My product arrived broken and I'm very disappointed","Issue is resolved|Customer requests supervisor|Refund is processed",12,"Product defect resolution"
"Understand how to use a new feature","A curious but non-technical user","I heard about a new feature but don't know how to use it","User successfully uses the feature|User gives up|Technical support is escalated",10,"Feature education scenario"
"Cancel a subscription","A polite customer who wants to downgrade service","I'd like to cancel my subscription please","Subscription is cancelled|Alternative plan is accepted|Retention offer is declined",8,"Subscription cancellation"

Sales and Consultation Scenarios

Table View

Goal	User Persona	Input	Stopping Criterias	Max Iterations	Scenario
Find the right product for specific needs	A thorough researcher who compares many options	I’m looking for a solution but need help choosing	Product recommendation is accepted|User requests human consultation|User decides to research more	15	Product recommendation scenario
Get pricing information for enterprise solution	A business decision maker focused on ROI	What are your enterprise pricing options?	Quote is requested|Meeting is scheduled|User indicates budget constraints	10	Enterprise sales scenario
Compare different service tiers	An analytical customer who wants detailed comparisons	Can you help me understand the differences between your plans?	Plan is selected|User requests trial|User needs more time to decide	12	Service comparison scenario

Raw CSV

goal,user_persona,input,stopping_criterias,max_iterations,scenario
"Find the right product for specific needs","A thorough researcher who compares many options","I'm looking for a solution but need help choosing","Product recommendation is accepted|User requests human consultation|User decides to research more",15,"Product recommendation scenario"
"Get pricing information for enterprise solution","A business decision maker focused on ROI","What are your enterprise pricing options?","Quote is requested|Meeting is scheduled|User indicates budget constraints",10,"Enterprise sales scenario"
"Compare different service tiers","An analytical customer who wants detailed comparisons","Can you help me understand the differences between your plans?","Plan is selected|User requests trial|User needs more time to decide",12,"Service comparison scenario"

This structure is required by Galtea to automatically generate the test cases for your behavior tests as tests within the platform. If this format is not provided, you can still manually create the test cases.

For automatic processing, the file format must be CSV.

The examples provided above are simplified demonstrations. In actual CSV files, behavior tests can be much more detailed and the number of test cases (rows) can be significantly higher to provide comprehensive conversation testing coverage.

Structure of Behavior Tests

Behavior tests have a specific structure designed to enable realistic conversation simulation:

goal · Text · required

The overall objective the simulated user is trying to achieve during the conversation. Example: “Book a one-way flight from San Francisco (SFO) to New York (JFK) for next Tuesday”

user_persona · Text · required

The personality and communication style of the simulated user that will be simulated throughout the conversation. Example: “A busy professional who is direct and values efficiency. They prefer to get things done quickly without much small talk.”

input · Text

The first message the simulated user sends to start the conversation. If not provided, the system will generate an appropriate opening based on the goal and persona. Accepts a plain string or a JSON object for structured inputs (e.g. {"user_message": "I need to book a flight", "channel": "web"}). When using structured JSON, the user_message field becomes the message content and additional fields are accessible via {{ input.field_name }} in endpoint templates and via messages[0].metadata in the SDK. Example: “Hello, I need to book a flight.”

stopping_criterias · Text

A delimited string of conditions that, if met, will cause the conversation simulation to end. Use ; or | as delimiters to separate multiple criteria. Example: “The user has confirmed the flight booking|The chatbot indicates it cannot fulfill the request;User expresses satisfaction with the service”

max_iterations · Number

The maximum number of conversation turns before the simulation automatically ends. This prevents infinite loops and controls test duration. Example: 10

scenario · Text

A brief description of the scenario for documentation and organizational purposes. This helps categorize and manage different types of conversation tests. Example: “Flight booking scenario for business travelers”

Using Behavior Tests with the Conversation Simulator

Behavior tests are specifically designed to work with Galtea’s Conversation Simulator. When you run evaluations using behavior tests, the system will:

1. Initialize the conversation using the input and user_persona
2. Generate realistic user responses based on the goal and conversation history
3. Continue the dialogue until one of the stopping_criterias is met or max_iterations is reached
4. Evaluate the conversation using your selected metrics to assess performance

For detailed implementation examples, see the Conversation Simulator Tutorial.

Data Catalog

When generating behavior tests, you can optionally upload a Data Catalog — a file containing reference data from your product’s domain. This helps the generator produce scenarios grounded in realistic values rather than fully fictional ones.

The data catalog does not need to contain real production data. Synthetic samples, anonymized records, or representative mock data all work equally well — what matters is that the values reflect the shape and variety of your domain.

How it works

Values from the data catalog are incorporated into the scenario field of each generated test case. User personas remain fully synthetic and are never populated from the catalog.

For example, if your product is a banking assistant, a data catalog might include sample account types, transaction descriptions, and card IDs. The generator would then reference those values in scenarios like “User asks about a pending charge of €47.80 at REST LA BARRACA on card C-001” instead of inventing generic placeholders.

Supported formats

CSV, JSON, JSONL, XML, YAML, and TXT files up to 50 MB.

What to include

• Domain entities your users would reference (product names, plan tiers, category labels)
• Sample identifiers (order IDs, ticket numbers, account codes)
• Representative amounts, dates, or statuses
• Any reference data that makes scenarios feel grounded in your specific domain

What to avoid

• Actual customer PII or production credentials
• Internal system IDs or backend states that end users would never see
• Extremely large datasets — the generator uses up to ~50,000 characters of content

Best Practices for Manually Creating Behavior Tests

Goal Definition

• Make goals specific and measurable
• Include realistic constraints (time, budget, preferences)
• Vary complexity across different scenarios
• Consider both successful and unsuccessful outcomes

User Persona Creation

• Include personality traits that affect communication style
• Specify technical knowledge level and domain expertise
• Define emotional states and motivations
• Consider cultural and contextual factors

Stopping Criteria Design

• Include both positive outcomes (goal achieved) and negative outcomes (failure, frustration)
• Account for edge cases where the conversation might stall
• Use multiple criteria connected with ”|” or ”;” to cover various scenarios
• Make criteria specific enough to be clearly identifiable

Scenario Coverage

• Test common user journeys and edge cases
• Include scenarios with different complexity levels
• Cover various user types and use cases
• Test both cooperative and challenging user behaviors

Related

Input Schema

Define structured, multi-field inputs for generated test cases.

Simulating Conversations

Tutorial on running multi-turn conversation simulations with the SDK.

Conversation Simulator

Learn how the Conversation Simulator creates realistic synthetic dialogues.