Create Test Case

Create a test case for your test.

Returns

Returns a TestCase object for the given parameters.

Example

test_case = galtea.test_cases.create(
    test_id=test_id,
    input="What is the capital of France?",
    expected_output="Paris",
    context="Geography facts",
    variant="original",
)

Parameters

test_id · string · required

The ID of the test you want to create the test case for.

input · string

The input for the test case. For Accuracy and Safety tests, this is the question or prompt you want to send to your model for evaluation. For Behaviour tests, this is the first user message to use in the conversation (bypassing the simulator for the first turn).

expected_output · string

The expected output for the input given. This is the correct answer or response you want to evaluate against.

expected_tools · list[string]

The expected tools for the input given. This is the list of tools that the model should use to generate the expected output.

context · string

The context for the test case. This is the additional information that can help the model generate the expected output.

variant · string

An optional variant to categorize or label the test case (e.g., “original”, “paraphrased”).

reviewed_by_id · string

The ID of the user who has reviewed and validated this test case.

language · string

Language for the test case. Accepts a BCP-47 language tag: a bare ISO 639-1 code ("es") or a code with a region subtag ("es-MX", "en-GB"). The region subtag selects the regional variety in the simulated user’s messages and in voice synthesis (the accent the simulated caller speaks with); generated test text keeps the base language. The value is stored as the full tag.

gender · string

Persona gender, "MALE" or "FEMALE", for voice evaluations: it selects the male or female synthesized voice for the simulated caller. Stored only for now — voice selection is not active yet. Text evaluations ignore it. When unset, the voice uses the default for the language.

background_noise_profile · string

Background noise mixed into the simulated caller’s audio for voice evaluations run over a phone connection. Ignored for test cases that don’t run over a phone connection.

• Clip-backed: office, street, car
• Synthetic: white, pink

This method creates the test case directly, so it does not inherit the parent test’s background noise — omitting this parameter leaves the test case with no noise. (Test cases generated by Galtea from a ground truth file or a CSV upload do inherit the test’s noise automatically; this only applies to cases created one-by-one through this method.)

Must be set together with background_noise_level — passing only one of the two is rejected by the API.

background_noise_level · string

How loud the background noise is relative to the caller’s speech, as a fixed signal-to-noise ratio per level. One of light, medium, or heavy. Must be set together with background_noise_profile.

user_score · int

User vote for the test case (1 for good quality, -1 for bad quality, 0 for unreviewed).

user_score_reason · string

Reason for the user vote given.

confidence · float

Confidence for the test case.

confidence_reason · string

Reason for the confidence given.

goal · string

The user’s objective for behavior tests. Used when creating BEHAVIOR test cases.

user_persona · string

Description of the user persona used to generate scenario-based test cases.

scenario · string

Description of the scenario context for scenario-based test cases.

stopping_criterias · list[string]

List of criteria to end the conversation in scenario-based simulations.