Create a Custom Test

Learn how to create and upload custom tests using the SDK

While Galtea can automatically generate tests from a knowledge file, you can also create custom Tests by providing your own set of Test Cases.

Creating Tests with the SDK

Tests can be generated automatically by Galtea providing a knowledge file via the sdk or the dashboard.

The Galtea SDK provides methods to create both Quality Tests and Red Teaming Tests.

Upload Existing Test File

If you have a prepared test file in CSV format, you can upload it directly.

from galtea import Galtea
import os

galtea = Galtea(api_key=os.getenv("GALTEA_API_KEY"))

# Upload a pre-existing test file to the Galtea Platform
test = galtea.tests.create(
    name="financial-qa-test",
    type="QUALITY",
    product_id="YOUR_PRODUCT_ID",
    test_file_path="path/to/your/finance_db_sample.csv"
)

print(f"Test created with ID: {test.id}")

The test file must follow the structure specified for:

• Quality Tests: Standard format with input, expected_output, etc.
• Red Teaming Tests: Standard format with adversarial inputs
• Scenarios Tests: Conversation simulator format with goal, user_persona, etc.

If the file is not correctly formatted, test cases will not be created automatically, but you can still add them manually.

New Test Type: Use type="SCENARIOS" to create conversation simulation tests that enable multi-turn dialogue evaluation with synthetic users. See the Conversation Simulator Tutorial for complete examples.

Next Steps

Run Test-Based Evaluations

Run your test against a specific product version.

API Reference

View the complete Test Service API reference.