Create test

Create a new test. See Tests.

File uploads. The uri, groundTruthUri, and dataCatalogUri fields of TestInput each expect a presigned GET URL (S3 X-Amz-* or Azure Blob SAS) for a file that was already uploaded via the storage PUT-presign endpoint. The server strips the signing query and persists the underlying storage URI (s3://<bucket>/<key> for S3, or the query-less blob URL for Azure) on the created test.

Providing uri switches the request into “custom test file” mode: generation is skipped, the test is created in SUCCESS state, and credit checks / maxTestCases / generator-specific validations are bypassed.

Creation is the only time specificationId can be set directly. To attach an existing test to a specification afterwards, use specifications link-tests.

POST /tests

Create test

cURL

curl --request POST \
  --url https://api.galtea.ai/tests \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "productId": "prod_123",
  "name": "Quality Test",
  "type": "QUALITY",
  "specificationId": "spec_123",
  "groundTruthUri": "https://my-bucket.s3.amazonaws.com/tests/ground-truth.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...",
  "uri": "https://my-bucket.s3.amazonaws.com/tests/test.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...",
  "fewShot": "Q: What is 2+2? A: 4",
  "languageCode": "es-MX",
  "backgroundNoiseProfile": "street",
  "backgroundNoiseLevel": "medium",
  "variants": [
    "rag"
  ],
  "customVariantDescription": "Custom test variant",
  "strategies": [
    "original"
  ],
  "customUserPersona": "Business analyst",
  "maxTestCases": 100,
  "maxIterations": 10,
  "models": [
    "gpt-4"
  ],
  "sourceTestId": "test_123",
  "dataCatalogUri": "https://my-bucket.s3.amazonaws.com/tests/data-catalog.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...",
  "metadata": {
    "key": "value"
  }
}
'

import requests

url = "https://api.galtea.ai/tests"

payload = {
    "productId": "prod_123",
    "name": "Quality Test",
    "type": "QUALITY",
    "specificationId": "spec_123",
    "groundTruthUri": "https://my-bucket.s3.amazonaws.com/tests/ground-truth.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...",
    "uri": "https://my-bucket.s3.amazonaws.com/tests/test.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...",
    "fewShot": "Q: What is 2+2? A: 4",
    "languageCode": "es-MX",
    "backgroundNoiseProfile": "street",
    "backgroundNoiseLevel": "medium",
    "variants": ["rag"],
    "customVariantDescription": "Custom test variant",
    "strategies": ["original"],
    "customUserPersona": "Business analyst",
    "maxTestCases": 100,
    "maxIterations": 10,
    "models": ["gpt-4"],
    "sourceTestId": "test_123",
    "dataCatalogUri": "https://my-bucket.s3.amazonaws.com/tests/data-catalog.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...",
    "metadata": { "key": "value" }
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    productId: 'prod_123',
    name: 'Quality Test',
    type: 'QUALITY',
    specificationId: 'spec_123',
    groundTruthUri: 'https://my-bucket.s3.amazonaws.com/tests/ground-truth.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...',
    uri: 'https://my-bucket.s3.amazonaws.com/tests/test.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...',
    fewShot: 'Q: What is 2+2? A: 4',
    languageCode: 'es-MX',
    backgroundNoiseProfile: 'street',
    backgroundNoiseLevel: 'medium',
    variants: ['rag'],
    customVariantDescription: 'Custom test variant',
    strategies: ['original'],
    customUserPersona: 'Business analyst',
    maxTestCases: 100,
    maxIterations: 10,
    models: ['gpt-4'],
    sourceTestId: 'test_123',
    dataCatalogUri: 'https://my-bucket.s3.amazonaws.com/tests/data-catalog.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...',
    metadata: {key: 'value'}
  })
};

fetch('https://api.galtea.ai/tests', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

<?php

$curl = curl_init();

curl_setopt_array($curl, [
  CURLOPT_URL => "https://api.galtea.ai/tests",
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_ENCODING => "",
  CURLOPT_MAXREDIRS => 10,
  CURLOPT_TIMEOUT => 30,
  CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
  CURLOPT_CUSTOMREQUEST => "POST",
  CURLOPT_POSTFIELDS => json_encode([
    'productId' => 'prod_123',
    'name' => 'Quality Test',
    'type' => 'QUALITY',
    'specificationId' => 'spec_123',
    'groundTruthUri' => 'https://my-bucket.s3.amazonaws.com/tests/ground-truth.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...',
    'uri' => 'https://my-bucket.s3.amazonaws.com/tests/test.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...',
    'fewShot' => 'Q: What is 2+2? A: 4',
    'languageCode' => 'es-MX',
    'backgroundNoiseProfile' => 'street',
    'backgroundNoiseLevel' => 'medium',
    'variants' => [
        'rag'
    ],
    'customVariantDescription' => 'Custom test variant',
    'strategies' => [
        'original'
    ],
    'customUserPersona' => 'Business analyst',
    'maxTestCases' => 100,
    'maxIterations' => 10,
    'models' => [
        'gpt-4'
    ],
    'sourceTestId' => 'test_123',
    'dataCatalogUri' => 'https://my-bucket.s3.amazonaws.com/tests/data-catalog.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...',
    'metadata' => [
        'key' => 'value'
    ]
  ]),
  CURLOPT_HTTPHEADER => [
    "Authorization: Bearer <token>",
    "Content-Type: application/json"
  ],
]);

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
  echo "cURL Error #:" . $err;
} else {
  echo $response;
}

package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.galtea.ai/tests"

	payload := strings.NewReader("{\n  \"productId\": \"prod_123\",\n  \"name\": \"Quality Test\",\n  \"type\": \"QUALITY\",\n  \"specificationId\": \"spec_123\",\n  \"groundTruthUri\": \"https://my-bucket.s3.amazonaws.com/tests/ground-truth.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...\",\n  \"uri\": \"https://my-bucket.s3.amazonaws.com/tests/test.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...\",\n  \"fewShot\": \"Q: What is 2+2? A: 4\",\n  \"languageCode\": \"es-MX\",\n  \"backgroundNoiseProfile\": \"street\",\n  \"backgroundNoiseLevel\": \"medium\",\n  \"variants\": [\n    \"rag\"\n  ],\n  \"customVariantDescription\": \"Custom test variant\",\n  \"strategies\": [\n    \"original\"\n  ],\n  \"customUserPersona\": \"Business analyst\",\n  \"maxTestCases\": 100,\n  \"maxIterations\": 10,\n  \"models\": [\n    \"gpt-4\"\n  ],\n  \"sourceTestId\": \"test_123\",\n  \"dataCatalogUri\": \"https://my-bucket.s3.amazonaws.com/tests/data-catalog.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...\",\n  \"metadata\": {\n    \"key\": \"value\"\n  }\n}")

	req, _ := http.NewRequest("POST", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(string(body))

}

HttpResponse<String> response = Unirest.post("https://api.galtea.ai/tests")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{\n  \"productId\": \"prod_123\",\n  \"name\": \"Quality Test\",\n  \"type\": \"QUALITY\",\n  \"specificationId\": \"spec_123\",\n  \"groundTruthUri\": \"https://my-bucket.s3.amazonaws.com/tests/ground-truth.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...\",\n  \"uri\": \"https://my-bucket.s3.amazonaws.com/tests/test.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...\",\n  \"fewShot\": \"Q: What is 2+2? A: 4\",\n  \"languageCode\": \"es-MX\",\n  \"backgroundNoiseProfile\": \"street\",\n  \"backgroundNoiseLevel\": \"medium\",\n  \"variants\": [\n    \"rag\"\n  ],\n  \"customVariantDescription\": \"Custom test variant\",\n  \"strategies\": [\n    \"original\"\n  ],\n  \"customUserPersona\": \"Business analyst\",\n  \"maxTestCases\": 100,\n  \"maxIterations\": 10,\n  \"models\": [\n    \"gpt-4\"\n  ],\n  \"sourceTestId\": \"test_123\",\n  \"dataCatalogUri\": \"https://my-bucket.s3.amazonaws.com/tests/data-catalog.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...\",\n  \"metadata\": {\n    \"key\": \"value\"\n  }\n}")
  .asString();

require 'uri'
require 'net/http'

url = URI("https://api.galtea.ai/tests")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{\n  \"productId\": \"prod_123\",\n  \"name\": \"Quality Test\",\n  \"type\": \"QUALITY\",\n  \"specificationId\": \"spec_123\",\n  \"groundTruthUri\": \"https://my-bucket.s3.amazonaws.com/tests/ground-truth.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...\",\n  \"uri\": \"https://my-bucket.s3.amazonaws.com/tests/test.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...\",\n  \"fewShot\": \"Q: What is 2+2? A: 4\",\n  \"languageCode\": \"es-MX\",\n  \"backgroundNoiseProfile\": \"street\",\n  \"backgroundNoiseLevel\": \"medium\",\n  \"variants\": [\n    \"rag\"\n  ],\n  \"customVariantDescription\": \"Custom test variant\",\n  \"strategies\": [\n    \"original\"\n  ],\n  \"customUserPersona\": \"Business analyst\",\n  \"maxTestCases\": 100,\n  \"maxIterations\": 10,\n  \"models\": [\n    \"gpt-4\"\n  ],\n  \"sourceTestId\": \"test_123\",\n  \"dataCatalogUri\": \"https://my-bucket.s3.amazonaws.com/tests/data-catalog.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...\",\n  \"metadata\": {\n    \"key\": \"value\"\n  }\n}"

response = http.request(request)
puts response.read_body

201

{
  "id": "test_123",
  "productId": "prod_123",
  "userId": "user_123",
  "name": "Quality Test",
  "type": "QUALITY",
  "groundTruthUri": "s3://my-bucket/tests/test_123/ground-truth.csv",
  "uri": "s3://my-bucket/tests/test_123/test.csv",
  "error": "<string>",
  "status": "SUCCESS",
  "fewShot": "Example few-shot learning data",
  "languageCode": "es-MX",
  "backgroundNoiseProfile": "street",
  "backgroundNoiseLevel": "medium",
  "variants": [
    "rag",
    "summarization"
  ],
  "customVariantDescription": "Custom variant description",
  "strategies": [
    "original"
  ],
  "customUserPersona": "Business analyst",
  "maxTestCases": 100,
  "models": [
    "gpt-4",
    "claude-3"
  ],
  "sourceTestId": "<string>",
  "dataCatalogUri": "s3://my-bucket/tests/test_123/data-catalog.json",
  "metadata": {
    "key": "value"
  },
  "specificationId": "spec_123",
  "isExtendable": true,
  "createdAt": "2023-11-07T05:31:56Z",
  "deletedAt": "2023-11-07T05:31:56Z"
}

{
  "error": "Error type",
  "message": "Error message description"
}

{
  "error": "Error type",
  "message": "Error message description"
}

Authorizations

Authorization · string · header · required

API key authorization. Pass your API key in the Authorization header as a Bearer token. Both new (gsk_*) and legacy (gsk-) API keys are accepted, e.g. Authorization: Bearer gsk_... or Authorization: Bearer gsk-....

Body

application/json

Test data

productId · string · required

Product ID

Example:

"prod_123"

name · string · required

Test name

Example:

"Quality Test"

type · enum<string> · required

Available options: QUALITY, RED_TEAMING, SCENARIOS

Example:

"QUALITY"

specificationId · string

Specification ID (can auto-derive type/variants)

Example:

"spec_123"

groundTruthUri · string

Presigned GET URL (S3 X-Amz-* query or Azure Blob SAS) for a ground-truth file previously uploaded via the storage PUT-presign endpoint. The server strips the signing query and persists the underlying storage URI. Cannot be combined with sourceTestId.

Example:

"https://my-bucket.s3.amazonaws.com/tests/ground-truth.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=..."

uri · string

Presigned GET URL for a fully prepared custom test file previously uploaded via the storage PUT-presign endpoint. When provided, no test-case generation runs: the file is ingested as-is, the test is created in SUCCESS state, and maxTestCases, generator-related fields, and credit checks are skipped. Cannot be combined with sourceTestId or dataCatalogUri. Must be non-empty.

Example:

"https://my-bucket.s3.amazonaws.com/tests/test.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=..."

fewShot · string

Optional few-shot examples used to guide test-case generation for QUALITY tests.

Example:

"Q: What is 2+2? A: 4"

languageCode · string

BCP-47 language tag (e.g. en, es-MX). The region subtag selects the regional variety in simulated user messages and voice synthesis; generated test text keeps the base language. The value is stored as the full tag.

Example:

"es-MX"

backgroundNoiseProfile · enum<string> | null

Background noise mixed into the simulated caller audio during a voice test. Clip-backed (office, street, car) or synthetic (white, pink). Null means off; set together with backgroundNoiseLevel.

Available options: office, street, car, white, pink

Example:

"street"

backgroundNoiseLevel · enum<string> | null

How loud the background noise is relative to the caller speech. Null means off; set together with backgroundNoiseProfile.

Available options: light, medium, heavy

Example:

"medium"

variants · string[]

Test variants. QUALITY: rag, entity_extraction, summarization, classification, translation, correction, other. RED_TEAMING: data_leakage, financial_attacks, illegal_activities, misuse, toxicity, custom.

Example:

["rag"]

customVariantDescription · string

Custom variant description

Example:

"Custom test variant"

strategies · string[]

Generation strategies. "original" is the default for RED_TEAMING. At least one strategy is required for SCENARIOS tests.

Example:

["original"]

customUserPersona · string

Example:

"Business analyst"

maxTestCases · integer

Example:

100

maxIterations · integer

SCENARIOS only. Maximum number of conversation turns per generated scenario when scenario generation is executed. If omitted, the scenario-generator default is used. If uri is provided, no scenario generation runs, so this field is ignored at test-creation time. When generation runs, the value is forwarded to the scenario-generator and persisted on each generated test case. Ignored for QUALITY and RED_TEAMING tests.

Required range: x >= 1

Example:

10

models · string[]

Example:

["gpt-4"]

sourceTestId · string

Source QUALITY test ID whose approved test cases seed a SCENARIOS test. Only valid when type = SCENARIOS. Cannot be combined with uri or groundTruthUri.

Example:

"test_123"

dataCatalogUri · string

Presigned GET URL for a data-catalog file previously uploaded via the storage PUT-presign endpoint. Only valid when type = SCENARIOS. Cannot be combined with uri.

Example:

"https://my-bucket.s3.amazonaws.com/tests/data-catalog.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=..."

metadata · object | null

Arbitrary key-value metadata

Example:

{ "key": "value" }

Response

201

application/json

Test created successfully

id · string

Example:

"test_123"

productId · string | null

Example:

"prod_123"

userId · string | null

Example:

"user_123"

name · string

Example:

"Quality Test"

type · enum<string>

Available options: QUALITY, RED_TEAMING, SCENARIOS

Example:

"QUALITY"

groundTruthUri · string | null

Canonical storage URI of the ground-truth file, derived from the presigned URL supplied at creation. S3: s3://<bucket>/<key>. Azure Blob: blob URL with the SAS query stripped (e.g. https://<account>.blob.core.windows.net/<container>/<path>).

Example:

"s3://my-bucket/tests/test_123/ground-truth.csv"

uri · string | null

Canonical storage URI of the uploaded custom test file. Same format rules as groundTruthUri.

Example:

"s3://my-bucket/tests/test_123/test.csv"

error · string | null

status · enum<string>

Available options: PENDING, SUCCESS, FAILED, AUGMENTING, EXTENDING, CANCELLED

Example:

"SUCCESS"

fewShot · string | null

Optional few-shot examples (input/output pairs) used to guide test-case generation for QUALITY tests.

Example:

"Example few-shot learning data"

languageCode · string | null

BCP-47 language tag (e.g. en, es-MX). The region subtag selects the regional variety in simulated user messages and voice synthesis; generated test text keeps the base language. The value is stored as the full tag.

Example:

"es-MX"

backgroundNoiseProfile · enum<string> | null

Background noise mixed into the simulated caller audio during a voice test. Clip-backed (office, street, car) or synthetic (white, pink). Null means off; set together with backgroundNoiseLevel.

Available options: office, street, car, white, pink

Example:

"street"

backgroundNoiseLevel · enum<string> | null

How loud the background noise is relative to the caller speech. Null means off; set together with backgroundNoiseProfile.

Available options: light, medium, heavy

Example:

"medium"

variants · string[]

Test variants. QUALITY: rag, entity_extraction, summarization, classification, translation, correction, other. RED_TEAMING: data_leakage, financial_attacks, illegal_activities, misuse, toxicity, custom.

Example:

["rag", "summarization"]

customVariantDescription · string | null

Example:

"Custom variant description"

strategies · string[]

Generation strategies. "original" is the default for RED_TEAMING. At least one strategy is required for SCENARIOS tests.

Example:

["original"]

customUserPersona · string | null

Example:

"Business analyst"

maxTestCases · integer | null

Example:

100

models · string[]

Example:

["gpt-4", "claude-3"]

sourceTestId · string | null

dataCatalogUri · string | null

Canonical storage URI of the data-catalog file (SCENARIOS tests only). Same format rules as groundTruthUri.

Example:

"s3://my-bucket/tests/test_123/data-catalog.json"

metadata · object | null

Example:

{ "key": "value" }

specificationId · string | null

Example:

"spec_123"

isExtendable · boolean

True when the test can be extended by re-running its original generator (generated tests; QUALITY only when it has a ground-truth document). False for uploaded tests.

Example:

true

createdAt · string<date-time>

deletedAt · string<date-time> | null