Create dataset

Create a new dataset. See Datasets.

File uploads. The uri, groundTruthUri, and dataCatalogUri fields of TestInput each expect a presigned GET URL (S3 X-Amz-* or Azure Blob SAS) for a file that was already uploaded via the storage PUT-presign endpoint. The server strips the signing query and persists the underlying storage URI (s3://<bucket>/<key> for S3, or the query-less blob URL for Azure) on the created dataset. A file URI the API returned under STORAGE_PUBLIC_BASE_URL is accepted too, and stored in that same canonical form.

Providing uri switches the request into “custom dataset file” mode: generation is skipped, the dataset is created in SUCCESS state, and credit checks / maxTestCases / generator-specific validations are bypassed.

Sending empty: true creates the dataset with no test cases, in SUCCESS state, with no generation and no credit check. Add test cases afterwards with POST /testCases. It cannot be combined with uri, groundTruthUri, dataCatalogUri, sourceTestId, or any generation field (maxTestCases, strategies, variants, customVariantDescription, fewShot, customUserPersona, maxIterations, endpointConnectionId).

specificationIds links the dataset to each listed specification of its product. Every one must be a POLICY or CAPABILITY specification with the same testType, and generation reads the first one. The deprecated specificationId reads as a one-item specificationIds; a body with both answers 400 unless they name the same specification. To change the links later, use PATCH /tests/{id} or specifications link-datasets.

POST /tests

Create dataset

cURL

curl --request POST \
  --url https://api.galtea.ai/tests \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "productId": "prod_123",
  "name": "Quality Test",
  "type": "QUALITY",
  "specificationIds": [
    "spec_123"
  ],
  "specificationId": "spec_123",
  "groundTruthUri": "https://my-bucket.s3.amazonaws.com/tests/ground-truth.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...",
  "uri": "https://my-bucket.s3.amazonaws.com/tests/test.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...",
  "empty": false,
  "fewShot": "Q: What is 2+2? A: 4",
  "languageCode": "es-MX",
  "backgroundNoiseProfile": "street",
  "backgroundNoiseLevel": "medium",
  "variants": [
    "rag"
  ],
  "customVariantDescription": "Custom test variant",
  "strategies": [
    "original"
  ],
  "customUserPersona": "Business analyst",
  "maxTestCases": 100,
  "maxIterations": 10,
  "models": [
    "gpt-4"
  ],
  "sourceTestId": "test_123",
  "dataCatalogUri": "https://my-bucket.s3.amazonaws.com/tests/data-catalog.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=...",
  "endpointConnectionId": "endpoint_connection_123",
  "metadata": {
    "key": "value"
  }
}
'

201

{
  "id": "test_123",
  "productId": "prod_123",
  "userId": "user_123",
  "name": "Quality Test",
  "type": "QUALITY",
  "models": [
    "gpt-4",
    "claude-3"
  ],
  "groundTruthUri": "https://files.galtea.ai/tests/test_123/ground-truth.csv",
  "uri": "https://files.galtea.ai/tests/test_123/test.csv",
  "error": "<string>",
  "status": "SUCCESS",
  "fewShot": "Example few-shot learning data",
  "languageCode": "es-MX",
  "backgroundNoiseProfile": "street",
  "backgroundNoiseLevel": "medium",
  "variants": [
    "rag",
    "summarization"
  ],
  "customVariantDescription": "Custom variant description",
  "strategies": [
    "original"
  ],
  "customUserPersona": "Business analyst",
  "maxTestCases": 100,
  "sourceTestId": "<string>",
  "dataCatalogUri": "https://files.galtea.ai/tests/test_123/data-catalog.json",
  "metadata": {
    "key": "value"
  },
  "specificationIds": [
    "spec_123"
  ],
  "specificationId": "spec_123",
  "isExtendable": true,
  "createdAt": "2023-11-07T05:31:56Z",
  "deletedAt": "2023-11-07T05:31:56Z"
}

Authorizations

Authorization · string · header · required

API key authorization. Pass your API key in the Authorization header as a Bearer token. Both new (gsk_*) and legacy (gsk-) API keys are accepted, e.g. Authorization: Bearer gsk_... or Authorization: Bearer gsk-....

Headers

Prefer · string

Send return=minimal to get no body back from a write and skip reading the resource: a create answers 201 with only Location, and an update answers 204.

Example:

"return=minimal"

Body

application/json

productId · string | null · required

Product ID. Null creates a dataset of no product.

Example:

"prod_123"

name · string · required

Test name

Example:

"Quality Test"

type · enum<string> | null

Dataset type. Required unless specificationIds are sent; then the dataset takes the specifications' test type.

Available options: QUALITY, RED_TEAMING, SCENARIOS, null

Example:

"QUALITY"

specificationIds · string[] | null

Specifications of the product to link the dataset to. They must share one testType, which the dataset takes. Generation reads the first one.

Example:

["spec_123"]

specificationId · string | null · deprecated

Deprecated, send specificationIds. Read as a one-item specificationIds; sending both answers 400 unless they name the same specification.

Example:

"spec_123"

groundTruthUri · string | null

Presigned GET URL (S3 X-Amz-* query or Azure Blob SAS) for a ground-truth file previously uploaded via the storage PUT-presign endpoint. The server strips the signing query and persists the underlying storage URI. Cannot be combined with sourceTestId.

Example:

"https://my-bucket.s3.amazonaws.com/tests/ground-truth.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=..."

uri · string | null

Presigned GET URL for a fully prepared custom test file previously uploaded via the storage PUT-presign endpoint. When provided, no test-case generation runs: the file is ingested as-is, the test is created in SUCCESS state, and maxTestCases, generator-related fields, and credit checks are skipped. Cannot be combined with sourceTestId or dataCatalogUri. Must be non-empty.

Example:

"https://my-bucket.s3.amazonaws.com/tests/test.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=..."

empty · boolean | null · default: false

Sending empty: true creates the dataset with no test cases, in SUCCESS state, with no generation and no credit check. Add test cases afterwards with POST /testCases. It cannot be combined with uri, groundTruthUri, dataCatalogUri, sourceTestId, or any generation field (maxTestCases, strategies, variants, customVariantDescription, fewShot, customUserPersona, maxIterations, endpointConnectionId). Not supported on POST /tests/batch.

fewShot · string | null

Optional few-shot examples used to guide test-case generation for QUALITY tests.

Example:

"Q: What is 2+2? A: 4"

languageCode · string | null

BCP-47 language tag (e.g. en, es-MX). The region subtag selects the regional variety in simulated user messages and voice synthesis; generated test text keeps the base language. The value is stored as the full tag. A bare code for ca, de, es, eu, fr, gl, it, nl or pt defaults to the European variety of that language, in simulated user messages only (pt -> pt-PT); an explicit region always overrides it, and neither voice synthesis nor the stored value is affected.

Example:

"es-MX"

backgroundNoiseProfile · enum<string> | null

Background noise mixed into the simulated caller audio during a voice test. Clip-backed (office, street, car) or synthetic (white, pink). Null means off; set together with backgroundNoiseLevel.

Available options: office, street, car, white, pink, null

Example:

"street"

backgroundNoiseLevel · enum<string> | null

How loud the background noise is relative to the caller speech. Null means off; set together with backgroundNoiseProfile.

Available options: light, medium, heavy, null

Example:

"medium"

variants · string[] | null

Test variants. QUALITY: rag, entity_extraction, summarization, classification, translation, correction, other. RED_TEAMING: data_leakage, financial_attacks, illegal_activities, misuse, toxicity, custom.

Example:

["rag"]

customVariantDescription · string | null

Custom variant description

Example:

"Custom test variant"

strategies · string[] | null

Generation strategies. "original" is the default for RED_TEAMING. At least one strategy is required for SCENARIOS tests.

Example:

["original"]

customUserPersona · string | null

Example:

"Business analyst"

maxTestCases · integer | null

Example:

100

maxIterations · integer | null

SCENARIOS only. Maximum number of conversation turns per generated scenario when scenario generation is executed. If omitted, the scenario-generator default is used. If uri is provided, no scenario generation runs, so this field is ignored at test-creation time. When generation runs, the value is forwarded to the scenario-generator and persisted on each generated test case. Ignored for QUALITY and RED_TEAMING tests.

Required range: x >= 1

Example:

10

models · string[] | null

Example:

["gpt-4"]

sourceTestId · string | null

Source QUALITY test ID whose approved test cases seed a SCENARIOS test. Only valid when type = SCENARIOS. Cannot be combined with uri or groundTruthUri.

Example:

"test_123"

dataCatalogUri · string | null

Presigned GET URL for a data-catalog file previously uploaded via the storage PUT-presign endpoint. Only valid when type = SCENARIOS. Cannot be combined with uri.

Example:

"https://my-bucket.s3.amazonaws.com/tests/data-catalog.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=..."

endpointConnectionId · string | null

Endpoint connection whose input schema the generated test cases follow. Ignored when uri is provided.

Example:

"endpoint_connection_123"

metadata · object | null

Arbitrary key-value metadata

Example:

{ "key": "value" }

Response

201

application/json

Dataset created successfully. On Prefer: return=minimal, or when the read after the write fails, the answer has no body.

id · string · required

Example:

"test_123"

productId · string | null · required

Example:

"prod_123"

userId · string | null · required

Example:

"user_123"

name · string · required

Example:

"Quality Test"

type · enum<string> · required

Available options: QUALITY, RED_TEAMING, SCENARIOS

Example:

"QUALITY"

models · string[] · required

Example:

["gpt-4", "claude-3"]

groundTruthUri · string | null · required

Storage URI of the ground-truth file, derived from the presigned URL supplied at creation. Returned as <STORAGE_PUBLIC_BASE_URL>/<key> when the deployment configures a public file base, otherwise in its stored form. S3: s3://<bucket>/<key>. Azure Blob: blob URL with the SAS query stripped (e.g. https://<account>.blob.core.windows.net/<container>/<path>). Either form resolves through GET /storage/generate-get-presigned-url.

Example:

"https://files.galtea.ai/tests/test_123/ground-truth.csv"

uri · string | null · required

Storage URI of the uploaded custom test file. Same format rules as groundTruthUri.

Example:

"https://files.galtea.ai/tests/test_123/test.csv"

error · string | null · required

status · enum<string> · required

Available options: PENDING, SUCCESS, FAILED, AUGMENTING, EXTENDING, CANCELLED

Example:

"SUCCESS"

fewShot · string | null · required

Optional few-shot examples (input/output pairs) used to guide test-case generation for QUALITY tests.

Example:

"Example few-shot learning data"

languageCode · string | null · required

BCP-47 language tag (e.g. en, es-MX). The region subtag selects the regional variety in simulated user messages and voice synthesis; generated test text keeps the base language. The value is stored as the full tag. A bare code for ca, de, es, eu, fr, gl, it, nl or pt defaults to the European variety of that language, in simulated user messages only (pt -> pt-PT); an explicit region always overrides it, and neither voice synthesis nor the stored value is affected.

Example:

"es-MX"

backgroundNoiseProfile · enum<string> | null · required

Background noise mixed into the simulated caller audio during a voice test. Clip-backed (office, street, car) or synthetic (white, pink). Null means off; set together with backgroundNoiseLevel.

Available options: office, street, car, white, pink, null

Example:

"street"

backgroundNoiseLevel · enum<string> | null · required

How loud the background noise is relative to the caller speech. Null means off; set together with backgroundNoiseProfile.

Available options: light, medium, heavy, null

Example:

"medium"

variants · string[] · required

Test variants. QUALITY: rag, entity_extraction, summarization, classification, translation, correction, other. RED_TEAMING: data_leakage, financial_attacks, illegal_activities, misuse, toxicity, custom.

Example:

["rag", "summarization"]

customVariantDescription · string | null · required

Example:

"Custom variant description"

strategies · string[] · required

Generation strategies. "original" is the default for RED_TEAMING. At least one strategy is required for SCENARIOS tests.

Example:

["original"]

customUserPersona · string | null · required

Example:

"Business analyst"

maxTestCases · integer | null · required

Example:

100

sourceTestId · string | null · required

dataCatalogUri · string | null · required

Storage URI of the data-catalog file (SCENARIOS tests only). Same format rules as groundTruthUri.

Example:

"https://files.galtea.ai/tests/test_123/data-catalog.json"

metadata · any | null · required

Arbitrary metadata, returned as stored. Usually an object, but any JSON value is accepted.

Example:

{ "key": "value" }

specificationIds · string[] · required

The specifications this dataset is linked to, oldest link first.

Example:

["spec_123"]

specificationId · string | null · required · deprecated

Deprecated, read specificationIds. The oldest linked specification, or null when there is none.

Example:

"spec_123"

isExtendable · boolean · required

True when the test can be extended by re-running its original generator (generated tests; QUALITY only when it has a ground-truth document). False for uploaded tests.

Example:

true

createdAt · string<date-time> · required

deletedAt · string<date-time> | null · required