Create metric

Create a new metric. See Metrics.

Accepted values for source on new metrics are SELF_HOSTED, PARTIAL_PROMPT, HUMAN_EVALUATION, and CLASSIFIER (the Classifier judge; 400 where the deployment does not enable it). FULL_PROMPT is deprecated for creation — requests with source=FULL_PROMPT (or that omit source and supply a judgePrompt without evaluationParams, which previously inferred FULL_PROMPT) are rejected with 400. Use PARTIAL_PROMPT instead — it offers the same capabilities with user-configurable evaluation parameters. Existing FULL_PROMPT metrics continue to be listed, read, filtered, and evaluated.

POST /metrics

Create metric

cURL

curl --request POST \
  --url https://api.galtea.ai/metrics \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "Accuracy",
  "source": "PARTIAL_PROMPT",
  "evaluationParams": [
    "input",
    "actual_output"
  ],
  "judgePrompt": "<string>",
  "evaluatorModelName": "gpt-4.1",
  "definition": "<string>",
  "description": "<string>",
  "documentationUrl": "<string>",
  "tags": [
    "<string>"
  ],
  "areEvalParamsTop": true,
  "organizationId": "org_123",
  "parentMetricId": "metric_122",
  "userGroupIds": [
    "ug_123"
  ],
  "specificationIds": [
    "spec_123"
  ],
  "judgeGenerationSettings": {
    "temperature": 0.3,
    "topP": 0.9,
    "maxOutputTokens": 512,
    "reasoningEffort": "low"
  },
  "passThreshold": 0.7,
  "applicabilityScope": "PRODUCTION",
  "applicabilityThreshold": 0.7,
  "classifierQuestion": {
    "instructions": "<string>",
    "criteria": {
      "yes": "<string>",
      "no": "<string>"
    },
    "levels": [
      {
        "description": "<string>",
        "score": 123
      }
    ],
    "doesNotApply": "<string>"
  }
}
'

201

{
  "id": "metric_123",
  "metricGroupId": "metric_123",
  "parentMetricId": "metric_122",
  "organizationId": "org_123",
  "userId": "user_123",
  "name": "Accuracy",
  "evaluationParams": [
    "input",
    "actual_output",
    "expected_output"
  ],
  "source": "PARTIAL_PROMPT",
  "judgePrompt": "Evaluate the accuracy of the response",
  "tags": [
    "accuracy",
    "quality"
  ],
  "description": "Measures the accuracy of responses",
  "documentationUrl": "https://docs.example.com/metrics/accuracy",
  "evaluatorModelName": "gpt-4",
  "areEvalParamsTop": true,
  "isBeingOptimized": true,
  "optimizationStatus": "READY_FOR_REVIEW",
  "judgeGenerationSettings": {
    "temperature": 0.3,
    "topP": 0.9,
    "maxOutputTokens": 512,
    "reasoningEffort": "low"
  },
  "passThreshold": 0.7,
  "applicabilityScope": "PRODUCTION",
  "applicabilityThreshold": 0.7,
  "classifierQuestion": {
    "type": "yes_no",
    "instructions": "<string>",
    "goodAnswer": "yes",
    "criteria": {
      "yes": "<string>",
      "no": "<string>"
    },
    "levels": [
      {
        "description": "<string>",
        "score": 123
      }
    ],
    "doesNotApply": "<string>"
  },
  "optimizationValidation": {
    "used": true,
    "heldOutEvaluationIds": [
      "evaluation_123"
    ],
    "results": {
      "optimization": {
        "rowsScored": 25,
        "rowsTotal": 25,
        "source": {
          "correct": 20,
          "alignment": 0.71
        },
        "candidate": {
          "correct": 20,
          "alignment": 0.71
        }
      },
      "validation": {
        "rowsScored": 25,
        "rowsTotal": 25,
        "source": {
          "correct": 20,
          "alignment": 0.71
        },
        "candidate": {
          "correct": 20,
          "alignment": 0.71
        }
      }
    },
    "rows": [
      {
        "evaluationId": "evaluation_123",
        "humanScore": 1,
        "sourceScore": 0,
        "candidateScore": 1
      }
    ]
  },
  "optimizationRun": {
    "strategies": [
      "ADD_RUBRIC",
      "SIMPLIFY"
    ],
    "maxSteps": 6,
    "stepsUsed": 4
  },
  "specificationIds": [
    "spec_123"
  ],
  "userGroupIds": [
    "ug_123"
  ],
  "createdAt": "2023-11-07T05:31:56Z",
  "legacyAt": "2023-11-07T05:31:56Z",
  "disabledAt": "2023-11-07T05:31:56Z",
  "excludedFromAnalyticsAt": "2023-11-07T05:31:56Z",
  "excludedByUserId": "<string>"
}

Authorizations

Authorization · string · header · required

API key authorization. Pass your API key in the Authorization header as a Bearer token. Both new (gsk_*) and legacy (gsk-) API keys are accepted, e.g. Authorization: Bearer gsk_... or Authorization: Bearer gsk-....

Headers

Prefer · string

Send return=minimal to get no body back from a write and skip reading the resource: a create answers 201 with only Location, and an update answers 204.

Example:

"return=minimal"

Body

application/json

name · string · required

Required string length: 1 - 200

Example:

"Accuracy"

source · enum<string> | null

Available options: SELF_HOSTED, FULL_PROMPT, PARTIAL_PROMPT, HUMAN_EVALUATION, GEVAL, DEEPEVAL, DETERMINISTIC, CLASSIFIER

Example:

"PARTIAL_PROMPT"

evaluationParams · string[] | null

Maximum array length: 20

Required string length: 1 - 64

Example:

["input", "actual_output"]

judgePrompt · string | null

Maximum string length: 20000

evaluatorModelName · string | null

Required for PARTIAL_PROMPT. For CLASSIFIER: The selectable classifier model to run the question under, as listed by GET /evaluatorModels with kind=CLASSIFIER and selectable=true. Omitted or null, the one selectable classifier model is used. When several are selectable, the request is refused with 400 and this field must be sent.

Example:

"gpt-4.1"

definition · string | null

description · string | null

Maximum string length: 5000

documentationUrl · string | null

tags · string[] | null

Maximum array length: 20

Required string length: 1 - 64

areEvalParamsTop · boolean | null

organizationId · string | null

Example:

"org_123"

parentMetricId · string | null

Example:

"metric_122"

userGroupIds · string[] | null

Minimum string length: 1

Example:

["ug_123"]

specificationIds · string[] | null

Minimum string length: 1

Example:

["spec_123"]

judgeGenerationSettings · object | null

The generation settings this metric's judge runs with. Immutable: changing one creates a revision. Only the settings GET /evaluatorModels reports as configurable for the chosen model take effect; a value the provider rejects ends that evaluation as SKIPPED carrying the provider's own message.

Show child attributes

passThreshold · number | null

Score from 0 to 1 at or above which an evaluation of this metric passes. Omitted or null, it defaults to 0.5, or to the parent metric's threshold for a revision.

Required range: 0 <= x <= 1

Example:

0.7

applicabilityScope · enum<string> | null

Which sessions get the applicability check before this metric is judged: PRODUCTION, DEVELOPMENT, ALL or NONE (no check). A session the check finds the metric does not apply to is skipped and no judge call is made. Omitted or null, it defaults to PRODUCTION for a metric the platform scores and NONE for any other. A revision keeps the parent metric's value when the check runs for both. Fixed at create: a change creates a revision. Any value but NONE is refused with a 400 for a metric the platform does not score. A metric that scores traces is never checked, so it takes NONE and refuses any other value. A FULL_PROMPT metric whose judge prompt reads traces but does not list it in evaluationParams is stored as checked, yet the evaluator judges it without the check. The check runs only where the deployment enables it.

Available options: NONE, DEVELOPMENT, PRODUCTION, ALL

Example:

"PRODUCTION"

applicabilityThreshold · number | null

The lowest probability, from 0 to 1, that the metric applies to a session at which it is judged. Below it, the session is skipped. Omitted or null, it defaults to 0.5, or to the parent metric's value for a revision when the check runs for both. Fixed at create: a change creates a revision. Refused with a 400 for a metric the platform does not score, and for a metric that scores traces. The check runs only where the deployment enables it.

Required range: 0 <= x <= 1

Example:

0.7

classifierQuestion · object | null

CLASSIFIER only, and required there: the one question the classifier answers. evaluationParams is derived from its {{ param }} placeholders.

Show child attributes

Response

201

application/json

Metric created successfully. On Prefer: return=minimal, or when the read after the write fails, the answer has no body.

id · string · required

Example:

"metric_123"

metricGroupId · string · required · read-only

Identifier shared by every metric in the same revision family. Server-managed — derived from parentMetricId on create (or generated for roots). Cannot be set by the caller.

Example:

"metric_123"

parentMetricId · string | null · required

Id of the direct parent metric. On create, providing this value turns the new metric into a revision: it joins the parent's family and (if the parent is active) flips the parent to legacy. Omit or null to create a root metric in a fresh group. On responses, this is the recorded parent edge (null for roots).

Example:

"metric_122"

organizationId · string | null · required

Example:

"org_123"

userId · string | null · required

Example:

"user_123"

name · string · required

Example:

"Accuracy"

evaluationParams · string[] · required

Ordered list of trace fields the evaluator needs, written in snake_case (e.g. input, actual_output, expected_output, retrieval_context). Determines which data the evaluation engine extracts from each trace. Full list of accepted values: https://docs.galtea.ai/concepts/metric/evaluation-parameters

Example:

["input", "actual_output", "expected_output"]

source · enum<string> | null · required

Evaluation method for the metric. FULL_PROMPT is deprecated for creation — POST /metrics rejects it with a 400. Use PARTIAL_PROMPT for new AI Evaluation metrics. The value remains in the enum because existing FULL_PROMPT metrics are still returned by reads and filters. CLASSIFIER is the Classifier judge, accepted only where the deployment enables it.

Available options: SELF_HOSTED, FULL_PROMPT, PARTIAL_PROMPT, HUMAN_EVALUATION, GEVAL, DEEPEVAL, DETERMINISTIC, CLASSIFIER

Example:

"PARTIAL_PROMPT"

judgePrompt · string | null · required

Example:

"Evaluate the accuracy of the response"

tags · string[] · required

Example:

["accuracy", "quality"]

description · string | null · required

Example:

"Measures the accuracy of responses"

documentationUrl · string | null · required

Example:

"https://docs.example.com/metrics/accuracy"

evaluatorModelName · string | null · required

Example:

"gpt-4"

areEvalParamsTop · boolean | null · required

When true, evaluationParams are injected at the top level of the evaluator prompt instead of nested inside the conversation context.

isBeingOptimized · boolean · required · read-only

Whether the metric is currently being optimized.

optimizationStatus · enum<string> | null · required · read-only

Where an optimization attempt stands. Null for a metric that did not come from an optimization. A candidate waiting for review is legacy until it is activated.

Available options: OPTIMIZING, READY_FOR_REVIEW, FAILED, NO_IMPROVEMENT, DECLINED, ACTIVATED

Example:

"READY_FOR_REVIEW"

judgeGenerationSettings · object | null · required

The generation settings this metric's judge runs with. Null runs the platform defaults. Immutable: changing one creates a revision.

Show child attributes

passThreshold · number · required

Score from 0 to 1 at or above which an evaluation of this metric passes. Immutable: changing it creates a revision.

Required range: 0 <= x <= 1

Example:

0.7

applicabilityScope · enum<string> · required

Which sessions get the applicability check before this metric is judged. NONE for a metric the platform does not score. A metric that scores traces is never checked, so it takes NONE and refuses any other value. A FULL_PROMPT metric whose judge prompt reads traces but does not list it in evaluationParams is stored as checked, yet the evaluator judges it without the check. Immutable: changing it creates a revision. The check runs only where the deployment enables it.

Available options: NONE, DEVELOPMENT, PRODUCTION, ALL

Example:

"PRODUCTION"

applicabilityThreshold · number | null · required

The lowest probability that the metric applies to a session at which it is judged. Null means the platform default, 0.5. Immutable: changing it creates a revision.

Required range: 0 <= x <= 1

Example:

0.7

classifierQuestion · object · required

The one question of a CLASSIFIER metric; null for every other source.

Show child attributes

optimizationValidation · object | null · required · read-only

How an optimization attempt was validated: the held-out evaluations and, once the optimizer reported, both prompts scored on the search rows and on the held-out rows. Set on an optimization attempt only, and null for any other metric, for an attempt that predates it, and for an attempt copied from another organization.

Show child attributes

optimizationRun · object | null · required · read-only

What an optimization attempt ran: the strategies, the step budget and the steps used. Null while the attempt runs, and when it was not reported: an attempt from before the report existed, one run by an older optimizer, a failed run, an attempt copied from another organization, or a metric that is not an optimization attempt. Null never means a run of 0 steps.

Show child attributes

specificationIds · string[] · required

Specifications linked to this metric. A MEMBER, READER or HUMAN_EVALUATOR caller sees only the specifications of the products it can read.

Example:

["spec_123"]

userGroupIds · string[] · required

Example:

["ug_123"]

createdAt · string<date-time> · required

legacyAt · string<date-time> | null · required · read-only

Earliest non-null of this metric's own legacy date and its evaluator model's unselectableAt, so a model no longer selectable makes every metric linked to it report as legacy even though the metric row itself is untouched. Present only once that date has passed: a scheduled future cutoff reports null, because the metric is still active until then. Unlike disabledAt, this field never carries a future date.

disabledAt · string<date-time> | null · required · read-only

Earliest non-null of this metric's own disabled date and its evaluator model's disabledAt — a disabled model makes every metric linked to it report as disabled even though the metric row itself is untouched.

excludedFromAnalyticsAt · string<date-time> | null · required · read-only

When set, results produced by this metric revision do not feed analytics.

excludedByUserId · string | null · required · read-only

User who last excluded this metric revision from analytics.