Optimize metric

Creates an optimized copy of a metric by sending its human-annotated evaluations to the metrics-generator service, which uses the gap between AI and human scores to produce a better judge prompt. Only FULL_PROMPT and PARTIAL_PROMPT metrics owned by an organization can be optimized. Rejected with 400 unless the metric has at least minAnnotations evaluations annotated by a human AND minDisagreements of those disagree with the AI score; both thresholds are returned by GET /metrics/{id}/human-annotation-count (defaults 10 and 5, configurable via the OPTIMIZE_MIN_ANNOTATIONS_HARD / OPTIMIZE_MIN_DISAGREEMENTS_HARD env vars). 30+ annotations recommended for best optimizer signal. Returns a placeholder metric with isBeingOptimized=true. See Metrics.

POST /metrics/{id}/optimize

Optimize metric

cURL

curl --request POST \
  --url https://api.galtea.ai/metrics/{id}/optimize \
  --header 'Authorization: Bearer <token>'

201

{
  "id": "metric_123",
  "metricGroupId": "metric_123",
  "parentMetricId": "metric_122",
  "organizationId": "org_123",
  "userId": "user_123",
  "name": "Accuracy",
  "evaluationParams": [
    "input",
    "actualOutput",
    "expectedOutput"
  ],
  "source": "PARTIAL_PROMPT",
  "judgePrompt": "Evaluate the accuracy of the response",
  "tags": [
    "accuracy",
    "quality"
  ],
  "description": "Measures the accuracy of responses",
  "documentationUrl": "https://docs.example.com/metrics/accuracy",
  "evaluatorModelName": "GPT-4",
  "areEvalParamsTop": true,
  "isBeingOptimized": true,
  "specificationIds": [
    "spec_123"
  ],
  "userGroupIds": [
    "ug_123"
  ],
  "createdAt": "2023-11-07T05:31:56Z",
  "legacyAt": "2023-11-07T05:31:56Z",
  "disabledAt": "2023-11-07T05:31:56Z"
}

Authorizations

Authorization · string · header · required

API key authorization. Pass your API key in the Authorization header as a Bearer token. Both new (gsk_*) and legacy (gsk-) API keys are accepted, e.g. Authorization: Bearer gsk_... or Authorization: Bearer gsk-....

Path Parameters

id · string · required

Source Metric ID

Response

201

application/json

Optimized metric placeholder created successfully

id · string

Example:

"metric_123"

metricGroupId · string · read-only

Identifier shared by every metric in the same revision family. Server-managed — derived from parentMetricId on create (or generated for roots). Cannot be set by the caller.

Example:

"metric_123"

parentMetricId · string | null

Id of the direct parent metric. On create, providing this value turns the new metric into a revision: it joins the parent's family and (if the parent is active) flips the parent to legacy. Omit or null to create a root metric in a fresh group. On responses, this is the recorded parent edge (null for roots).

Example:

"metric_122"

organizationId · string | null

Example:

"org_123"

userId · string | null

Example:

"user_123"

name · string

Example:

"Accuracy"

evaluationParams · string[]

Ordered list of inference-result fields the evaluator needs (e.g. input, actualOutput, expectedOutput, retrievalContext). Determines which data the evaluation engine extracts from each inference result.

Example:

["input", "actualOutput", "expectedOutput"]

source · enum<string> | null

Evaluation method for the metric. FULL_PROMPT is deprecated for creation — POST /metrics rejects it with a 400. Use PARTIAL_PROMPT for new AI Evaluation metrics. The value remains in the enum because existing FULL_PROMPT metrics are still returned by reads and filters.

Available options: SELF_HOSTED, FULL_PROMPT, PARTIAL_PROMPT, HUMAN_EVALUATION, GEVAL, DEEPEVAL, DETERMINISTIC

Example:

"PARTIAL_PROMPT"

judgePrompt · string | null

Example:

"Evaluate the accuracy of the response"

tags · string[]

Example:

["accuracy", "quality"]

description · string | null

Example:

"Measures the accuracy of responses"

documentationUrl · string | null

Example:

"https://docs.example.com/metrics/accuracy"

evaluatorModelName · string | null

Example:

"GPT-4"

areEvalParamsTop · boolean | null

When true, evaluationParams are injected at the top level of the evaluator prompt instead of nested inside the conversation context.

isBeingOptimized · boolean · read-only

Whether the metric is currently being optimized.

specificationIds · string[]

Example:

["spec_123"]

userGroupIds · string[]

Example:

["ug_123"]

createdAt · string<date-time>

legacyAt · string<date-time> | null

disabledAt · string<date-time> | null