Include or exclude a metric revision from analytics

Excluding a revision removes every result produced by it from analytics scores and coverage. It does not delete or change any result.

PATCH /metrics/{id}/analytics-exclusion

Include or exclude a metric revision from analytics

cURL

curl --request PATCH \
  --url https://api.galtea.ai/metrics/{id}/analytics-exclusion \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "excluded": true
}
'

200

{
  "id": "metric_123",
  "metricGroupId": "metric_123",
  "parentMetricId": "metric_122",
  "organizationId": "org_123",
  "userId": "user_123",
  "name": "Accuracy",
  "evaluationParams": [
    "input",
    "actual_output",
    "expected_output"
  ],
  "source": "PARTIAL_PROMPT",
  "judgePrompt": "Evaluate the accuracy of the response",
  "tags": [
    "accuracy",
    "quality"
  ],
  "description": "Measures the accuracy of responses",
  "documentationUrl": "https://docs.example.com/metrics/accuracy",
  "evaluatorModelName": "gpt-4",
  "areEvalParamsTop": true,
  "isBeingOptimized": true,
  "optimizationStatus": "READY_FOR_REVIEW",
  "judgeGenerationSettings": {
    "temperature": 0.3,
    "topP": 0.9,
    "maxOutputTokens": 512,
    "reasoningEffort": "low"
  },
  "passThreshold": 0.7,
  "applicabilityScope": "PRODUCTION",
  "applicabilityThreshold": 0.7,
  "classifierQuestion": {
    "type": "yes_no",
    "instructions": "<string>",
    "goodAnswer": "yes",
    "criteria": {
      "yes": "<string>",
      "no": "<string>"
    },
    "levels": [
      {
        "description": "<string>",
        "score": 123
      }
    ],
    "doesNotApply": "<string>"
  },
  "optimizationValidation": {
    "used": true,
    "heldOutEvaluationIds": [
      "evaluation_123"
    ],
    "results": {
      "optimization": {
        "rowsScored": 25,
        "rowsTotal": 25,
        "source": {
          "correct": 20,
          "alignment": 0.71
        },
        "candidate": {
          "correct": 20,
          "alignment": 0.71
        }
      },
      "validation": {
        "rowsScored": 25,
        "rowsTotal": 25,
        "source": {
          "correct": 20,
          "alignment": 0.71
        },
        "candidate": {
          "correct": 20,
          "alignment": 0.71
        }
      }
    },
    "rows": [
      {
        "evaluationId": "evaluation_123",
        "humanScore": 1,
        "sourceScore": 0,
        "candidateScore": 1
      }
    ]
  },
  "optimizationRun": {
    "strategies": [
      "ADD_RUBRIC",
      "SIMPLIFY"
    ],
    "maxSteps": 6,
    "stepsUsed": 4
  },
  "specificationIds": [
    "spec_123"
  ],
  "userGroupIds": [
    "ug_123"
  ],
  "createdAt": "2023-11-07T05:31:56Z",
  "legacyAt": "2023-11-07T05:31:56Z",
  "disabledAt": "2023-11-07T05:31:56Z",
  "excludedFromAnalyticsAt": "2023-11-07T05:31:56Z",
  "excludedByUserId": "<string>"
}

Authorizations

Authorization · string · header · required

API key authorization. Pass your API key in the Authorization header as a Bearer token. Both new (gsk_*) and legacy (gsk-) API keys are accepted, e.g. Authorization: Bearer gsk_... or Authorization: Bearer gsk-....

Headers

Prefer · string

Send return=minimal to get no body back from a write and skip reading the resource: a create answers 201 with only Location, and an update answers 204.

Example:

"return=minimal"

Path Parameters

id · string · required

Metric revision ID

Minimum string length: 1

Body

application/json

excluded · boolean · required

True to exclude this revision, false to include it again

Response

200

application/json

Updated metric revision. The body is the resource as its GET answers it.

id · string · required

Example:

"metric_123"

metricGroupId · string · required · read-only

Identifier shared by every metric in the same revision family. Server-managed — derived from parentMetricId on create (or generated for roots). Cannot be set by the caller.

Example:

"metric_123"

parentMetricId · string | null · required

Id of the direct parent metric. On create, providing this value turns the new metric into a revision: it joins the parent's family and (if the parent is active) flips the parent to legacy. Omit or null to create a root metric in a fresh group. On responses, this is the recorded parent edge (null for roots).

Example:

"metric_122"

organizationId · string | null · required

Example:

"org_123"

userId · string | null · required

Example:

"user_123"

name · string · required

Example:

"Accuracy"

evaluationParams · string[] · required

Ordered list of trace fields the evaluator needs, written in snake_case (e.g. input, actual_output, expected_output, retrieval_context). Determines which data the evaluation engine extracts from each trace. Full list of accepted values: https://docs.galtea.ai/concepts/metric/evaluation-parameters

Example:

["input", "actual_output", "expected_output"]

source · enum<string> | null · required

Evaluation method for the metric. FULL_PROMPT is deprecated for creation — POST /metrics rejects it with a 400. Use PARTIAL_PROMPT for new AI Evaluation metrics. The value remains in the enum because existing FULL_PROMPT metrics are still returned by reads and filters. CLASSIFIER is the Classifier judge, accepted only where the deployment enables it.

Available options: SELF_HOSTED, FULL_PROMPT, PARTIAL_PROMPT, HUMAN_EVALUATION, GEVAL, DEEPEVAL, DETERMINISTIC, CLASSIFIER

Example:

"PARTIAL_PROMPT"

judgePrompt · string | null · required

Example:

"Evaluate the accuracy of the response"

tags · string[] · required

Example:

["accuracy", "quality"]

description · string | null · required

Example:

"Measures the accuracy of responses"

documentationUrl · string | null · required

Example:

"https://docs.example.com/metrics/accuracy"

evaluatorModelName · string | null · required

Example:

"gpt-4"

areEvalParamsTop · boolean | null · required

When true, evaluationParams are injected at the top level of the evaluator prompt instead of nested inside the conversation context.

isBeingOptimized · boolean · required · read-only

Whether the metric is currently being optimized.

optimizationStatus · enum<string> | null · required · read-only

Where an optimization attempt stands. Null for a metric that did not come from an optimization. A candidate waiting for review is legacy until it is activated.

Available options: OPTIMIZING, READY_FOR_REVIEW, FAILED, NO_IMPROVEMENT, DECLINED, ACTIVATED

Example:

"READY_FOR_REVIEW"

judgeGenerationSettings · object | null · required

The generation settings this metric's judge runs with. Null runs the platform defaults. Immutable: changing one creates a revision.

Show child attributes

passThreshold · number · required

Score from 0 to 1 at or above which an evaluation of this metric passes. Immutable: changing it creates a revision.

Required range: 0 <= x <= 1

Example:

0.7

applicabilityScope · enum<string> · required

Which sessions get the applicability check before this metric is judged. NONE for a metric the platform does not score. A metric that scores traces is never checked, so it takes NONE and refuses any other value. A FULL_PROMPT metric whose judge prompt reads traces but does not list it in evaluationParams is stored as checked, yet the evaluator judges it without the check. Immutable: changing it creates a revision. The check runs only where the deployment enables it.

Available options: NONE, DEVELOPMENT, PRODUCTION, ALL

Example:

"PRODUCTION"

applicabilityThreshold · number | null · required

The lowest probability that the metric applies to a session at which it is judged. Null means the platform default, 0.5. Immutable: changing it creates a revision.

Required range: 0 <= x <= 1

Example:

0.7

classifierQuestion · object · required

The one question of a CLASSIFIER metric; null for every other source.

Show child attributes

optimizationValidation · object | null · required · read-only

How an optimization attempt was validated: the held-out evaluations and, once the optimizer reported, both prompts scored on the search rows and on the held-out rows. Set on an optimization attempt only, and null for any other metric, for an attempt that predates it, and for an attempt copied from another organization.

Show child attributes

optimizationRun · object | null · required · read-only

What an optimization attempt ran: the strategies, the step budget and the steps used. Null while the attempt runs, and when it was not reported: an attempt from before the report existed, one run by an older optimizer, a failed run, an attempt copied from another organization, or a metric that is not an optimization attempt. Null never means a run of 0 steps.

Show child attributes

specificationIds · string[] · required

Specifications linked to this metric. A MEMBER, READER or HUMAN_EVALUATOR caller sees only the specifications of the products it can read.

Example:

["spec_123"]

userGroupIds · string[] · required

Example:

["ug_123"]

createdAt · string<date-time> · required

legacyAt · string<date-time> | null · required · read-only

Earliest non-null of this metric's own legacy date and its evaluator model's unselectableAt, so a model no longer selectable makes every metric linked to it report as legacy even though the metric row itself is untouched. Present only once that date has passed: a scheduled future cutoff reports null, because the metric is still active until then. Unlike disabledAt, this field never carries a future date.

disabledAt · string<date-time> | null · required · read-only

Earliest non-null of this metric's own disabled date and its evaluator model's disabledAt — a disabled model makes every metric linked to it report as disabled even though the metric row itself is untouched.

excludedFromAnalyticsAt · string<date-time> | null · required · read-only

When set, results produced by this metric revision do not feed analytics.

excludedByUserId · string | null · required · read-only

User who last excluded this metric revision from analytics.