Get evaluations

Get list of evaluations with pagination and filtering. See Evaluations.

GET /evaluations

Get evaluations

cURL

curl --request GET \
  --url https://api.galtea.ai/evaluations \
  --header 'Authorization: Bearer <token>'

200

[
  {
    "id": "eval_123",
    "metricId": "metric_123",
    "sessionId": "session_123",
    "productId": "product_123",
    "userId": "user_123",
    "status": "SUCCESS",
    "inferenceResultId": "ir_123",
    "score": 0.95,
    "reason": "High quality response",
    "error": "<string>",
    "canRetry": false,
    "creditsUsed": 1,
    "judgeInputTokens": 4100,
    "judgeOutputTokens": 16800,
    "judgeReasoningTokens": 12400,
    "conversationSimulatorVersion": "1.0.0",
    "humanEvaluatorId": "<string>",
    "humanEvaluatorStartedAt": "2023-11-07T05:31:56Z",
    "humanScore": 123,
    "humanReason": "<string>",
    "humanEvaluatorFinishedAt": "2023-11-07T05:31:56Z",
    "failedTurns": [
      "<string>"
    ],
    "monitorId": "<string>",
    "monitorSettledAtTurn": 123,
    "monitorSamplingPercentage": 123,
    "runId": "<string>",
    "deletedAt": "2023-11-07T05:31:56Z",
    "evaluatedAt": "2023-11-07T05:31:56Z",
    "metricLegacyAt": "2023-11-07T05:31:56Z",
    "passed": true,
    "metricPassThreshold": 0.5,
    "metricDisabledAt": "2023-11-07T05:31:56Z",
    "metricExcludedFromAnalyticsAt": "2023-11-07T05:31:56Z",
    "metricExcludedByUserId": "<string>",
    "testCaseLegacyAt": "2023-11-07T05:31:56Z",
    "scoreBreakdown": {
      "version": 123,
      "model": "<string>",
      "type": "yes_no",
      "value": 123,
      "confidence": 123,
      "pYes": 0.5,
      "goodAnswer": "yes",
      "probabilities": {},
      "levelScores": [
        123
      ],
      "doesNotApply": true
    },
    "confidence": 0.9,
    "longInput": true,
    "createdAt": "2023-11-07T05:31:56Z"
  }
]

Authorizations

Authorization · string · header · required

API key authorization. Pass your API key in the Authorization header as a Bearer token. Both new (gsk_*) and legacy (gsk-) API keys are accepted, e.g. Authorization: Bearer gsk_... or Authorization: Bearer gsk-....

Query Parameters

ids · string[]

Filter by evaluation IDs

productIds · string[]

Filter by product IDs

sessionIds · string[]

Filter by session IDs

inferenceResultIds · string[]

Filter by trace IDs (for single-turn evaluations)

metricIds · string[]

Filter by metric IDs

metricGroupIds · string[]

Filter by metric group IDs (matches all revisions of a metric)

monitorIds · string[]

Filter by monitor IDs (returns only evaluations dispatched by the given monitors)

runIds · string[]

Filter by run IDs (returns only evaluations launched by the given runs)

monitorAlertIds · string[]

Filter by monitor alert IDs (returns only the evaluations of the sessions the alert linked when it opened (its bad sessions, or its sessions with no score), made by the alert's monitor, on the alert's metric family, or on the families a Specification alert measured when it opened). At most 10 IDs.

Maximum array length: 10

testCaseIds · string[]

Filter by test case IDs

testIds · string[]

Filter by test IDs (include only). Use an empty string ("") to select evaluations without a test

excludeTestIds · string[]

Omit evaluations linked to the specified test IDs. Use an empty string ("") to omit evaluations without a test

versionIds · string[]

Filter by version IDs

specificationIds · string[]

Filter by specification IDs (returns evaluations whose metric is linked to any of the given specifications)

statuses · enum<string>[]

Filter by evaluation statuses

Available options: PENDING, PENDING_HUMAN, SUCCESS, FAILED, SKIPPED, CANCELLED, OUTDATED

evaluationTypes · string[]

Filter by evaluation types

metricSources · string[]

Filter by metric sources

sort · string[]

Sort instructions (field and direction pairs). Sorting by confidence puts evaluations with no confidence last in both directions. A caller gets 400 for a sort on a field it sees as null on any product the list reaches (evaluation.readInference).

canRetry · boolean

Filter evaluations that can be retried

augmentedTestCase · boolean

Filter evaluations by whether their test case is augmented

passed · boolean

Filter by verdict: true returns evaluations whose reported score reaches their metric pass threshold, false the ones below it. Evaluations with no verdict match neither. Any value other than true or false returns 400.

isProduction · boolean

Filter evaluations by the joined Session.isProduction flag. Prefer this over the legacy testIds/excludeTestIds empty-string trick.

includeExcludedFromAnalytics · boolean

Include evaluations whose metric revision is excluded from analytics. Omit or pass false to hide them.

humanEvaluatorId · string

Filter by human evaluator user ID

userIds · string[]

Filter by the launching user's id

fromCreatedAt · string<date-time>

Filter evaluations created at or after this timestamp (ISO 8601 format)

toCreatedAt · string<date-time>

Filter evaluations created at or before this timestamp (ISO 8601 format)

scoreGt · number

Keep evaluations whose judge score is greater than this value (0 to 1). Reads the judge's score, not the human review. An evaluation with no judge score never matches. A value outside 0 to 1 returns 400.

scoreGte · number

Keep evaluations whose judge score is greater than or equal to this value (0 to 1). Reads the judge's score, not the human review. An evaluation with no judge score never matches. A value outside 0 to 1 returns 400.

scoreLt · number

Keep evaluations whose judge score is less than this value (0 to 1). Reads the judge's score, not the human review. An evaluation with no judge score never matches. A value outside 0 to 1 returns 400.

scoreLte · number

Keep evaluations whose judge score is less than or equal to this value (0 to 1). Reads the judge's score, not the human review. An evaluation with no judge score never matches. A value outside 0 to 1 returns 400.

humanScoreGt · number

Keep evaluations whose human review score is greater than this value (0 to 1). An evaluation with no human score never matches. A value outside 0 to 1 returns 400.

humanScoreGte · number

Keep evaluations whose human review score is greater than or equal to this value (0 to 1). An evaluation with no human score never matches. A value outside 0 to 1 returns 400.

humanScoreLt · number

Keep evaluations whose human review score is less than this value (0 to 1). An evaluation with no human score never matches. A value outside 0 to 1 returns 400.

humanScoreLte · number

Keep evaluations whose human review score is less than or equal to this value (0 to 1). An evaluation with no human score never matches. A value outside 0 to 1 returns 400.

confidenceGt · number

Keep evaluations whose classifier confidence is greater than this value (0 to 1). An evaluation with no confidence never matches. A value outside 0 to 1 returns 400.

confidenceGte · number

Keep evaluations whose classifier confidence is greater than or equal to this value (0 to 1). An evaluation with no confidence never matches. A value outside 0 to 1 returns 400.

confidenceLt · number

Keep evaluations whose classifier confidence is less than this value (0 to 1). An evaluation with no confidence never matches. A value outside 0 to 1 returns 400.

confidenceLte · number

Keep evaluations whose classifier confidence is less than or equal to this value (0 to 1). An evaluation with no confidence never matches. A value outside 0 to 1 returns 400.

limit · integer · default: 10000

Maximum number of results

Required range: 0 <= x <= 9007199254740991

offset · integer · default: 0

Number of results to skip

Required range: x >= 0

Response

200

application/json

Evaluations retrieved successfully

id · string · required

Example:

"eval_123"

metricId · string · required

Example:

"metric_123"

sessionId · string · required

Example:

"session_123"

productId · string | null · required

Product the evaluated session belongs to

Example:

"product_123"

userId · string | null · required

Example:

"user_123"

status · enum<string> · required

Available options: PENDING, PENDING_HUMAN, SUCCESS, FAILED, SKIPPED, CANCELLED, OUTDATED

Example:

"SUCCESS"

inferenceResultId · string | null · required

Example:

"ir_123"

score · number | null · required

Example:

0.95

reason · string | null · required

Example:

"High quality response"

error · string | null · required

canRetry · boolean · required

Example:

false

creditsUsed · integer | null · required

Example:

1

judgeInputTokens · integer | null · required

Tokens the judge received, prompt plus session content

Example:

4100

judgeOutputTokens · integer | null · required

Tokens the judge produced, reasoning included

Example:

16800

judgeReasoningTokens · integer | null · required

Reasoning tokens, a subset of judgeOutputTokens

Example:

12400

conversationSimulatorVersion · string | null · required

Example:

"1.0.0"

humanEvaluatorId · string | null · required

User ID of the human evaluator

humanEvaluatorStartedAt · string<date-time> | null · required

humanScore · number | null · required

Human-provided annotation score

humanReason · string | null · required

Human-provided annotation reason

humanEvaluatorFinishedAt · string<date-time> | null · required

Timestamp when human evaluation was submitted

failedTurns · string[] · required

Conversation turns that failed

monitorId · string | null · required

ID of the monitor that dispatched this evaluation

monitorSettledAtTurn · integer | null · required

Session turn count when the monitor scored it

monitorSamplingPercentage · number | null · required

Sampling percentage in effect when the monitor scored this session

runId · string | null · required

ID of the run that launched this evaluation

deletedAt · string<date-time> | null · required

evaluatedAt · string<date-time> | null · required

metricLegacyAt · string<date-time> | null · required

passed · boolean | null · required

Whether the reported score (the reviewer's when set, else the judge's) reaches the metric's pass threshold. Null when the evaluation has no reported score, is outdated or deleted, is a long classifier input, or its metric is not loaded.

metricPassThreshold · number · required

The pass threshold that decides passed: the metric's, copied onto the evaluation when it was created.

Required range: 0 <= x <= 1

metricDisabledAt · string<date-time> | null · required

metricExcludedFromAnalyticsAt · string<date-time> | null · required

metricExcludedByUserId · string | null · required

testCaseLegacyAt · string<date-time> | null · required

scoreBreakdown · object | null · required

CLASSIFIER only: what the classifier answered; null for every other source.

Show child attributes

confidence · number | null · required

CLASSIFIER only: how sure the classifier was, from 0 to 1; null for every other source. Values do not compare across question types.

Required range: 0 <= x <= 1

Example:

0.9

longInput · boolean · required

True for a CLASSIFIER evaluation whose input ran past what the classifier scores reliably (CLASSIFIER_RELIABLE_INPUT_TOKENS judge input tokens) and that no reviewer has scored; false for every other evaluation.

createdAt · string<date-time>