Evaluation Parameters

Data fields available to metric evaluators during scoring

When creating or configuring a metric, you select which parameters are relevant for your evaluation. These parameters are made available to the evaluator during scoring.

• For AI Evaluation, the selected parameters are automatically prepended to your judge_prompt.
• For Human Evaluation, they determine which data fields are displayed to annotators.
• For Self-Hosted metrics, evaluation parameters do not apply.

Only select parameters that your evaluation criteria actually reference. Including unnecessary parameters can reduce evaluation quality by adding noise to the evaluator’s context.

Parameter Reference

Parameter	Description	Availability
input	The prompt or query sent to the model.	Accuracy, Security & Safety, and Behavior
actual_output	The actual output generated by the model.	Accuracy, Security & Safety, and Behavior
expected_output	The ideal answer for the given input.	Accuracy and Security & Safety
context	Additional background information provided to the model alongside the input.	All metrics
retrieval_context	The context retrieved by your RAG system before sending the user query to your LLM. Derived from your RETRIEVER spans; you can also supply it directly and it is stored as a RETRIEVER span, not a directly-readable field.	Accuracy, Security & Safety, and Behavior
traces	Execution traces from the agent, including tool calls, LLM invocations, and other internal operations.	All metrics
expected_tools	List of tools expected to be used by the agent to accomplish the task.	All metrics
tools_used	List of tools actually used by the agent during execution (automatically inferred from traces).	All metrics
product_description	The description of the product.	All metrics
product_capabilities	The capabilities of the product.	All metrics
product_inabilities	The product’s known inabilities or restrictions.	All metrics
product_security_boundaries (deprecated)	The security boundaries of the product. Deprecated — no longer accepted for new metrics. Use product_capabilities and product_inabilities instead.	N/A (rejected by API for new metrics)
user_persona	Information about the user interacting with the agent.	Behavior tests
goal	The user’s objective in the conversation.	Behavior tests
scenario	The context or situation for the conversation.	Behavior tests
stopping_criterias	List of criteria that define when a conversation should end.	Behavior tests
conversation_turns	All turns in a conversation, including user and assistant messages.	Behavior tests (Human Evaluation only)

Troubleshooting skipped evaluations

When an evaluation runs against a session that does not have all the data the metric needs, Galtea marks the evaluation as SKIPPED instead of producing a misleading score. The evaluation’s error field describes which parameters are missing, grouped by where you provide them.

The categorized error message looks like this:

Metric "Contextual Relevancy" requires data that is not available.

Missing from product settings:
  • product_description - Fill in the "Description" field in your product settings.

Missing from test case:
  • expected_output - Provide an "Expected Output" value on your test cases.

Missing span data:
  • retrieval_context - "retrieval_context" is derived from RETRIEVER spans. Configure tracing so your product emits retrieval steps — see the tracing setup docs.
  • traces - This metric needs span data. Configure your product to send spans — see the tracing setup docs.

Learn more: https://docs.galtea.ai/concepts/metric/evaluation-parameters#troubleshooting-skipped-evaluations

Each section points to a different place to look:

Section	Where to fix it
Cannot be read by the judge	The value is there, but the judge cannot use it. The judge is sent the conversation turns, so the turns decide this: a metric that reads input is skipped when no turn is left with text of its own, counting neither a prompt sent beside a file nor an audio part with no transcript. One turn carrying text of its own is enough, so an attachment on one turn of a long session does not skip the rest. Score those sessions with a metric that does not read the input, such as a JSON Field Match metric or a self-hosted metric. A metric that reads conversation_turns is never skipped for this: the turns are scored as you logged them. Nine of the eleven predefined Galtea metrics declare input, so expect most of them to skip on a session carrying a document. The Run Evaluation dialog greys these metrics out before the run, with the same reason under the name.
Missing from product settings	Open the product and fill in the missing field (Description, Capabilities, or Inabilities). Capabilities and Inabilities can also be derived from the product’s Specifications.
Missing from test case	Edit the test case and provide the missing value. Some fields (Goal, User Persona, Scenario, Stopping Criteria) are only populated on SCENARIOS-type test cases. If your test cases use a different type, use a metric that does not require these parameters.
Missing from endpoint connection’s output mapping	Edit the endpoint connection and add the missing key to the output mapping. actual_output is extracted from the output key. See Templates & Mapping for the JSONPath syntax used in mapping values.
Missing trace data	The session has no conversation turns yet. Run the metric against a session that contains at least one turn.
Missing span data	The metric needs span-derived data (traces, tools_used, or retrieval_context). Configure your product to send span data — see the Tracing Agent Operations tutorial for setup, and the Span concept page for what gets captured. tools_used (from TOOL spans) and retrieval_context (from RETRIEVER spans) are automatically extracted from your span data; you do not provide them directly.

A skipped evaluation consumes no credits, and it is never dispatched to the evaluator.

Once you have provided the missing data, re-run the evaluation against the same session. A Cannot be read by the judge section is the exception: no data is missing, so re-running changes nothing until you pick a different metric or log the turn with the text you want scored, without the file.

Related

Evaluation Types

Understand AI Evaluation, Human Evaluation, and Self-Hosted scoring.

Metrics Overview

Browse all available metrics and create custom ones.