Pass Through Solver
Reads pre-recorded trace / conversation history from a dataset column instead of calling a model. Use this to evaluate outputs that were generated outside of AI GO!, or to replay a previous conversation.
Examples
Example: Pre-recorded QA Traces. Reads a full conversation trace from the trace column of the dataset and presents it to the scorers without calling any model. Useful for scoring outputs generated outside AI GO! or replaying a previous run.
Pass Through Solver
# ...
definition:
# ...
solver:
type: "pass_through_solver"
message_format: "open_responses"
trace_column: "trace"Given a dataset row whose trace column contains a pre-recorded single-turn exchange:
| Turn | Role | Content |
|---|---|---|
| 1 | system | You are a helpful assistant. Answer concisely with a single word. |
| 2 | user | What is the capital city of Japan? |
| 3 | assistant | Tokyo |
The solver replays all three turns verbatim. The last assistant message (Tokyo) becomes solver_output.output passed to the scorers.
Multi-turn traces with tool calls are also supported:
| Turn | Role | Content |
|---|---|---|
| 1 | user | What is the current temperature in Seoul? |
| 2 | function_call | get_current_weather({"city": "Seoul", "unit": "celsius"}) |
| 3 | function_call_output | {"temperature": 11, "condition": "partly cloudy"} |
| 4 | assistant | The current temperature in Seoul is 11°C with partly cloudy skies. |
| 5 | user | And what about Berlin? |
| 6 | function_call | get_current_weather({"city": "Berlin", "unit": "celsius"}) |
| 7 | function_call_output | {"temperature": 4, "condition": "overcast"} |
| 8 | assistant | In Berlin it is currently 4°C and overcast. |
Configuration
Properties
postprocessor PythonPostprocessorTemplate
An optional postprocessor applied to solver outputs before scoring.
Default: None
type Literal “pass_through_solver” required
The type of the solver.
trace_column string required
The name of the dataset column that contains the trace. The value must be deserializable as a Trace object.
message_format string
The message format used by the solver for recording its output. Defaults to open_responses; latticeflow is deprecated.
Default: open_responses