Overview

A solver defines how the model under test is used and how its responses are collected. AI GO! provides declarative solvers for common interaction patterns, and a Python solver for cases that require fully custom logic. Each task defines exactly one solver inline in its YAML file under the solver key, and its output is passed to the task’s scorers.

# tasks/my_task.yaml
definition:
  solver:
    type: single_turn_solver
    input_builder:
      type: chat_completion
      input_messages:
        - role: user
          content: "{{ sample.question }}"

Available Solvers

Solver	Type value	Description
Single Turn Solver	single_turn_solver	Sends input messages to the model once and collects a single response. Use this for question-answering, summarisation, and any task with no back-and-forth.
Multi Turn Solver	multi_turn_solver	Runs a multi-turn conversation driven by a configurable sequence of message builders. Use this for dialogue evaluation and agentic tasks.
Python Solver	python	Runs a custom Python function as the solver. Use this when the declarative solvers cannot express the interaction you need.
Pass Through Solver	pass_through_solver	Reads a pre-recorded trace from a dataset column instead of calling a model. Use this to score outputs generated outside AI GO! or to replay a conversation.