Judges

ChatCompletionJudgeInput

Properties

---

sample object required

Sample

---

solver_output SingleSolverOutput, GroupedSolverOutput, SolverTrace, GroupedSolverTrace required

Solver Output

---

messages array[ChatCompletionInputMessage, Any, array[string], ChatCompletionOutputMessage, array[array[number]]] required

Messages

---

model_output OpenResponsesModelOutput, RAGCompletionOutput, ChatCompletionModelOutput, EmbeddingsModelOutput, Any required

Model Output

---

input_prompt string required

Input Prompt

JudgeInput

Properties

---

sample object required

Sample

SingleSolverJudgeInput

Properties

---

sample object required

Sample

---

solver_output SingleSolverOutput, GroupedSolverOutput, SolverTrace, GroupedSolverTrace required

Solver Output

---

messages array[ChatCompletionInputMessage, Any, array[string], ChatCompletionOutputMessage, array[array[number]]] required

Messages

---

model_output OpenResponsesModelOutput, RAGCompletionOutput, ChatCompletionModelOutput, EmbeddingsModelOutput, Any required

Model Output

SolverJudgeInput

Properties

---

sample object required

Sample

---

solver_output SingleSolverOutput, GroupedSolverOutput, SolverTrace, GroupedSolverTrace required

Solver Output

SolverTraceJudgeInput

Properties

---

sample object required

Sample

---

trace Trace required

---

model_outputs array[OpenResponsesModelOutput, RAGCompletionOutput, ChatCompletionModelOutput, EmbeddingsModelOutput, Any] required

Model Outputs

---

solver_output SingleSolverOutput, GroupedSolverOutput, SolverTrace, GroupedSolverTrace

Solver Output

Default: None

---

messages array[ChatCompletionInputMessage, Any, array[string], ChatCompletionOutputMessage, array[array[number]]]

Messages

Default: None

---

model_output OpenResponsesModelOutput, RAGCompletionOutput, ChatCompletionModelOutput, EmbeddingsModelOutput, Any

Model Output

Default: None

---

input_prompt string

Input Prompt

Default: None