Inspect AI

Imports Inspect AI eval log files. Use it to turn Inspect AI transcripts into an evaluable dataset.

Behaviour

path may point to a directory of .eval log files, a single .eval file, or a transcript database directory. Use limit to cap the number of imported transcripts.

Examples

Example: Import eval logs. Imports Inspect AI transcripts from a local logs directory.

Inspect AI Dataset Source

# ...
source:
  type: inspect_ai
  path: "./logs/"

Configuration

Properties

---

type Literal “inspect_ai” required

Inspect AI eval log source.

---

path string required

Path to a directory containing .eval log files, a single .eval file, or a transcript database directory.

---

limit integer

Maximum number of transcripts to import.

Default: None