Overview

Entities are the configurable resources that make up an LF AI Platform evaluation. Each entity is defined declaratively in a YAML file (or constructed via the SDK) and referenced by the CLI. Together they describe what you evaluate (AI apps, models, datasets), how you evaluate it (tasks, evaluations, evaluation runs), what comes out (task result logs, traces), and how you govern it (policies, risk policies, frameworks).

Available Entities

Entity	Description
AI Apps	A workspace for executing technical evaluations for a given AI use case.
Models	An inference endpoint served by an external provider or your own infrastructure.
Model Adapters	Converts models with different input/output formats into a common schema.
Datasets	A collection of data used as a source of samples for model evaluation.
Dataset Generators	A modular, declarative way to create synthetic datasets.
Tasks	Defines the execution flow of an evaluation of a model or a dataset.
Evaluations and Evaluation Runs	A set of configured tasks run against a model or a dataset.
Run	A unified entry point that defines what evaluation to run and all of its dependencies.
Evaluation Plans	A collection of run configs executed together as a single named plan.
Task Result Logs	Captures the full output of a task execution: metrics, per-sample evidence, traces, and errors.
Policies	Quality gates for an AI app that check whether evaluation results meet defined thresholds.
Risk Policies	Associates a risk scorer with an evaluation scope to continuously monitor an AI app’s risk level.
Risk Scorers	Defines how a metric value is converted into a risk score for an AI app.
Frameworks	A catalogue of risks, sub-risks and controls, optionally mapped to the evaluations that provide evidence for them.
Config Specification	Defines the allowed configuration options for a task, dataset generator, or evaluation.
Trace	A conversation between a user and an agent: a sequence of messages, function calls, and function call outputs.