v3.14.0

This release introduces Frameworks as first-class entities that map risks and controls to the evaluations covering them. Atlas evaluations and tasks are now synchronized into the platform automatically, and entities are no longer scoped to a single AI app.

What’s new

  • OWASP Agentic Top 10: AI Atlas now contains an extensive coverage of evaluations for OWASP Agentic Top 10
  • Frameworks as entities: A framework - the EU AI Act or a custom risk taxonomy - can now be defined as a tenant-wide entity with its risks, sub-risks and controls, managed through the CLI and browsed in a new read-only Frameworks section of the UI.

Improvements

  • Connection status check: The AI App overview now verifies that the app’s model under test and judge model are configured and reachable, and reflects the outcome of the last connection test.
  • Global entity dump and restore: lf dump entities and lf restore entities dump and restore all of a tenant’s global entities, complementing the per-app lf dump and lf restore.
  • Sample evidence can be copied in full as formatted JSON with a new copy button.
  • Very large values in dataset and result tables are replaced with a download hint.
  • Python snippets can process much larger inputs — script input is now passed over stdin, removing the previous 128 KB argument-size limit.
  • Task specifications are shown on the Task Overview and Dataset Generator pages in the UI.

Bug fixes

  • lf list returned internal stored types and incorrect paths instead of the SDK types.
  • lf add failed to resolve paths relative to evaluation plan and run config files.
  • lf test task offered arguments that did not apply to the task type, and lf configure showed a misleading value hint.
  • The dataset viewer no longer breaks on very long strings, and trailing whitespace is stripped from JSON values.

Breaking changes

  • Entities are no longer scoped to an AI app. Models, datasets, tasks, dataset generators, model adapters and tags are now tenant-global. Their API routes move from /api/ai-apps/{ai_app_id}/... to /api/..., and existing keys that collided across AI apps are made unique during migration. lf dump and lf restore still cover a full AI app, but the entities they carry are now limited to runs and policies.
  • Evaluations must declare their entity dependencies explicitly. Hardcoding a model_key or dataset_key in a task specification template is now rejected; entities must be referenced through a dynamic config placeholder such as << config.my_model >>. See the migration guide.

Note that further changes will be introduced in the upcoming releases to simplify the execution and reuse of evaluations which may further affect the API.