v3.10.0

Run untrusted and model-generated code in a safe execution sandbox, connect external agent providers (Dify, LangSmith, Claude Managed Agents, AWS Bedrock, Azure Foundry), re-run evaluations against the latest versions of your assets, and back up an entire AI app with lf dump/lf restore. Task result evidence is now paginated, filterable, and sortable for faster loads.

What’s new

  • Safe code execution sandbox to run untrusted and model-generated code in an isolated environment.

  • Connect external agent providers — Dify, LangSmith, Claude Managed Agents, AWS Bedrock, and Azure Foundry — as models in your AI apps.

  • Register a placeholder model connection (connection_type: placeholder) to stand in for a model before its real integration exists — it returns empty completions so you can set up apps and pipelines ahead of time:

    display_name: "Placeholder model"
key: "placeholder_model"
task: "chat_completion"
config:
  connection_type: "placeholder"

  • Re-run an evaluation against the latest versions of its models, datasets, adapters, tasks, and secrets, or keep the original versions.

  • CLI: back up and restore an entire AI app with lf dump and lf restore.

  • CLI: run lf test task on specific samples via --sample-ids.

  • CLI: export risk scorers and risk policies with lf export risk-scorer and lf export risk-policies.

  • SDK: write custom scorers with the new latticeflow.core.experimental.scorers module, including agent scorers such as FunctionCallCoverage, DisallowedFunctionCalls, FunctionCallOutputContains, and FunctionCallOutputNotContains.

  • Risk policy status is now shown for risk policies that haven’t been evaluated yet.

Improvements

  • Task result evidence now loads faster: it is served through paginated endpoints with server-side filtering and sorting.
  • Clearer error message when you try to connect to an HTTPS server over http://.
  • lf test now surfaces errors immediately instead of logging them and continuing.
  • Renamed the evidence statistic “Used for Metrics” to “Excluded from Metrics” for clarity.

Bug fixes

  • Filter order is now preserved when adding or removing filter values.
  • Removed the rnj-1-instruct Together AI model, which was no longer responding.
  • Fixed upload of the new task result log format.