Manage Secrets
Secrets store sensitive values (API keys, authentication tokens, certificates) that your evaluations need at runtime. The LF AI Platform keeps them server-side so they do not appear in plain text in your YAML configs or evaluation logs by mistake.
Inline Secrets in YAML Configs
You can declare secrets directly inside your YAML configuration. Commands lf run or lf add process the config and automatically upload (create or update) the declared inline secrets to the server before creating any entities.
Top-Level Secrets
Declare a secrets dictionary at the root of your run config. Values are typically read from environment variables using the $VAR YAML directive:
# datasets: ...
# models: ...
# tasks: ...
secrets:
OPENAI_KEY: $OPENAI_API_KEY # or directly "sk-..."
JUDGE_KEY: $JUDGE_API_KEY # or directly "sk-..."Model-Level Secrets
Each model can also declare its own secrets dictionary.
models:
- key: my-model
secrets:
API_KEY_1: $OPENAI_API_KEY # or directly "sk-..."
config:
connection_type: custom_connection
api_key: "<< secrets.API_KEY_1 >>"
# ...
- key: judge-model
secrets:
API_KEY_2: $ANTHROPIC_API_KEY
config:
connection_type: custom_connection
api_key: "<< secrets.API_KEY_2 >>"
# ...Dataset-Generator-Level Secrets
A dataset generator can declare its own secrets dictionary in exactly the same way - see Dataset Generators below for a full example.
dataset_generators:
- key: my-generator
secrets:
OPENAI_API_KEY: $OPENAI_API_KEYTop-level, model-level and dataset-generator-level secrets are all merged before upload. If the same secret name appears in more than one place with different values, the CLI raises an error.
Secret names may contain only letters, digits, _ and -. Secrets are scoped per tenant, not per AI app, so a name is shared across every app of the tenant.
If you manage a secret exclusively through lf secret add, you do not need to redeclare it in YAML. You only need to reference it with the << secrets.name >> syntax.
Manage Secrets with the CLI
The lf secret subcommand lets you manage secrets directly.
Create or Update a Secret
lf secret add --name OPENAI_KEY --value "sk-..."If a secret with that name already exists, its value is replaced.
List Secrets
lf secret listDelete a Secret
lf secret delete --name OPENAI_KEYSee the full lf secret CLI reference for all available options.
Reference Secrets
Once secrets are stored in the LF AI Platform (via lf secret add or inline in YAML - see Inline Secrets in YAML Configs and Manage Secrets with the CLI), you can reference them in your configuration using the << secrets.NAME >> placeholder syntax. The LF AI Platform resolves these placeholders at runtime, so the actual values never appear in your config files or evaluation logs.
Syntax
Reference a secret by wrapping its name with << secrets. >>:
<< secrets.SECRET_NAME >>Where Secrets Can Be Used
Model Connection Config
Secrets are most commonly used to inject API keys and authentication headers into model connection configs.
api_key
models:
- key: my-model
config:
connection_type: custom_connection
adapter:
key: "latticeflow$openai_chat_completion"
url: "https://api.openai.com/v1/chat/completions"
api_key: "<< secrets.OPENAI_KEY >>"
model_key: "gpt-4.1-mini"
secrets:
OPENAI_KEY: $OPENAI_API_KEYcustom_headers
models:
- key: my-model
config:
connection_type: custom_connection
url: "https://my-endpoint.example.com/v1/chat/completions"
custom_headers:
X-Auth-Token: "<< secrets.AUTH_TOKEN >>"
X-Api-Version: "2024-01"
secrets:
AUTH_TOKEN: $MY_AUTH_TOKENenvironment
For custom inference models that need environment variables at runtime:
models:
- key: custom-model
config:
connection_type: custom_inference
environment:
HF_TOKEN: "<< secrets.HF_TOKEN >>"
secrets:
HF_TOKEN: $HUGGING_FACE_TOKENTLS trusted_ca
models:
- key: my-model
config:
connection_type: custom_connection
url: "https://internal-endpoint.corp.net/v1/chat/completions"
tls_context:
validation_context:
trusted_ca: "<< secrets.CA_CERT >>"
secrets:
CA_CERT: $MY_CA_CERTIFICATEDataset Generators
Secrets can also be specified in dataset generators to hide sensitive information such as API keys. The secret is then referenced in Python synthesizer snippets or other synthesizer templates.
dataset_generators:
- key: my-dataset-generator
config_spec: []
definition:
data_source:
type: "empty"
synthesizers:
- type: "python"
synthesize_snippet: !include "./synthesize.py"
secrets:
OPENAI_API_KEY: $OPENAI_API_KEYfrom typing import Any
from openai import OpenAI
def synthesize(source: dict[str, Any]) -> list[dict[str, Any]]:
client = OpenAI(api_key="<< secrets.OPENAI_API_KEY >>")
response = client.responses.create(
model="gpt-5.5",
instructions="You are a helpful data synthesis assistant.",
input="Please write several comma-separated Harry Potter questions in a single line.",
)
return [
{"question": question.strip()} for question in response.output_text.split(",")
]Task Definitions
Secret placeholders can also appear inside task definitions - for example, in custom Python scorer snippets or other templated fields. The LF AI Platform resolves all << secrets.NAME >> placeholders in the entire task definition at execution time.
A task has no secrets block of its own, so the secrets it references must either already exist on the instance or be declared at the top level of the same run config. lf add task -f task.yaml on its own uploads no secrets.
secrets:
TARGET_FIELD: "is_correct"
CONTENT_KEY: "message"
tasks:
- key: my-task
display_name: "My Task"
description: "Scores whether the model answered YES."
config_spec:
- type: "dataset"
key: "dataset_key"
display_name: "Dataset"
default_value: "my-dataset"
definition:
dataset:
key: "<< config.my-dataset >>"
solver:
type: single_turn_solver
input_builder:
type: chat_completion
input_messages:
- role: user
content: "{{ sample.question }}"
scorers:
- type: python
compute_scores_snippet: |
def compute_scores(sample, solver_output):
content = solver_output.trace.get_last_assistant_text() or ""
return {"<< secrets.TARGET_FIELD >>": content.strip() == "YES"}
metrics:
- key: accuracy
type: mean
field: << secrets.TARGET_FIELD >>Note that scorers lives under definition, and that display_name and description are required on a task.
Agent Platform Credentials
Every credential field of the agent connection types accepts a secret reference too - api_key for langsmith, dify and claude_managed_agents, tenant_id/client_id/client_secret for azure_foundry, and access_key_id/secret_access_key/session_token for aws_bedrock. See Integrations for the full configuration of each platform.
Traceability of Secrets Used in Evaluations
Secrets are snapshotted the same way as any other entity used in an evaluation. That means that if an evaluation was run with a secret (used in a model or task), even if the secret was later updated, the original value of the secret is preserved for traceability purposes, and it is possible to find out which secret value the evaluation was run with
Exporting Evaluation runs with Secret Values
By default, if you run lf export eval-run, the secret values are not exported. However, to achieve full traceability, admin users can use the --with-secrets flag to export evaluation results with the secret values.
lf export eval-run --id <eval ID> --output ./results --with-secretslf dump takes the same flag, which is what makes a dumped AI app restorable on an instance that does not have the secrets yet:
lf dump --to ./my-app-dump --with-secretsBoth commands write credentials to disk in plain form, so treat the output accordingly. See Back up and move an AI App.