Custom LLM Provider

For any OpenAI-compatible LLM endpoint that isn’t in the built-in provider list (lf integration add), define it directly with a model.yaml using connection_type: custom_connection.

Let an agent do it for you

Install the LatticeFlow skill so your AI coding agent can write the model YAML, register it, and test it for you:

lf skills install --agent claude_code   # or: cursor, opencode, codex, antigravity

This drops the lf-skill playbook into your agent’s skills directory, giving it the lf CLI reference and the YAML schemas it needs to complete the integration. Otherwise, follow the manual steps below.

Step 1 - Write `model.yaml`

Point url at your OpenAI-compatible /v1/chat/completions endpoint and use the built-in latticeflow$openai_chat_completion adapter to shape the request/response. If your wire format differs from OpenAI’s, write your own model adapter and reference it by key instead.

custom-model.yaml

key: "my-custom-model"
display_name: "My Custom Model"
description: "OpenAI-compatible endpoint served via custom_connection."
task: "chat_completion"
rate_limit: 60
config:
  connection_type: "custom_connection"
  adapter:
    key: "latticeflow$openai_chat_completion"
  url: "https://my-endpoint.example.com/v1/chat/completions"
  api_key: "<< secrets.CUSTOM_API_KEY >>"
  model_key: "my-model-name"
secrets:
  CUSTOM_API_KEY: $CUSTOM_API_KEY

Config fields

Field	Required	Secret	Description
`connection_type`	yes	no	Must be `custom_connection`.
`url`	yes	no	The endpoint URL (e.g. `.../v1/chat/completions`).
`adapter.key`	no	no	Request/response adapter; use the built-in `latticeflow$openai_chat_completion` for OpenAI-compatible endpoints.
`api_key`	no	yes	Sent as `Authorization: Bearer <key>`.
`model_key`	no	no	Model name passed in the request body.
`custom_headers`	no	yes	Extra request headers; can override defaults.

rate_limit (requests per minute) and max_concurrent_requests cap how hard AI Platform calls the endpoint - start conservatively and raise them once the endpoint proves stable. Every supported field is documented in the Models CLI reference.

API key in a custom header

Some endpoints expect the token in a header of their own, such as X-API-Key, rather than in Authorization:

curl <url> -H "X-API-Key: <token>" -d '...'

Leave api_key empty and pass the token through custom_headers. Header values are restricted to printable ASCII - a value with newlines or non-ASCII characters is rejected.

custom-model.yaml

config:
  connection_type: "custom_connection"
  api_key: ""
  custom_headers:
    X-API-Key: "<< secrets.CUSTOM_API_KEY >>"

Endpoint behind a private CA

An endpoint on your own infrastructure may present a certificate signed by a private CA rather than a public root, which fails validation by default. Configure the trust chain through tls_context:

custom-model.yaml

config:
  connection_type: "custom_connection"
  tls_context:
    validation_context:
      trusted_ca: "<< secrets.CA_CERT >>"
      trust_chain_verification: "verify_trust_chain"

trusted_ca must be the base64 encoding of the PEM file, not the PEM text - a raw -----BEGIN CERTIFICATE----- block is rejected with Invalid base64-encoded TLS certificate. Concatenate the chain into one PEM file, then encode it:

cat chain.pem | base64 -w 0

For local development against a throwaway endpoint you can skip validation entirely with trust_chain_verification: "accept_untrusted". Never do this against a system you do not control.

Step 2 - Provide credentials

The secrets block uploads values (from your .env or environment) to server-side secret storage, and << secrets.NAME >> references them from the config so no key is stored in plaintext. Provide the value in your .env:

CUSTOM_API_KEY=sk-...

See Manage secrets for the other ways to supply a secret, including storing it server-side up front with lf secret add.

Step 3 - Add and test

lf add model -f custom-model.yaml
lf test model my-custom-model

lf test model sends one sample request through the connection and adapter, so you find out whether the endpoint is reachable and the response well-formed before an evaluation depends on it - see Testing models.

Step 4 - Use in an evaluation

Reference the model key from a task specification in your run config:

evaluation:
  config_spec:
    - key: "model_key"
      type: "model"
      display_name: "Model"
      description: "Model to be used for the task."
  task_specifications:
    - task_key: my-task
      model_key: << config.model_key >>

config:
  model_key: "my-custom-model"

Note

If your endpoint isn’t OpenAI-compatible, use connection_type: custom_inference instead - an arbitrary Python run_inference snippet that makes the call. The lf skills playbook ships a run_inference template that scaffolds this for you.