Custom LLM Provider

For any OpenAI-compatible LLM endpoint that isn’t in the built-in provider list (lf integration add), define it directly with a model.yaml using connection_type: custom_connection.

Let an agent do it for you

Install the LatticeFlow skill so your AI coding agent can write the model YAML, register it, and test it for you:

lf skills install --agent claude_code   # or: cursor, opencode, codex, antigravity

This drops the lf-skill playbook into your agent’s skills directory, giving it the lf CLI reference and the YAML schemas it needs to complete the integration. Otherwise, follow the manual steps below.

Step 1 — Write `model.yaml`

Point url at your OpenAI-compatible /v1/chat/completions endpoint and use the built-in latticeflow$openai_chat_completion adapter to shape the request/response:

custom-model.yaml

key: "my-custom-model"
display_name: "My Custom Model"
description: "OpenAI-compatible endpoint served via custom_connection."
task: "chat_completion"
rate_limit: 60
config:
  connection_type: "custom_connection"
  adapter:
    key: "latticeflow$openai_chat_completion"
  url: "https://my-endpoint.example.com/v1/chat/completions"
  api_key: "<< secrets.CUSTOM_API_KEY >>"
  model_key: "my-model-name"
secrets:
  CUSTOM_API_KEY: $CUSTOM_API_KEY

Config fields

Field	Required	Secret	Description
`connection_type`	yes	no	Must be `custom_connection`.
`url`	yes	no	The endpoint URL (e.g. `.../v1/chat/completions`).
`adapter.key`	no	no	Request/response adapter; use the built-in `latticeflow$openai_chat_completion` for OpenAI-compatible endpoints.
`api_key`	no	yes	Sent as `Authorization: Bearer <key>`.
`model_key`	no	no	Model name passed in the request body.
`custom_headers`	no	yes	Extra request headers; can override defaults.

Step 2 — Provide credentials

The secrets block uploads values (from your .env or environment) to server-side secret storage, and << secrets.NAME >> references them from the config so no key is stored in plaintext. Provide the value in your .env:

CUSTOM_API_KEY=sk-...

Step 3 — Add and test

lf add model -f custom-model.yaml
lf test model my-custom-model

Step 4 — Use in an evaluation

Reference the model key from a task specification in your run config:

evaluation:
  task_specifications:
    - task_key: my-task
      model_key: "my-custom-model"

Note

If your endpoint isn’t OpenAI-compatible, use connection_type: custom_inference instead — an arbitrary Python run_inference snippet that makes the call. The lf skills playbook ships a run_inference template that scaffolds this for you.