LangSmith

Connect a LangGraph agent deployed on the LangGraph Platform (LangSmith) to AI GO! as a native model, using the typed langsmith connection. AI GO! drives the conversation one user turn at a time and the integration keeps the LangGraph thread_id in sync automatically, so multi-turn evaluations preserve conversation state without any extra work.

Prerequisites

A LangGraph deployment on the LangGraph Platform, and its base deploy URL.
The assistant ID of the graph to invoke.
A LangSmith API key.

Configuration

Define the model with connection_type: langsmith and reference credentials as secrets:

key: "langsmith-agent"
display_name: "LangSmith Agent"
description: "Agent on the LangGraph Platform, served via the typed `langsmith` connection config."
task: "chat_completion"
rate_limit: 15
config:
  connection_type: "langsmith"
  deploy_url: $LANGSMITH_DEPLOY_URL
  assistant_id: $LANGGRAPH_ASSISTANT_ID
  api_key: "<< secrets.LANGSMITH_API_KEY >>"
secrets:
  LANGSMITH_API_KEY: $LANGSMITH_API_KEY

Config fields

Field	Required	Secret	Description
`connection_type`	yes	no	Must be `langsmith`.
`deploy_url`	yes	no	Base URL of the LangGraph deployment.
`assistant_id`	yes	no	Identifier of the LangGraph assistant to invoke.
`api_key`	yes	yes	LangSmith API key.

Credentials

The secrets block uploads values (from your .env or environment) to server-side secret storage, and << secrets.NAME >> references them from the config so no key is stored in plaintext. Provide the values in your .env:

LANGSMITH_DEPLOY_URL=https://my-deployment.us.langgraph.app
LANGGRAPH_ASSISTANT_ID=my-assistant
LANGSMITH_API_KEY=lsv2_...

Add and test

lf add model -f langsmith.yaml
lf test model langsmith-agent

Use in an evaluation

Reference the model key from a task specification in your run config:

evaluation:
  task_specifications:
    - task_key: my-task
      model_key: "langsmith-agent"

Conversation state

On the first turn the integration creates a LangGraph thread; the thread_id is attached to the assistant message and read back on each subsequent turn, so the agent remembers earlier turns. The full per-turn trace (tool calls, tool outputs, and the final reply) is returned in Open Responses format for trace-aware scorers.