Link Claim Quotes

Links each extracted claim to the verbatim quotes from the candidate answer that support it, using a chat-completion model. Reads the question, answer, and claims from the source columns configured in columns, and writes the quotes to output_field. Run it after the Claim Extraction synthesizer.

Output

One output sample per source sample, with a list written to output_field holding one entry per claim - in the same order as the claims - each listing the verbatim quotes supporting that claim. The list of quotes is empty for a claim with no matching text.

Examples

Example: Link claim quotes. Links the claims of the claims column to quotes from the answer, writing them to the claim_source_quotes column.

Link Claim Quotes Synthesizer
# ...
config_spec:
  - key: "synthesizer_model_key"
    type: "model"
    display_name: "Synthesizer Model"
    description: "Model used to link claims to verbatim quotes."
definition:
  # ...
  synthesizers:
    - type: link_claim_quotes
      model_key: "<< config.synthesizer_model_key >>"
      columns:
        question: question
        answer: expected_answer
        claims: claims

The answer of the source sample wraps the actual statement in conversational filler, and the claim was extracted from it beforehand:

Source Sample
{
  "question": "What is the capital of France?",
  "expected_answer": "That is an interesting question. Luckily, I have a great answer for you -- the capital of France is no other city than Paris. Would you like to know more about Paris?",
  "claims": [
    {"claim": {"subject": "Paris", "predicate": "is the capital of", "object": "France"}}
  ]
}

The output sample keeps the source columns and adds one entry per claim, holding the shortest span of the answer that supports it, copied character for character:

Output Sample
{
  "question": "What is the capital of France?",
  "expected_answer": "That is an interesting question. Luckily, I have a great answer for you -- the capital of France is no other city than Paris. Would you like to know more about Paris?",
  "claims": [
    {"claim": {"subject": "Paris", "predicate": "is the capital of", "object": "France"}}
  ],
  "claim_source_quotes": [
    {"quotes": ["the capital of France is no other city than Paris"]}
  ]
}
Note

A claim supported by several disconnected parts of the answer gets several quotes in its quotes list, and a claim with no supporting text gets an empty list. The quotes are produced by a model, so the exact spans vary between runs.

Configuration

Properties

type Literal “link_claim_quotes” required

The type of the synthesizer.

model_key TemplateValue required

The key of the chat completion model to be used as a synthesizer.

output_field string, TemplateValue

Name of the structured output field written to each generated sample.

Default: claim_source_quotes

instructions string, TemplateValue

Optional domain-specific instructions appended to the built-in system prompt.

Default: None

columns LinkClaimQuotesColumnBindings

Mapping from built-in inputs to source sample columns.