Precision

Computes precision (TP / (TP + FP)) from per-sample true positive and false positive counts. Use this with scorers that produce counts - for example, a scorer that computes how many items in the model’s output were actually correct and how many were not. Pair with Recall and F1 Score for a full picture of overlap quality.

Output

A single metric named precision by default, or the value of name if provided. Value is in the [0, 1] range; 1.0 means no false positives.

Examples

Example: Function Call Overlap. A Python scorer checks which required function calls the model made. Precision measures how many of the calls the model made were actually required.

scorers:
  - type: python
    compute_scores_snippet: !include "function_overlap_scorer.py"
    # scorer returns: num_true_positives, num_false_positives, num_false_negatives
    metrics:
      - type: precision
        num_true_positives_field: num_true_positives
        num_false_positives_field: num_false_positives
        name: Call Precision
      - type: recall
        num_true_positives_field: num_true_positives
        num_false_negatives_field: num_false_negatives
        name: Call Recall
      - type: f1_score
        num_true_positives_field: num_true_positives
        num_false_positives_field: num_false_positives
        num_false_negatives_field: num_false_negatives
        name: Call F1

Configuration

Properties

---

type Literal “precision”

Type

Default: precision

---

num_true_positives_field string required

The field that contains the number of true positives.

---

num_false_positives_field string required

The field that contains the number of false positives.

---

name string

The name given to the metric value. If not specified, it is precision.

Default: None

---

key string

Unique identifier assigned to the entity in AI GO!.

Default: None