Risk Scorers

A risk scorer is the reusable function that converts one metric value into a risk score. It is registered once and referenced by any number of risk policies, each supplying its own parameter values, so the same notion of risk can be applied with different weights across an app.

# risk_scorers/safety_risk_scorer.yaml
key: "safety_risk_scorer"
display_name: "Safety Risk Scorer"
description: "Maps the safety score metric to a 0-10 risk score."
function: "impact * (1 - safety_score)"
metric_key: "safety_score"
config_spec:
  - type: "float"
    key: "impact"
    display_name: "Impact"
    default_value: 5.0
    min: 0.0
    max: 10.0

The scoring function

function is a mathematical expression over named variables. One name is reserved for the metric: the variable named by metric_key receives the value produced by the evaluation. Every other variable must be declared in config_spec and is filled in by the risk policy that uses the scorer - an identifier that is neither is rejected when the scorer is registered.

Design the expression so its result falls in the 0–10 range, which is how scores map to the low, medium, high and critical levels; values outside the range are clamped to it.

Parameters

config_spec uses the config specification format, restricted to the float, int and categorical types. Since the function is numeric, a categorical parameter must also carry a values_mapping that assigns a number to every allowed value, e.g. mapping internal to 0.5 and external to 1.0 so that deployment reach can amplify or dampen the score.

Working with risk scorers

Register a scorer with lf add risk-scorer, which creates it or updates the existing scorer with the same key, and list the registered ones with lf list risk-scorer. lf export risk-scorer writes a scorer back to YAML and lf delete risk-scorer removes it. See the risk policies guide for the full workflow.

Configuration

Properties

---

key string required

Unique identifier assigned to the entity in AI Platform.

Pattern: ^[a-zA-Z0-9_\-]+$
Max Length: 250

---

display_name string required

The risk scorer’s name displayed to the user.

---

description string

Short description of the risk scorer.

Default: None

---

function string required

An function expression written in Python-like math syntax (e.g. impact * (1 - metric_key)) where every variable must be covered by config_spec or metric_key.

---

metric_key string required

The variable in function that will receive the metric value.

---

config_spec array[FloatParameterSpec, IntParameterSpec, CategoricalParameterSpec]

Parameter specifications that configure this risk scorer.

Default: []

Performance Risk Scorer

key: "performance_risk_scorer"
display_name: "Performance Risk Scorer"
description: >
  Converts a model accuracy metric into a risk score by treating low accuracy as
  failure and scaling it by the business impact and deployment reach of the system.
function: "(1 - accuracy) * impact * deployment_reach"
metric_key: "accuracy"
config_spec:
  - type: float
    key: "impact"
    display_name: "Impact"
    description: >
      How severely poor performance in this area affects the system. Higher values
      amplify the risk score for the same accuracy drop.
    default_value: 5.0
    min: 0.0
    max: 10.0
  - type: categorical
    key: "deployment_reach"
    display_name: "Deployment Reach"
    description: >
      Whether the system is externally user-facing or internal only. External
      deployments amplify the score; internal deployments reduce it.
    allowed_values:
      - "internal"
      - "external"
    default_value: "external"
    values_mapping:
      internal: 0.5
      external: 1.0