Risk Scoring Methodologies

A risk scoring methodology is the reusable function that converts one metric value into a risk score. It is registered once and referenced by any number of risk scorers, each supplying its own parameter values, so the same notion of risk can be applied with different weights across an app.

# risk_scoring_methodologies/safety_risk_scoring_methodology.yaml
key: "safety_risk_scoring_methodology"
display_name: "Safety Risk Scoring Methodology"
description: "Maps the safety score metric to a 0-10 risk score."
function: "impact * (1 - safety_score)"
metric_key: "safety_score"
config_spec:
  - type: "float"
    key: "impact"
    display_name: "Impact"
    default_value: 5.0
    min: 0.0
    max: 10.0

The scoring function

function is a mathematical expression over named variables. One name is reserved for the metric: the variable named by metric_key receives the value produced by the evaluation. Every other variable must be declared in config_spec and is filled in by the risk scorer that uses the methodology - an identifier that is neither is rejected when the methodology is registered.

Design the expression so its result falls in the 0–10 range, which is how scores map to the low, medium, high and critical levels; values outside the range are clamped to it.

Parameters

config_spec uses the config specification format, restricted to the float, int and categorical types. Since the function is numeric, a categorical parameter must also carry a values_mapping that assigns a number to every allowed value, e.g. mapping internal to 0.5 and external to 1.0 so that deployment reach can amplify or dampen the score.

Working with risk scoring methodologies

Register a methodology with lf add risk-scoring-methodology, which creates it or updates the existing methodology with the same key, and list the registered ones with lf list risk-scoring-methodology. lf export risk-scoring-methodology writes a methodology back to YAML and lf delete risk-scoring-methodology removes it. See the risk scorers guide for the full workflow.

Configuration

Properties

---

key string required

Unique identifier assigned to the entity in AI Platform.

Pattern: ^[a-zA-Z0-9_\-]+$
Max Length: 250

---

display_name string required

The risk scoring methodology’s name displayed to the user.

---

description string

Short description of the risk scoring methodology.

Default: None

---

function string required

An function expression written in Python-like math syntax (e.g. impact * (1 - metric_key)) where every variable must be covered by config_spec or metric_key.

---

metric_key string required

The variable in function that will receive the metric value.

---

config_spec array[FloatParameterSpec, IntParameterSpec, CategoricalParameterSpec]

Parameter specifications that configure this risk scoring methodology.

Default: []

Performance Risk Scoring Methodology

key: "performance_risk_scoring_methodology"
display_name: "Performance Risk Scoring Methodology"
description: >
  Converts a model accuracy metric into a risk score by treating low accuracy as
  failure and scaling it by the business impact and deployment reach of the system.
function: "(1 - accuracy) * impact * deployment_reach"
metric_key: "accuracy"
config_spec:
  - type: float
    key: "impact"
    display_name: "Impact"
    description: >
      How severely poor performance in this area affects the system. Higher values
      amplify the risk score for the same accuracy drop.
    default_value: 5.0
    min: 0.0
    max: 10.0
  - type: categorical
    key: "deployment_reach"
    display_name: "Deployment Reach"
    description: >
      Whether the system is externally user-facing or internal only. External
      deployments amplify the score; internal deployments reduce it.
    allowed_values:
      - "internal"
      - "external"
    default_value: "external"
    values_mapping:
      internal: 0.5
      external: 1.0