Risk Policies

Risk policies associate a risk scorer with an evaluation scope and aggregation strategy. They let you continuously monitor the risk level of your AI app across evaluations. Consult the Risk Policies guide for an overview and quickstart.

Risk Policy Overview

Properties

---

key string required

Unique identifier assigned to the entity in AI GO!.

Pattern: ^[a-zA-Z0-9_\-]+$
Max Length: 250

---

display_name string required

The risk policy’s name displayed to the user.

---

description string

Short description of the risk policy.

Default: None

---

domain string required

The domain this risk policy belongs to.

---

risk_scorer_key string required

The key of the risk scorer used by this policy.

---

scope PolicyRuleSimpleScope, PolicyRuleFinegrainedScope required

The scope of metrics this policy operates on.

---

config object

Configuration values for the risk scorer’s config_spec parameters.

Default: {}

---

aggregation string required

Aggregation method used to combine risk scores across the scope.

---

metric string required

The metric key this policy filters on when evaluating the risk score.

Risk Policies

policies:

- key: "quality_internal"
  display_name: "Quality Risk Policy (Internal)"
  description: >
    Monitors model accuracy on the QA evaluation for internally deployed systems.
  domain: "quality"
  risk_scorer_key: "performance_risk_scorer"
  scope:
    evaluation_keys: ["qa_evaluation"]
  config:
    impact: 4.0
    deployment_reach: "internal"
  metric: "accuracy"
  aggregation: "mean"

- key: "quality_external"
  display_name: "Quality Risk Policy (External)"
  description: >
    Monitors model accuracy on the QA evaluation for externally deployed systems.
    External deployment amplifies the risk score relative to the internal policy.
  domain: "quality"
  risk_scorer_key: "performance_risk_scorer"
  scope:
    evaluation_keys: ["qa_evaluation"]
  config:
    impact: 4.0
    deployment_reach: "external"
  metric: "accuracy"
  aggregation: "mean"

Definitions

PolicyRuleSimpleScope

The scope of the policy rule.

The available choices and their meanings are: - all_latest - All metrics from task results in the latest evaluations per evaluation key are included.

Allowed Values:

• all_latest

PolicyRuleFinegrainedScope

A precise scope definition that specifies the source of the metric values used by the policy rules, including the relevant evaluations, task specifications, scorers, and metrics.

Properties

---

evaluation_keys array[string]

The keys of the evaluations to take into account.

Default: None

---

task_specification_keys array[string]

The keys of the task specifications to take into account.

Default: None

---

scorer_keys array[string]

The keys of the scorers to take into account.

Default: None

---

metric_keys array[string]

The keys of the metrics to take into account.

Default: None