Other Types
This page collects the type aliases exported from latticeflow.core.dtypes and the supporting nested types referenced by the documented models.
Type Aliases
ChatCompletionResponseFormat = ChatCompletionResponseFormatJSONSchema | ChatCompletionResponseFormatText
ConversationItem = Message | CustomTaskInputMessage | CustomTaskOutputMessage
DType = Type | Tuple
InputMessageContent = str | List
JSONType = NoneType | int | str | bool | float | List | Mapping
LFInputMessage = ChatCompletionInputMessage | Any | list
LFMessage = ChatCompletionInputMessage | Any | list | ChatCompletionOutputMessage | list
LFModelInput = ChatCompletionInput | list | Any
LFModelOutput = OpenResponsesModelOutput | RAGCompletionOutput | ChatCompletionModelOutput | EmbeddingsModelOutput | Any
LFOutputMessage = ChatCompletionOutputMessage | Any | list
ResultDType = pd.DataFrame | NoneType | int | str | bool | float | List | Mapping | BaseModel
RuleDefinition = ExistsRuleDefinition | ThresholdRuleDefinition
RuleScope = PolicyRuleSimpleScope | PolicyRuleFinegrainedScope
TraceEvent = MessageEvent | FunctionCallEvent | ModelCallEvent | SpanBeginEvent | SpanEndEvent | CompactionEvent | ErrorEvent | CustomEvent
TraceItem = Message | FunctionCall | FunctionCallOutput | CustomTaskInputMessage | CustomTaskOutputMessage
Supporting Types
ActionRule
Properties
key string required
Key: 1-250 chars, allowed: a-z A-Z 0-9 _ -
Pattern: ^[a-zA-Z0-9_\-]+$
Max Length: 250
action ActionRuleAction required
The action to be applied to samples that match the filter.
filter FilterComparison, FilterMembership, FilterUnary required
The filter that determines which samples the action applies to.
ActionRuleAction
Allowed Values:
exclude_from_metrics
AwsBedrockConnectionConfig
Connection configuration for an agent deployed as an AWS Bedrock AgentCore harness.
Properties
connection_type Literal “aws_bedrock” required
The type of connection config.
region string required
The AWS region in which the AgentCore harness is deployed.
harness_arn string required
The ARN of the AWS Bedrock AgentCore harness to invoke.
access_key_id string, Secret required
The AWS access key ID. Provide a raw string or reference a secret.
secret_access_key string, Secret required
The AWS secret access key. Provide a raw string or reference a secret.
session_token string, Secret
The optional AWS session token. Provide a raw string or reference a secret.
Default: None
AzureFoundryConnectionConfig
Connection configuration for an agent deployed on Azure AI Foundry.
Properties
connection_type Literal “azure_foundry” required
The type of connection config.
project_endpoint string required
The endpoint URL of the Azure AI Foundry project.
agent_name string required
The name of the Azure AI Foundry agent to invoke.
tenant_id string, Secret required
The Azure service-principal tenant ID. Provide a raw string or reference a secret.
client_id string, Secret required
The Azure service-principal client ID. Provide a raw string or reference a secret.
client_secret string, Secret required
The Azure service-principal client secret. Provide a raw string or reference a secret.
BenchmarkTaskDefinitionTemplate
Properties
type Literal “benchmark_task” required
The type of task definition.
evaluated_entity_type EvaluatedEntityType required
dataset TaskDatasetTemplate
The dataset used by this task
Default: None
solver TaskSolverTemplate
The solver used by this task
Default: None
scorers array[TaskScorerTemplate] required
The scorers used by this task
trials TrialsDefinitionTemplate
Default: None
actions array[ActionRule]
The actions used by this task
Default: None
BooleanParameterSpec
Properties
type Literal “boolean” required
The type of the parameter.
key string required
The key of the parameter.
display_name string required
The display name of the parameter.
description string
The description of the parameter.
Default: None
default_value boolean
The default value to use.
Default: None
nullable boolean
Whether this parameter is nullable.
Default: False
CachePolicy
The caching policy to use for the task results in the evaluation. Supported values: - reuse - Use a cached task result if one is available (the default). Partial task results are also reused automatically - if a task is the same as another, completed task for all of its configuration except the scorers configuration, then only the scores, metrics and errors and failures related to them will be recomputed. This saves queries to the model during the solver part of the evaluation. - update - Do not use cached task results, but cache the results of the execution. - no-cache - Do not use cached task results and do not cache the results of the execution.
Allowed Values:
reuseupdateno-cache
CategoricalParameterSpec
Properties
type Literal “categorical” required
The type of parameter.
key string required
The key of the parameter.
display_name string required
The display name of the parameter.
description string
The description of the parameter.
Default: None
allowed_values array[string] required
Allowed Values
multiple boolean
Whether the parameter can have multiple values.
Default: False
default_value string
The default value to use.
Default: None
nullable boolean
Whether this parameter is nullable.
Default: False
values_mapping object
A mapping over the categorical values.
Default: None
CertificateValidationContext
Defines how server certificates should be validated.
Properties
trusted_ca string, Secret
base64 representation of PEM-encoded certificate(s).
Provide a raw base64 string or reference a secret.
For example: cat cert.pem \| base64 -w 0
Default: None
trust_chain_verification TrustChainVerification
Settings for verifying the trust chain of the server certificate.
Default: None
ClaudeManagedAgentsConnectionConfig
Connection configuration for an agent deployed as a Claude Managed Agent.
Properties
connection_type Literal “claude_managed_agents” required
The type of connection config.
agent_id string required
The identifier of the Claude managed agent to invoke.
environment_id string required
The identifier of the Claude managed-agent environment.
vault_id string required
The identifier of the Claude managed-agent vault.
api_key string, Secret required
The Anthropic API key. Provide a raw string or reference a secret.
ConfigurationDatasetGenerationError
Properties
stage Literal “configuration” required
Stage
error_type string required
The type of the error.
message string required
The specific error message that occurred during generation.
CustomHeaders
String header value must contain only printable ASCII characters.
CustomInferenceModelConfig
Client configuration for a model, that is provided manually by the user.
Properties
adapter_id string required
The ID of the model adapter to be used with this model.
connection_type Literal “custom_inference” required
The type of connection config.
run_inference_snippet string required
The code snippet to make a call to the model.
environment object required
Environment variables required to run the model client snippet. Values may reference secrets.
timeout number required
Timeout in seconds for the total runtime of the Python snippet.
DataSourceDatasetGenerationError
Properties
stage Literal “data_source” required
Stage
error_type string required
The type of the error.
message string required
The specific error message that occurred during generation.
iteration integer required
The iteration number of the data source generation that caused the error.
DatasetColumnParameterSpec
Properties
type Literal “dataset_column” required
The type of the parameter.
key string required
The key of the parameter.
display_name string required
The display name of the parameter.
description string
The description of the parameter.
Default: None
default_value string
The default value to use.
Default: None
nullable boolean
Whether this parameter is nullable.
Default: False
DatasetGenerateTokenUsagePurpose
Properties
type Literal “generate” required
The model is used as a dataset synthesizer during dataset generation.
dataset_id string
Optional. If missing the dataset was deleted.
Default: None
dataset_display_name string
Optional. If missing the dataset was deleted.
Default: None
DatasetGenerationDebugOptions
Properties
enabled boolean required
When true, the response will include a full pipeline trace for each source sample, which contains the source sample itself and the input and output at each synthesizer stage.
include_io boolean required
When true, the model input and output are included in the trace for each synthesizer call that produced I/O. Has no effect when enabled is false.
DatasetGenerationMetadata
Dataset generation metadata.
Properties
dataset_generator_id string
Dataset Generator Id
Default: None
execution_status ExecutionStatus required
dataset_generation_id string required
The dataset generation ID.
dataset_generation_request DatasetGenerationRequest required
The dataset generation request.
progress ExecutionProgress
Default: None
result_status ResultStatus
Default: None
errors array[ConfigurationDatasetGenerationError, SynthesizerDatasetGenerationError, DataSourceDatasetGenerationError]
List of errors that occurred during dataset generation.
Default: None
DatasetGenerationRequest
Properties
dataset_generator_config object required
The configuration used by the dataset generator.
num_samples integer required
The number of samples to generate. At least 1 sample must be requested.
debug DatasetGenerationDebugOptions
Default: None
DatasetMetadata
Dataset metadata.
Properties
num_rows integer required
Num Rows
columns array[string] required
Columns
download_url string required
URL to download the dataset in JSONL format.
data_version string required
Data Version
DatasetParameterSpec
Properties
type Literal “dataset” required
The type of the parameter.
key string required
The key of the parameter.
display_name string required
The display name of the parameter.
description string
The description of the parameter.
Default: None
default_value string
The default value to use.
Default: None
nullable boolean
Whether this parameter is nullable.
Default: False
DictParameterSpec
Properties
type Literal “dict” required
The type of the parameter.
value_dtype ScalarDtype required
The data type of the values in the dict.
key string required
The key of the parameter.
display_name string required
The display name of the parameter.
description string
The description of the parameter.
Default: None
default_value object
The default value to use.
Default: None
nullable boolean
Whether this parameter is nullable.
Default: False
DifyConnectionConfig
Connection configuration for an agent deployed on Dify.
Properties
connection_type Literal “dify” required
The type of connection config.
url string required
The base URL of the Dify deployment.
api_key string, Secret required
The Dify API key. Provide a raw string or reference a secret.
EvaluatedEntityType
Allowed Values:
datasetmodel
EvaluationConfig
Parameters required when starting an evaluation.
Properties
num_samples integer
The number of samples to evaluate. If not specified, all samples will be evaluated.
Default: None
subsampling Subsampling
Default: None
cache_policy CachePolicy
The caching policy to use for the task results in the evaluation. Supported values: - reuse - Use a cached task result if one is available (the default). Partial task results are also reused automatically - if a task is the same as another, completed task for all of its configuration except the scorers configuration, then only the scores, metrics and errors and failures related to them will be recomputed. This saves queries to the model during the solver part of the evaluation. - update - Do not use cached task results, but cache the results of the execution. - no-cache - Do not use cached task results and do not cache the results of the execution.
Default: reuse
trials_config TrialsConfig
Default: None
ExecutionProgress
Properties
progress number required
A progress indicator for the task result.
num_total_samples integer
The total number of samples to be processed for this task result.
Default: None
num_processed_samples integer
The number of samples already processed for this task result.
Default: None
num_samples_with_errors integer
The number of samples for which an error occurred for this task result.
Default: None
ExecutionStatus
Allowed Values:
not_startedpendingcancelledfinished
FilterComparison
Properties
op FilterComparisonOp required
expression string required
An expression encoding what to compare against the value.
Depending on the context, it can refer to different variables:
- When filtering a dataset: it can refer to the
sampleand use dot or bracket notation to access the columns. If filtering a dataset with column names that are illegal under jinja substitution rules (e.g. containing spaces), use bracket notation to access the column. - When used within a task action: it can refer to the
sample, thesolver_outputor thescores(which is a mapping between scorer keys and their corresponding score values dict).
value string, number, integer, boolean required
The value against which the expression is compared.
FilterComparisonOp
The comparison operator to apply.
Allowed Values:
equalsnot_equalsgreater_thanless_thangreater_or_equalless_or_equal
FilterMembership
Properties
op FilterMembershipOp required
expression string required
An expression encoding what to check membership against the values.
Depending on the context, it can refer to different variables:
- When filtering a dataset: it can refer to the
sampleand use dot or bracket notation to access the columns. If filtering a dataset with column names that are illegal under jinja substitution rules (e.g. containing spaces), use bracket notation to access the column. - When used within a task action: it can refer to the
sample, thesolver_outputor thescores(which is a mapping between scorer keys and their corresponding score values dict).
values array[string, number, boolean] required
The set of values to test membership against.
FilterMembershipOp
The membership operator to apply.
Allowed Values:
innot_in
FilterUnary
Properties
op FilterUnaryOp required
expression string required
An expression encoding what to apply the unary operator to.
Depending on the context, it can refer to different variables:
- When filtering a dataset: it can refer to the
sampleand use dot or bracket notation to access the columns. If filtering a dataset with column names that are illegal under jinja substitution rules (e.g. containing spaces), use bracket notation to access the column. - When used within a task action: it can refer to the
sample, thesolver_outputor thescores(which is a mapping between scorer keys and their corresponding score values dict).
FilterUnaryOp
The unary operator to apply.
Allowed Values:
existsnot_existsis_trueis_false
FloatParameterSpec
Properties
type Literal “float” required
The type of the parameter.
key string required
The key of the parameter.
display_name string required
The display name of the parameter.
description string
The description of the parameter.
Default: None
min number
The minimum value of the parameter.
Default: None
max number
The maximum value of the parameter.
Default: None
default_value number
The default value to use.
Default: None
nullable boolean
Whether this parameter is nullable.
Default: False
ImageDetail
Allowed Values:
lowhighauto
InputVideoContent
A content block representing a video input to the model.
Properties
type Literal “input_video”
The type of the input content. Always input_video.
Default: input_video
video_url string required
A base64 or remote url that resolves to a video file.
IntParameterSpec
Properties
type Literal “int” required
The type of the parameter.
key string required
The key of the parameter.
display_name string required
The display name of the parameter.
description string
The description of the parameter.
Default: None
min integer
The minimum value of the parameter.
Default: None
max integer
The maximum value of the parameter.
Default: None
default_value integer
The default value to use.
Default: None
nullable boolean
Whether this parameter is nullable.
Default: False
IntegrationModelProviderId
The internal identifiers for all model providers known by the system.
Allowed Values:
anthropicfireworksgeminilatticeflownovitaopenaisambanovatogether
LangSmithConnectionConfig
Connection configuration for an agent deployed on LangSmith / LangGraph Platform.
Properties
connection_type Literal “langsmith” required
The type of connection config.
deploy_url string required
The base URL of the LangGraph deployment.
assistant_id string required
The identifier of the LangGraph assistant to invoke.
api_key string, Secret required
The LangSmith API key. Provide a raw string or reference a secret.
ListParameterSpec
Properties
type Literal “list” required
The type of the parameter.
dtype ScalarDtype required
The data type of the elements in the list.
key string required
The key of the parameter.
display_name string required
The display name of the parameter.
description string
The description of the parameter.
Default: None
default_value array[Any]
The default value to use.
Default: None
nullable boolean
Whether this parameter is nullable.
Default: False
MLTask
The type of machine learning task to be performed.
Allowed Values:
chat_completionembeddingscustom
MaxAggregator
Aggregates numeric scores by taking the maximum value.
Properties
function Literal “max” required
Function
score_name string
The name to give to the aggregated score.
Default: None
MeanAggregator
Aggregates numeric scores by computing the mean.
Properties
function Literal “mean” required
Function
score_name string
The name to give to the aggregated score.
Default: None
MinAggregator
Aggregates numeric scores by taking the minimum value.
Properties
function Literal “min” required
Function
score_name string
The name to give to the aggregated score.
Default: None
ModelCustomConnectionConfig
Connection configuration for a model, that is provided manually by the user.
Properties
connection_type Literal “custom_connection” required
The type of connection config.
adapter_id string required
The ID of the model adapter to be used with this model.
url string required
The model endpoint URL.
api_key string, Secret
The key to be passed as the authorization header (Authorization: Bearer API_KEY). Provide a raw string (deprecated) or reference a secret.
Default: None
model_key string
This field is used in case the model is not specified in the URL but in the body instead. For the “openai” adapter, this will be passed as the “model” parameter. For custom adapters, this value is available as model_info.model_key.
Default: None
tls_context TLSContext
TLS configuration for secure connections to the model endpoint.
Default: None
custom_headers object
Additional headers to include in requests to the model endpoint. Values may reference secrets.
Default: None
ModelParameterSpec
Properties
type Literal “model” required
The type of the parameter.
key string required
The key of the parameter.
display_name string required
The display name of the parameter.
description string
The description of the parameter.
Default: None
default_value string
The default value to use.
Default: None
nullable boolean
Whether this parameter is nullable.
Default: False
ModelProviderConnectionConfig
Connection configuration for a model, that is retrieved from a well-known provider integrated with the system.
Properties
connection_type Literal “provider_connection” required
The type of connection config.
provider_id ModelProviderId required
The id of the model provider.
model_key string required
A key used to identify the model in the external provider.
ModelProviderId
The internal identifiers for all model providers known by the system.
ModelTokenUsageInstance
Properties
purpose DatasetGenerateTokenUsagePurpose, TaskResultTokenUsagePurpose required
Purpose
prompt_tokens integer required
Prompt Tokens
completion_tokens integer required
Completion Tokens
created_at integer required
Unix timestamp (in seconds).
ModelUsageStats
An object that contains the model usage summary for the task result.
Properties
num_samples integer required
Num Samples
num_completion_tokens integer
Num Completion Tokens
Default: None
num_prompt_tokens integer
Num Prompt Tokens
Default: None
PassAtKAggregator
Aggregates binary (True/False) scores using the pass@k estimator. Estimates the probability that at least one of k independent attempts will succeed, computed as 1 - (1 - p)^k where p is the empirical pass rate across trials.
Properties
function Literal “pass@k” required
Function
k integer required
The number of independent attempts in the scenario being modelled.
score_name string
The name to give to the aggregated score.
Default: None
PassPowerKAggregator
Aggregates binary (True/False) scores using the pass^k estimator. Estimates the probability that an agent would succeed on all k independent attempts, computed as p^k where p is the empirical pass rate across trials.
Properties
function Literal “pass^k” required
Function
k integer required
The number of independent attempts in the scenario being modelled.
score_name string
The name to give to the aggregated score.
Default: None
PlaceholderConnectionConfig
Connection configuration for a placeholder external model that performs no inference and returns an empty completion.
Properties
connection_type Literal “placeholder” required
The type of connection config.
RepeatabilityConfig
Configuration for a repeatability task result.
Properties
num_runs integer required
Number of times to run the task.
ResultStatus
Allowed Values:
succeededfailed
ScalarDtype
The scalar data type.
Allowed Values:
stringintegerfloatboolean
ScoreAggregator
Aggregation configuration for one or more score keys.
Properties
score_name string required
The name of the score that will be aggregated.
aggregator MeanAggregator, MinAggregator, MaxAggregator, PassAtKAggregator, PassPowerKAggregator required
Aggregator
ScorerPurpose
Allowed Values:
scoreqa
StoredDataset
Properties
display_name string required
The display name of the dataset.
description string
An optional description of the dataset.
Default: None
long_description string
Long description of the dataset in Markdown format.
Default: None
key string required
Key: 1-250 chars, allowed: a-z A-Z 0-9 _ -
Pattern: ^[a-zA-Z0-9_\-]+$
Max Length: 250
id string required
Id
dataset_metadata DatasetMetadata
Dataset metadata.
Default: None
dataset_generation_metadata DatasetGenerationMetadata
Dataset generation metadata.
Default: None
created_at integer
Unix timestamp (in seconds).
Default: None
updated_at integer
Unix timestamp (in seconds).
Default: None
tags array[StoredTag]
Tags associated with the dataset.
Default: []
StoredModel
Properties
id string required
Id
display_name string required
The name of the Model.
key string required
Unique identifier assigned to the entity in AI GO!.
Pattern: ^((together|gemini|openai|fireworks|sambanova|anthropic|novita|latticeflow)\$)?[a-zA-Z0-9_-]+$
Max Length: 250
description string
Description
Default: None
rate_limit integer
The maximum allowed number of requests per minute.
Default: None
max_concurrent_requests integer
The maximum number of concurrent inference requests.
Default: None
task MLTask required
config ModelCustomConnectionConfig, CustomInferenceModelConfig, ModelProviderConnectionConfig, LangSmithConnectionConfig, AzureFoundryConnectionConfig, AwsBedrockConnectionConfig, ClaudeManagedAgentsConnectionConfig, DifyConnectionConfig, PlaceholderConnectionConfig required
The configuration for connecting to the model.
adapter_id string required
The ID of the model adapter to be used with this model.
usage TokenUsageWithBreakdown required
created_at integer
Unix timestamp (in seconds).
Default: None
updated_at integer
Unix timestamp (in seconds).
Default: None
StoredTag
Properties
id string required
Id
value string required
The text value of the tag.
color string required
The color (#RRGGBB or #RGB) associated with the tag, used for UI representation.
Pattern: ^#([0-9a-fA-F]{6}|[0-9a-fA-F]{3})$
StoredTask
Properties
id string required
Id
key string required
Key: 1-250 chars, allowed: a-z A-Z 0-9 _ -
Pattern: ^[a-zA-Z0-9_\-]+$
Max Length: 250
display_name string required
The display name of the task.
description string required
The description of the task.
long_description string
Long description of the task in Markdown format.
Default: None
tasks array[MLTask]
The ML tasks for which the task is applicable.
Default: []
config_spec array[FloatParameterSpec, IntParameterSpec, BooleanParameterSpec, StringParameterSpec, ModelParameterSpec, DatasetParameterSpec, DatasetColumnParameterSpec, ListParameterSpec, DictParameterSpec, CategoricalParameterSpec] required
Config Spec
definition BenchmarkTaskDefinitionTemplate, SystemTaskDefinitionTemplate required
Definition
provider TaskProvider required
The provider of the task.
tags array[StoredTag] required
Tags associated with the task.
created_at integer
Unix timestamp (in seconds).
Default: None
updated_at integer
Unix timestamp (in seconds).
Default: None
StringKind
Specifies the kind of string parameter.
Allowed Values:
freeformpythonjinja
StringParameterExample
Properties
value string required
The example value for the string parameter.
display_name string required
The display name of the example.
StringParameterSpec
Properties
type Literal “string” required
The type of the parameter.
key string required
The key of the parameter.
display_name string required
The display name of the parameter.
description string
The description of the parameter.
Default: None
default_value string
The default value of the parameter.
Default: None
nullable boolean
Whether this parameter is nullable.
Default: False
string_kind StringKind
Default: freeform
examples array[StringParameterExample]
Examples for the string parameter.
Default: None
Subsampling
The subsampling strategy to use when selecting samples for evaluation. Supported values: - head - Select the first N samples. - random - Select N random samples. The random seed is fixed for reproducibility. If not specified, defaults to ‘head’.
Allowed Values:
headrandom
SynthesizerDatasetGenerationError
Properties
stage Literal “synthesizer” required
Stage
error_type string required
The type of the error.
message string required
The specific error message that occurred during generation.
source_sample object required
The source sample for which an error occurred.
synthesizer_index integer required
The index of the synthesizer that caused the error.
SystemTaskDefinitionTemplate
Properties
type Literal “system_task” required
The type of task definition.
compute_evidence_snippet string required
Python source code defining a def compute_evidence() function (sync or async) that returns metrics and optional metadata.
TaskDatasetTemplate
The dataset that will be used to evaluate the model.
Properties
id string required
Id
TaskMetricTemplate
Properties
key string
The key of the metric.
Default: None
type string required
The type of metric.
TaskProvider
Allowed Values:
latticeflowuser
TaskResultErrorStage
Allowed Values:
configurationdatasetsolverscoremetricaction
TaskResultTokenUsagePurpose
Properties
type Type required
The purpose for which the model tokens were consumed: - test: The model is under evaluation (model under test / solver). - judge: The model acts as an LLM-as-a-judge
evaluation_id string
Optional. If missing the task result was deleted.
Default: None
task_result_id string
Optional. If missing the task result was deleted.
Default: None
task_result_display_name string
Optional. If missing the task result was deleted.
Default: None
TaskScorerTemplate
Properties
key string
The key of the scorer.
Default: None
type string required
The type of the scorer.
display_name string
The display name of the scorer.
Default: None
purpose ScorerPurpose
The purpose of the scorer.
Default: score
metrics array[TaskMetricTemplate]
The metrics associated with this scorer, which will produce per-task metrics.
Default: None
TaskSolverTemplate
Properties
type string required
The type of the solver.
TextContent
A text content.
Properties
type Literal “text”
Type
Default: text
text string required
Text
TokenUsageWithBreakdown
Properties
prompt_tokens integer required
Prompt Tokens
completion_tokens integer required
Completion Tokens
breakdown array[ModelTokenUsageInstance] required
Individual token usage entries, each tagged with a purpose (test, judge, audit, or generate). The aggregate totals in the parent TokenUsage fields are the sum of all entries in this list.
TrialsConfig
Configuration for trials in a task result/specification. Only relevant for benchmark tasks.
Properties
num_trials integer required
Number of trials to run per sample.
TrialsDefinitionTemplate
Configuration for trials in a benchmark task definition template.
Properties
num_trials integer, string required
Number of trials to run per sample.
score_aggregators array[ScoreAggregator]
Score aggregators that compute an aggregated score given the score values for the different trials. Scores with no matching aggregator default to mean for numeric and boolean values (for other dtypes, no default aggregation is computed).
Default: None
TrustChainVerification
How to trust the CA trust chain.
verify_trust_chain(default) will verify the server certificate against the configured CA trust.accept_untrustedwill not perform server certificate verification. NOTE: This is a security hazard and should be avoided.
Allowed Values:
verify_trust_chainaccept_untrusted
Type
The purpose for which the model tokens were consumed: - test: The model is under evaluation (model under test / solver). - judge: The model acts as an LLM-as-a-judge
Allowed Values:
testjudge