URL

Downloads a dataset from a remote CSV or JSONL URL. Use it to load data hosted elsewhere without keeping a local copy.

Behaviour

The URL must point directly to a .csv or .jsonl file. Filters are AND-combined and applied after the file is downloaded.

Examples

Example: Remote CSV. Loads a dataset from a public URL.

URL Dataset Source

# ...
source:
  type: "url"
  url: "https://huggingface.co/datasets/JailbreakBench/JBB-Behaviors/raw/main/data/judge-comparison.csv"

Configuration

Properties

type Literal “url” required

Remote URL dataset source.

url string required

URL pointing to a CSV or JSONL file. Must end with .csv or .jsonl extension.

filters array[FilterComparison, FilterMembership, FilterUnary]

Optional list of filters to apply after loading (AND-combined).

Related types

`FilterComparison`

Filter by Column Comparing to Value

# ...
source:
  # ...
  filters:
    - op: "equals"
      expression: "{{ sample.SemanticCategory }}"
      value: "illegal"

Properties

op enum FilterComparisonOp required

Possible FilterComparisonOp values

The comparison operator to apply.

Allowed Values:

equals
not_equals
greater_than
less_than
greater_or_equal
less_or_equal

expression string required

An expression encoding what to compare against the value.

Depending on the context, it can refer to different variables:

When filtering a dataset: it can refer to the sample and use dot or bracket notation to access the columns. If filtering a dataset with column names that are illegal under jinja substitution rules (e.g. containing spaces), use bracket notation to access the column.
When used within a task action: it can refer to the sample, the solver_output or the scores (which is a mapping between scorer keys and their corresponding score values dict).

value string, number, integer, boolean required

The value against which the expression is compared.

`FilterMembership`

Filter by Column Value Membership

# ...
source:
  # ...
  filters:
    - op: "not_in"
      expression: "{{ sample.SemanticCategory }}"
      values:
        - "copyright"
        - "chemical_biological"

Properties

op enum FilterMembershipOp required

Possible FilterMembershipOp values

The membership operator to apply.

Allowed Values:

in
not_in

expression string required

An expression encoding what to check membership against the values.

Depending on the context, it can refer to different variables:

When filtering a dataset: it can refer to the sample and use dot or bracket notation to access the columns. If filtering a dataset with column names that are illegal under jinja substitution rules (e.g. containing spaces), use bracket notation to access the column.
When used within a task action: it can refer to the sample, the solver_output or the scores (which is a mapping between scorer keys and their corresponding score values dict).

values array[string, number, boolean] required

The set of values to test membership against.

`FilterUnary`

Filter by Unary Expression

# ...
source:
  # ...
  filters:
    - op: "is_true"
      expression: "{{ sample.answer }}"

Properties

op enum FilterUnaryOp required

Possible FilterUnaryOp values

The unary operator to apply.

Allowed Values:

exists
not_exists
is_true
is_false

expression string required

An expression encoding what to apply the unary operator to.

Depending on the context, it can refer to different variables:

When filtering a dataset: it can refer to the sample and use dot or bracket notation to access the columns. If filtering a dataset with column names that are illegal under jinja substitution rules (e.g. containing spaces), use bracket notation to access the column.
When used within a task action: it can refer to the sample, the solver_output or the scores (which is a mapping between scorer keys and their corresponding score values dict).