Datasets
A dataset generator’s dataset produces the initial source samples that its synthesizers then transform into the final dataset. Every declarative dataset generator declares exactly one dataset under its definition, and each synthesizer runs once per source sample it yields.
Datasets are configured with a type discriminator. The available types are documented below.
A generator dataset is not the same as a dataset source. It seeds the generation pipeline of a single dataset generator; it is an implementation detail of that generator, not a standalone dataset.
Empty
Emits a fixed number of empty source samples. Use it when the synthesizers generate everything from scratch (e.g. an LLM driven purely by the config) and need no seed data.
Example: Generate from config only. An empty source drives an LLM synthesizer that produces questions about a config-provided topic.
Empty Dataset
# ...
definition:
# ...
dataset:
type: "empty"Configuration
Properties
type Literal “empty” required
The type of the dataset.
num_samples integer, TemplateValue
The number of (empty) source samples returned.
Default: 1
Dataset Samples
Emits the samples of an existing dataset, one source sample per dataset row. Use it to synthesize new columns from, or variations of, an existing dataset.
Example: Seed from an existing dataset. Each row of the referenced dataset becomes a source sample.
Dataset Samples
# ...
config_spec:
- key: "source_dataset_key"
type: "dataset"
display_name: "Source Dataset"
default_value: "fairllm_sensitive_attributes-country"
description: "Dataset containing country entries to generate user profiles from."
definition:
# ...
dataset:
type: "dataset_samples"
key: "<< config.source_dataset_key >>"Configuration
Properties
type Literal “dataset_samples” required
The type of the dataset.
key TemplateValue required
The key of the dataset whose samples should be used.
random_seed integer, TemplateValue
The random seed to use for dataset generation. If not provided, the processing order will be sequential and deterministic.
Default: None
Inline Samples
Emits samples defined inline in the config. Use it for small, fixed seed sets that live alongside the generator instead of in a separate dataset.
Example: Inline seed rows. A handful of topic/style pairs defined directly in the generator config.
Inline Samples
# ...
definition:
# ...
dataset:
type: "inline_samples"
samples:
- topic: "Biology"
style: "Multiple choice"
- topic: "Chemistry"
style: "Open-ended"
- topic: "Physics"
style: "Proofs"Configuration
Properties
type Literal “inline_samples” required
The type of the dataset.
samples array[object] required
The samples to use as the data source.
Task Result
Emits the sample evidence of a task result, one source sample per sample of the task result. Use it to build a dataset from the outcome of an evaluation, e.g. the samples a model answered incorrectly. Each source sample has the fields sample (the original sample), solver_output (None if not available), scores (scorer key to values and metadata) and errors.
Example: Collect failed samples. The task result is selected through a task_result config parameter, and the AI app that contains it through a string parameter.
Task Result
# ...
config_spec:
- key: "ai_app_key"
type: "string"
display_name: "AI App Key"
default_value: "dataset-generator-task-result"
description: "Key of the AI app that contains the task result."
- key: "task_result_key"
type: "task_result"
display_name: "Task Result"
default_value: "multilingual-trivia"
description: "Task result whose failed samples are collected."
definition:
dataset:
type: "task_result"
key: "<< config.task_result_key >>"
ai_app_key: "<< config.ai_app_key >>"Configuration
Properties
type Literal “task_result” required
The type of the dataset.
key TemplateValue required
The key of the task result whose samples evidences will be used as the data source.
ai_app_key TemplateValue required
The key of the AI app that contains the task result.
Dataset Sample Combinations
Emits the cartesian product of samples from multiple datasets, one source sample per combination. Use it to cover every pairing of independent dimensions (e.g. every city x language). Each source sample holds the columns of all the combined datasets, merged in the order they are listed under keys, so on a column collision the last dataset wins.
Example: All city x language pairs. Combines a cities dataset and a languages dataset into one source sample per pairing.
Dataset Sample Combinations
# ...
config_spec:
- key: "cities_dataset_key"
type: "dataset"
display_name: "Cities Dataset"
default_value: "cities_dataset"
description: "Dataset containing the cities to generate QA samples from."
- key: "languages_dataset_key"
type: "dataset"
display_name: "Languages Dataset"
default_value: "languages_dataset"
description: "Dataset containing the languages to generate QA samples in."
- key: "synthesizer_model_key"
type: "model"
display_name: "Synthesizer Model"
description: "Model used to generate QA samples."
definition:
# ...
dataset:
type: "dataset_sample_combinations"
keys:
- "<< config.cities_dataset_key >>"
- "<< config.languages_dataset_key >>"The two combined datasets both define a language column - the local language of the city, and the language to write the question in:
country,city,language
Germany,Berlin,German
France,Paris,French
Japan,Tokyo,Japaneselanguage
English
French
GermanThe 3 cities and 3 languages yield 9 source samples, each holding the columns of both datasets. languages_dataset is listed last, so its language value wins over the one of cities_dataset. The first three combinations are:
Source Samples
| country | city | language |
| :------ | :----- | :------- |
| Germany | Berlin | English |
| France | Paris | French |
| Japan | Tokyo | German |The language of cities_dataset (German for Berlin, Japanese for Tokyo) never reaches the synthesizers - it is overwritten by languages_dataset. Rename one of the columns to keep both values.
Combinations are enumerated in a deterministic, fair order rather than as a nested loop, so a generator that stops after num_samples samples still covers every value of every dataset. Set random_seed to shuffle the datasets before combining.
Configuration
Properties
type Literal “dataset_sample_combinations” required
The type of the dataset.
keys array[TemplateValue] required
The keys of the datasets whose samples should be used to generate combinations.
random_seed integer, TemplateValue
The random seed to use for dataset generation. If not provided, the processing order will be sequential and deterministic.
Default: None