Drop Columns

Removes the named columns from each output sample and keeps all other columns. Columns that are not present in a sample are silently ignored. Use this to strip intermediate fields (for example, columns added by an earlier synthesizer) before the dataset is written.

Output

One output sample per source sample, with the listed columns removed.

Examples

Example: Strip intermediate fields. Removes the claims and raw_response columns added by an earlier synthesizer before the dataset is written.

config_spec:
  - type: "dataset"
    key: "dataset_key"
    display_name: "Dataset"
    default_value: "extracted-claims"
definition:
  data_source:
    type: "dataset_samples"
    dataset_key: "<< config.dataset_key >>"
  synthesizers:
    - type: "drop_columns"
      columns: [ "claims", "raw_response" ]

Configuration

Properties

---

type Literal “drop_columns” required

The type of the synthesizer.

---

columns array[string] required

The list of column names to remove from each output sample. All other columns are kept. Specified columns that are not present in the sample are silently ignored.