lf test
Test models, dataset generation, and task specifications.
Usage
lf test [OPTIONS] COMMAND [ARGS]...Subcommands
| model | Test model connection, inference and I/O adapter mapping of the model with the provided key. |
| dataset | Preview a few samples from a generated dataset defined in a YAML file or a run config. |
| task | Test a task based on its specification, extracted from a run YAML file. |
lf test model
Test model connection, inference and I/O adapter mapping of the model with the provided key.
Usage
lf test model [OPTIONS] KEYOptions
| Name | Type | Description | Default |
|---|---|---|---|
--model-input |
path | Path to a JSON file with a model input in LatticeFlow AI format.If not provided, default is used, for more details see https://docs.latticeflow.ai/integrations/assessing-your-ai-system. |
lf test dataset
Preview a few samples from a generated dataset defined in a YAML file or a run config.
Usage
lf test dataset [OPTIONS]Options
| Name | Type | Description | Default |
|---|---|---|---|
--file, -f |
path | Path to a dataset YAML file or a run config YAML file. | |
--key, -k |
text | Key of the dataset to extract from run config. Only used when --file points to a run config. |
|
--num-samples, -n |
integer range (between 1 and 10) |
The number of samples to generate. Can be at most 10. | 3 |
--verbose, -v |
boolean | Show each source sample and the input/output of the synthesizers. | False |
--show-io |
boolean | Show the model I/O of synthesizers. Only meaningful with –verbose. | False |
lf test task
Test a task based on its specification, extracted from a run YAML file.
If neither --num-samples nor --sample-ids is provided, the first sample of the dataset will be used (i.e., --num-samples 1).
Usage
lf test task [OPTIONS]Options
| Name | Type | Description | Default |
|---|---|---|---|
--file, -f |
path | Path to a single run config. | |
--spec-key, -sk |
text | Key of the task specification to test. | |
--num-samples, -n |
integer range (1 and above) |
The number of samples to test the task with. Cannot be combined with –sample-id. | |
--sample-id, -id |
text | The ID of a specific sample to test the task with. Can be repeated for multiple samples. Cannot be combined with –num-samples. |