cli.preprocess
cli.preprocess
CLI to run preprocessing of a dataset.
Functions
| Name | Description |
|---|---|
| do_cli | Parses axolotl config, CLI args, and calls do_preprocess. |
| do_preprocess | Preprocesses dataset specified in axolotl config. |
do_cli
cli.preprocess.do_cli(config=Path('examples/'), **kwargs)Parses axolotl config, CLI args, and calls do_preprocess.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| config | Union[Path, str] | Path to axolotl config YAML file. |
Path('examples/') |
| kwargs | Additional keyword arguments to override config file values. | {} |
do_preprocess
cli.preprocess.do_preprocess(cfg, cli_args)Preprocesses dataset specified in axolotl config.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| cfg | DictDefault | Dictionary mapping axolotl config keys to values. |
required |
| cli_args | PreprocessCliArgs | Preprocessing-specific CLI arguments. | required |