cli.evaluate
cli.evaluate
CLI to run evaluation on a model.
Functions
| Name | Description |
|---|---|
| do_cli | Parses axolotl config, CLI args, and calls do_evaluate. |
| do_evaluate | Evaluates a transformers model by first loading the dataset(s) specified in the |
do_cli
cli.evaluate.do_cli(config=Path('examples/'), **kwargs)Parses axolotl config, CLI args, and calls do_evaluate.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| config | Union[Path, str] | Path to axolotl config YAML file. |
Path('examples/') |
| kwargs | Additional keyword arguments to override config file values. | {} |
do_evaluate
cli.evaluate.do_evaluate(cfg, cli_args)Evaluates a transformers model by first loading the dataset(s) specified in the
axolotl config, and then calling axolotl.evaluate.evaluate, which computes
evaluation metrics on the given dataset(s) and writes them to disk.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| cfg | DictDefault | Dictionary mapping axolotl config keys to values. |
required |
| cli_args | TrainerCliArgs | CLI arguments. | required |