evaluate

evaluate

Module for evaluating models.

Functions

Name	Description
evaluate	Evaluate a model on training and validation datasets.
evaluate_dataset	Helper function to evaluate a single dataset.

evaluate

evaluate.evaluate(cfg, dataset_meta)

Evaluate a model on training and validation datasets.

Parameters

Name	Type	Description	Default
cfg	DictDefault	Dictionary mapping axolotl config keys to values.	required
dataset_meta	TrainDatasetMeta	Dataset metadata containing training and evaluation datasets.	required

Returns

Name	Type	Description
	Dict[str, float]	Dictionary mapping metric names to their values.

evaluate_dataset

evaluate.evaluate_dataset(trainer, dataset, dataset_type, flash_optimum=False)

Helper function to evaluate a single dataset.

Parameters

Name	Type	Description	Default
trainer	Trainer	The trainer instance.	required
dataset	Dataset	Dataset to evaluate.	required
dataset_type	str	Type of dataset (‘train’ or ‘eval’).	required
flash_optimum	bool	Whether to use flash optimum.	False

Returns

Name	Type	Description
	Optional[Dict[str, float]]	Dictionary of metrics or None if dataset is None.