utils.callbacks.perplexity

utils.callbacks.perplexity

callback to calculate perplexity as an evaluation metric.

Classes

Name	Description
Perplexity	Calculate perplexity as defined in https://huggingface.co/docs/transformers/en/perplexity.

Perplexity

utils.callbacks.perplexity.Perplexity(self, tokenizer, max_seq_len, stride=512)

Calculate perplexity as defined in https://huggingface.co/docs/transformers/en/perplexity. This is a custom variant that doesn’t re-tokenize the input or re-load the model.

Methods

Name	Description
compute	Compute perplexity in a fixed length sliding window across the sequence.

compute

utils.callbacks.perplexity.Perplexity.compute(model, references=None)

Compute perplexity in a fixed length sliding window across the sequence.