utils.freeze
utils.freeze
module to freeze/unfreeze parameters by name
Classes
| Name | Description |
|---|---|
| LayerNamePattern | Represents a regex pattern for layer names, potentially including a parameter index range. |
LayerNamePattern
utils.freeze.LayerNamePattern(self, pattern)Represents a regex pattern for layer names, potentially including a parameter index range.
Methods
| Name | Description |
|---|---|
| match | Checks if the given layer name matches the regex pattern. |
match
utils.freeze.LayerNamePattern.match(name)Checks if the given layer name matches the regex pattern.
Parameters: - name (str): The layer name to check.
Returns: - bool: True if the layer name matches the pattern, False otherwise.
Functions
| Name | Description |
|---|---|
| freeze_layers_except | Freezes all layers of the given model except for the layers that match given regex patterns. |
freeze_layers_except
utils.freeze.freeze_layers_except(model, regex_patterns)Freezes all layers of the given model except for the layers that match given regex patterns. Periods in the patterns are treated as literal periods, not as wildcard characters.
Parameters: - model (nn.Module): The PyTorch model to be modified. - regex_patterns (list of str): List of regex patterns to match layer names to keep unfrozen. Note that you cannot use a dot as a wildcard character in the patterns since it is reserved for separating layer names. Also, to match the entire layer name, the pattern should start with “^” and end with “\(", otherwise it will match any part of the layer name. The range pattern part is optional and it is not compiled as a regex pattern which means you must put "\)” before the range pattern if you want to match the entire layer name. E.g., [“^model.embed_tokens.weight\([:32000]", "layers.2[0-9]+.block_sparse_moe.gate.[a-z]+\)”]
Returns: None; the model is modified in place.