Configuration

The base class [PretrainedConfig](/docs/transformers/v4.30.0/en/main_classes/configuration#transformers.PretrainedConfig) implements the common methods for loading/saving a configuration either from a local file or directory, or from a pretrained model configuration provided by the library (downloaded from HuggingFace’s AWS S3 repository).

Each derived config class implements model specific attributes. Common attributes present in all config classes are: `hidden_size`, `num_attention_heads`, and `num_hidden_layers`. Text models further implement: `vocab_size`.

### class transformers.PretrainedConfig

[](#transformers.PretrainedConfig)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/configuration_utils.py#L49)

( \*\*kwargs )

Base class for all configuration classes. Handles a few parameters common to all models’ configurations as well as methods for loading/downloading/saving configurations.

A configuration file can be loaded and saved to disk. Loading the configuration file and using this file to initialize a model does **not** load the model weights. It only affects the model’s configuration.

Class attributes (overridden by derived classes):

-   **model\_type** (`str`) — An identifier for the model type, serialized into the JSON file, and used to recreate the correct object in [AutoConfig](/docs/transformers/v4.30.0/en/model_doc/auto#transformers.AutoConfig).
-   **is\_composition** (`bool`) — Whether the config class is composed of multiple sub-configs. In this case the config has to be initialized from two or more configs of type [PretrainedConfig](/docs/transformers/v4.30.0/en/main_classes/configuration#transformers.PretrainedConfig) like: [EncoderDecoderConfig](/docs/transformers/v4.30.0/en/model_doc/encoder-decoder#transformers.EncoderDecoderConfig) or [~RagConfig](/docs/transformers/v4.30.0/en/model_doc/rag#transformers.RagConfig).
-   **keys\_to\_ignore\_at\_inference** (`List[str]`) — A list of keys to ignore by default when looking at dictionary outputs of the model during inference.
-   **attribute\_map** (`Dict[str, str]`) — A dict that maps model specific attribute names to the standardized naming of attributes.

Common attributes (present in all subclasses):

-   **vocab\_size** (`int`) — The number of tokens in the vocabulary, which is also the first dimension of the embeddings matrix (this attribute may be missing for models that don’t have a text modality like ViT).
-   **hidden\_size** (`int`) — The hidden size of the model.
-   **num\_attention\_heads** (`int`) — The number of attention heads used in the multi-head attention layers of the model.
-   **num\_hidden\_layers** (`int`) — The number of blocks in the model.

#### push\_to\_hub

[](#transformers.PretrainedConfig.push_to_hub)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/utils/hub.py#L734)

( repo\_id: struse\_temp\_dir: typing.Optional\[bool\] = Nonecommit\_message: typing.Optional\[str\] = Noneprivate: typing.Optional\[bool\] = Noneuse\_auth\_token: typing.Union\[bool, str, NoneType\] = Nonemax\_shard\_size: typing.Union\[int, str, NoneType\] = '10GB'create\_pr: bool = Falsesafe\_serialization: bool = False\*\*deprecated\_kwargs )

Upload the configuration file to the 🤗 Model Hub while synchronizing a local clone of the repo in `repo_path_or_name`.

Examples:

```
from transformers import AutoConfig

config = AutoConfig.from_pretrained("bert-base-cased")


config.push_to_hub("my-finetuned-bert")


config.push_to_hub("huggingface/my-finetuned-bert")```

#### dict\_torch\_dtype\_to\_str

[](#transformers.PretrainedConfig.dict_torch_dtype_to_str)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/configuration_utils.py#L907)

( d: typing.Dict\[str, typing.Any\] )

Checks whether the passed dictionary and its nested dicts have a _torch\_dtype_ key and if it’s not None, converts torch.dtype to a string of just the type. For example, `torch.float32` get converted into _“float32”_ string, which can then be stored in the json format.

#### from\_dict

[](#transformers.PretrainedConfig.from_dict)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/configuration_utils.py#L677)

( config\_dict: typing.Dict\[str, typing.Any\]\*\*kwargs ) → [PretrainedConfig](/docs/transformers/v4.30.0/en/main_classes/configuration#transformers.PretrainedConfig)

Parameters

-   [](#transformers.PretrainedConfig.from_dict.config_dict)**config\_dict** (`Dict[str, Any]`) — Dictionary that will be used to instantiate the configuration object. Such a dictionary can be retrieved from a pretrained checkpoint by leveraging the [get\_config\_dict()](/docs/transformers/v4.30.0/en/main_classes/configuration#transformers.PretrainedConfig.get_config_dict) method.
-   [](#transformers.PretrainedConfig.from_dict.kwargs)**kwargs** (`Dict[str, Any]`) — Additional parameters from which to initialize the configuration object.

The configuration object instantiated from those parameters.

Instantiates a [PretrainedConfig](/docs/transformers/v4.30.0/en/main_classes/configuration#transformers.PretrainedConfig) from a Python dictionary of parameters.

#### from\_json\_file

[](#transformers.PretrainedConfig.from_json_file)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/configuration_utils.py#L731)

( json\_file: typing.Union\[str, os.PathLike\] ) → [PretrainedConfig](/docs/transformers/v4.30.0/en/main_classes/configuration#transformers.PretrainedConfig)

Parameters

-   [](#transformers.PretrainedConfig.from_json_file.json_file)**json\_file** (`str` or `os.PathLike`) — Path to the JSON file containing the parameters.

The configuration object instantiated from that JSON file.

Instantiates a [PretrainedConfig](/docs/transformers/v4.30.0/en/main_classes/configuration#transformers.PretrainedConfig) from the path to a JSON file of parameters.

Instantiate a [PretrainedConfig](/docs/transformers/v4.30.0/en/main_classes/configuration#transformers.PretrainedConfig) (or a derived class) from a pretrained model configuration.

Examples:

```

config = BertConfig.from_pretrained(
    "bert-base-uncased"
)  
config = BertConfig.from_pretrained(
    "./test/saved_model/"
)  
config = BertConfig.from_pretrained("./test/saved_model/my_configuration.json")
config = BertConfig.from_pretrained("bert-base-uncased", output_attentions=True, foo=False)
assert config.output_attentions == True
config, unused_kwargs = BertConfig.from_pretrained(
    "bert-base-uncased", output_attentions=True, foo=False, return_unused_kwargs=True
)
assert config.output_attentions == True
assert unused_kwargs == {"foo": False}```

#### get\_config\_dict

[](#transformers.PretrainedConfig.get_config_dict)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/configuration_utils.py#L556)

( pretrained\_model\_name\_or\_path: typing.Union\[str, os.PathLike\]\*\*kwargs ) → `Tuple[Dict, Dict]`

Parameters

-   [](#transformers.PretrainedConfig.get_config_dict.pretrained_model_name_or_path)**pretrained\_model\_name\_or\_path** (`str` or `os.PathLike`) — The identifier of the pre-trained checkpoint from which we want the dictionary of parameters.

Returns

`Tuple[Dict, Dict]`

The dictionary(ies) that will be used to instantiate the configuration object.

From a `pretrained_model_name_or_path`, resolve to a dictionary of parameters, to be used for instantiating a [PretrainedConfig](/docs/transformers/v4.30.0/en/main_classes/configuration#transformers.PretrainedConfig) using `from_dict`.

#### register\_for\_auto\_class

[](#transformers.PretrainedConfig.register_for_auto_class)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/configuration_utils.py#L919)

( auto\_class = 'AutoConfig' )

Parameters

-   [](#transformers.PretrainedConfig.register_for_auto_class.auto_class)**auto\_class** (`str` or `type`, _optional_, defaults to `"AutoConfig"`) — The auto class to register this new configuration with.

Register this class with a given auto class. This should only be used for custom configurations as the ones in the library are already mapped with `AutoConfig`.

This API is experimental and may have some slight breaking changes in the next releases.

#### save\_pretrained

[](#transformers.PretrainedConfig.save_pretrained)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/configuration_utils.py#L423)

( save\_directory: typing.Union\[str, os.PathLike\]push\_to\_hub: bool = False\*\*kwargs )

Parameters

-   [](#transformers.PretrainedConfig.save_pretrained.save_directory)**save\_directory** (`str` or `os.PathLike`) — Directory where the configuration JSON file will be saved (will be created if it does not exist).
-   [](#transformers.PretrainedConfig.save_pretrained.push_to_hub)**push\_to\_hub** (`bool`, _optional_, defaults to `False`) — Whether or not to push your model to the Hugging Face model hub after saving it. You can specify the repository you want to push to with `repo_id` (will default to the name of `save_directory` in your namespace). kwargs — Additional key word arguments passed along to the [push\_to\_hub()](/docs/transformers/v4.30.0/en/internal/image_processing_utils#transformers.ImageProcessingMixin.push_to_hub) method.

Save a configuration object to the directory `save_directory`, so that it can be re-loaded using the [from\_pretrained()](/docs/transformers/v4.30.0/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained) class method.

#### to\_dict

[](#transformers.PretrainedConfig.to_dict)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/configuration_utils.py#L798)

( ) → `Dict[str, Any]`

Dictionary of all the attributes that make up this configuration instance.

Serializes this instance to a Python dictionary.

#### to\_diff\_dict

[](#transformers.PretrainedConfig.to_diff_dict)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/configuration_utils.py#L759)

( ) → `Dict[str, Any]`

Dictionary of all the attributes that make up this configuration instance,

Removes all attributes from config which correspond to the default config attributes for better readability and serializes to a Python dictionary.

#### to\_json\_file

[](#transformers.PretrainedConfig.to_json_file)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/configuration_utils.py#L845)

( json\_file\_path: typing.Union\[str, os.PathLike\]use\_diff: bool = True )

Parameters

-   [](#transformers.PretrainedConfig.to_json_file.json_file_path)**json\_file\_path** (`str` or `os.PathLike`) — Path to the JSON file in which this configuration instance’s parameters will be saved.
-   [](#transformers.PretrainedConfig.to_json_file.use_diff)**use\_diff** (`bool`, _optional_, defaults to `True`) — If set to `True`, only the difference between the config instance and the default `PretrainedConfig()` is serialized to JSON file.

Save this instance to a JSON file.

#### to\_json\_string

[](#transformers.PretrainedConfig.to_json_string)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/configuration_utils.py#L827)

( use\_diff: bool = True ) → `str`

Parameters

-   [](#transformers.PretrainedConfig.to_json_string.use_diff)**use\_diff** (`bool`, _optional_, defaults to `True`) — If set to `True`, only the difference between the config instance and the default `PretrainedConfig()` is serialized to JSON string.

String containing all the attributes that make up this configuration instance in JSON format.

Serializes this instance to a JSON string.

#### update

[](#transformers.PretrainedConfig.update)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/configuration_utils.py#L859)

( config\_dict: typing.Dict\[str, typing.Any\] )

Parameters

-   [](#transformers.PretrainedConfig.update.config_dict)**config\_dict** (`Dict[str, Any]`) — Dictionary of attributes that should be updated for this class.

Updates attributes of this class with attributes from `config_dict`.

#### update\_from\_string

[](#transformers.PretrainedConfig.update_from_string)[< source \>](https://github.com/huggingface/transformers/blob/v4.30.0/src/transformers/configuration_utils.py#L869)

( update\_str: str )

Parameters

-   [](#transformers.PretrainedConfig.update_from_string.update_str)**update\_str** (`str`) — String with attributes that should be updated for this class.

Updates attributes of this class with attributes from `update_str`.

The expected format is ints, floats and strings as is, and for booleans use `true` or `false`. For example: “n\_embd=10,resid\_pdrop=0.2,scale\_attn\_weights=false,summary\_type=cls\_index”

The keys to change have to already exist in the config object.