OATML-Markslab/Tranception_Large · Key error when loading the model using AutoModelForCausalLM or AutoModelWithLMHead

I'm running into an error when I try to load this model. When I run:

model = AutoModelForCausalLM.from_pretrained("OATML-Markslab/Tranception_Large")

I get:

KeyError                                  Traceback (most recent call last)
Input In [15], in <cell line: 2>()
      1 #tokenizer = AutoTokenizer.from_pretrained("OATML-Markslab/Tranception_Large")
----> 2 model = AutoModelForCausalLM.from_pretrained("OATML-Markslab/Tranception_Large")

File ~/work/phd/projects/protein-engineering-benchmark/conda_env/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py:434, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    432 hub_kwargs = {name: kwargs.pop(name) for name in hub_kwargs_names if name in kwargs}
    433 if not isinstance(config, PretrainedConfig):
--> 434     config, kwargs = AutoConfig.from_pretrained(
    435         pretrained_model_name_or_path,
    436         return_unused_kwargs=True,
    437         trust_remote_code=trust_remote_code,
    438         **hub_kwargs,
    439         **kwargs,
    440     )
    441 if hasattr(config, "auto_map") and cls.__name__ in config.auto_map:
    442     if not trust_remote_code:

File ~/work/phd/projects/protein-engineering-benchmark/conda_env/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py:873, in AutoConfig.from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
    871     return config_class.from_pretrained(pretrained_model_name_or_path, **kwargs)
    872 elif "model_type" in config_dict:
--> 873     config_class = CONFIG_MAPPING[config_dict["model_type"]]
    874     return config_class.from_dict(config_dict, **unused_kwargs)
    875 else:
    876     # Fallback: use pattern matching on the string.
    877     # We go from longer names to shorter names to catch roberta before bert (for instance)

File ~/work/phd/projects/protein-engineering-benchmark/conda_env/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py:579, in _LazyConfigMapping.__getitem__(self, key)
    577     return self._extra_content[key]
    578 if key not in self._mapping:
--> 579     raise KeyError(key)
    580 value = self._mapping[key]
    581 module_name = model_type_to_module_name(key)

KeyError: 'tranception'

Is this a bug or is there something I can do to fix this?

OS: macOS 12.6 21G115 arm64
Transformers version 4.26.1