bigscience

/

bloom-1b7

like 107

Currently, I think the transformers library hasn't support the bloom model. Our multilingual modeling group is starting to finetune it on new languages, so I wonder if there's any resource/pointer to adding model support into transformers.

I did a quick run through the HF's documentation but couldn't find any, and before I start looking at GitHub commits, I decide to post the question here.

>>> model = transformers.AutoModel.from_pretrained("bigscience/bloom-1b3", cache_dir="/users/zyong2/data/zyong2/huggingface/", use_auth_token=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/gpfs/data/sbach/zyong2/bigscience/env_lang_adapter/lib/python3.7/site-packages/transformers/models/auto/auto_factory.py", line 425, in from_pretrained
    pretrained_model_name_or_path, return_unused_kwargs=True, trust_remote_code=trust_remote_code, **kwargs
  File "/gpfs/data/sbach/zyong2/bigscience/env_lang_adapter/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 657, in from_pretrained
    config_class = CONFIG_MAPPING[config_dict["model_type"]]
  File "/gpfs/data/sbach/zyong2/bigscience/env_lang_adapter/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 372, in __getitem__
    raise KeyError(key)
KeyError: 'bloom'