KeyError: "bloom" #4

by yongzx - opened

Currently, I think the transformers library hasn't support the bloom model. Our multilingual modeling group is starting to finetune it on new languages, so I wonder if there's any resource/pointer to adding model support into transformers.

I did a quick run through the HF's documentation but couldn't find any, and before I start looking at GitHub commits, I decide to post the question here.

>>> model = transformers.AutoModel.from_pretrained("bigscience/bloom-1b3", cache_dir="/users/zyong2/data/zyong2/huggingface/", use_auth_token=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/gpfs/data/sbach/zyong2/bigscience/env_lang_adapter/lib/python3.7/site-packages/transformers/models/auto/auto_factory.py", line 425, in from_pretrained
    pretrained_model_name_or_path, return_unused_kwargs=True, trust_remote_code=trust_remote_code, **kwargs
  File "/gpfs/data/sbach/zyong2/bigscience/env_lang_adapter/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 657, in from_pretrained
    config_class = CONFIG_MAPPING[config_dict["model_type"]]
  File "/gpfs/data/sbach/zyong2/bigscience/env_lang_adapter/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 372, in __getitem__
    raise KeyError(key)
KeyError: 'bloom'

The BLOOM class is not yet merged into main. You'll have to checkout this PR: https://github.com/huggingface/transformers/pull/17202

Sorry, this PR: https://github.com/huggingface/transformers/pull/17474 (The previous one was moved into this one)

Gotcha! I am reading through the PR right now.