Remove static pretrained maps from the library's internals
Author: LysandreJik
Pull request: yes


Comments:
Before this ungodly PR gets merged, I need to check that every checkpoint referenced here behaves the same once its pretrained map has been removed. 

I'll link the PRs open as a result in this comment.

🟣: merged
🟢: open
🔴: closed
🟡: not open yet

## Repos with PRs opened

### ALBERT
🟣 [albert/albert-base-v1#2](https://huggingface.co/albert/albert-base-v1/discussions/2)
🟣 [albert/albert-base-v2#6](https://huggingface.co/albert/albert-base-v2/discussions/6)
🟣 [albert/albert-large-v1#2](https://huggingface.co/albert/albert-large-v1/discussions/2)
🟣 [albert/albert-large-v2#3](https://huggingface.co/albert/albert-large-v2/discussions/3)
🟣 [albert/albert-xlarge-v1#2](https://huggingface.co/albert/albert-xlarge-v1/discussions/2)
🟣 [albert/albert-xlarge-v2#2](https://huggingface.co/albert/albert-xlarge-v2/discussions/2)
🟣 [albert/albert-xxlarge-v1#3](https://huggingface.co/albert/albert-xxlarge-v1/discussions/3)
🟣 [albert/albert-xxlarge-v2#3](https://huggingface.co/albert/albert-xxlarge-v2/discussions/3)

### BERT

🟣 [google-bert/bert-base-cased#10](https://huggingface.co/google-bert/bert-base-cased/discussions/10)
🟣 [google-bert/bert-base-cased-finetuned-mrpc#2](https://huggingface.co/google-bert/bert-base-cased-finetuned-mrpc/discussions/2)
🟣 [google-bert/bert-base-chinese#16](https://huggingface.co/google-bert/bert-base-chinese/discussions/16)
🟣 [google-bert/bert-base-german-cased#5](https://huggingface.co/google-bert/bert-base-german-cased/discussions/5)
🟣 [google-bert/bert-base-german-dbmdz-cased#4](https://huggingface.co/google-bert/bert-base-german-dbmdz-cased/discussions/4)
🟣 [google-bert/bert-base-german-dbmdz-uncased#4](https://huggingface.co/google-bert/bert-base-german-dbmdz-uncased/discussions/4)
🟣 [google-bert/bert-base-multilingual-cased#5](https://huggingface.co/google-bert/bert-base-multilingual-cased/discussions/5)
🟣 [google-bert/bert-base-multilingual-uncased#5](https://huggingface.co/google-bert/bert-base-multilingual-uncased/discussions/5)
🟣 [google-bert/bert-base-uncased#62](https://huggingface.co/google-bert/bert-base-uncased/discussions/62)
🟣 [google-bert/bert-large-cased#3](https://huggingface.co/google-bert/bert-large-cased/discussions/3)
🟣 [google-bert/bert-large-cased-whole-word-masking#2](https://huggingface.co/google-bert/bert-large-cased-whole-word-masking/discussions/2)
🟣 [google-bert/bert-large-cased-whole-word-masking-finetuned-squad#2](https://huggingface.co/google-bert/bert-large-cased-whole-word-masking-finetuned-squad/discussions/2)
🟣 [google-bert/bert-large-uncased#3](https://huggingface.co/google-bert/bert-large-uncased/discussions/3)
🟣 [google-bert/bert-large-uncased-whole-word-masking#3](https://huggingface.co/google-bert/bert-large-uncased-whole-word-masking/discussions/3)
🟣 [google-bert/bert-large-uncased-whole-word-masking-finetuned-squad#4](https://huggingface.co/google-bert/bert-large-uncased-whole-word-masking-finetuned-squad/discussions/4)

### CamemBERT

🟢 [almanach/camembert-base#7](https://huggingface.co/almanach/camembert-base/discussions/7)

### CTRL

🟣 [Salesforce/ctrl#4](https://huggingface.co/Salesforce/ctrl/discussions/4)

### DistilBERT

🟣 [distilbert/distilgpt2#11](https://huggingface.co/distilbert/distilgpt2/discussions/11)
🟣 [distilbert/distilroberta-base#4](https://huggingface.co/distilbert/distilroberta-base/discussions/4)

### GPT-2

🟣 [openai-community/gpt2#80](https://huggingface.co/openai-community/gpt2/discussions/80)
🟣 [openai-community/gpt2-large#7](https://huggingface.co/openai-community/gpt2-large/discussions/7)
🟣 [openai-community/gpt2-medium#13](https://huggingface.co/openai-community/gpt2-medium/discussions/13)
🟣 [openai-community/gpt2-xl#9](https://huggingface.co/openai-community/gpt2-xl/discussions/9)

### OpenAI GPT

🟣 [openai-community/openai-gpt#6](https://huggingface.co/openai-community/openai-gpt/discussions/6)

### RoBERTa

🟣 [FacebookAI/roberta-base#12](https://huggingface.co/FacebookAI/roberta-base/discussions/12)
🟣 [openai-community/roberta-base-openai-detector#17](https://huggingface.co/openai-community/roberta-base-openai-detector/discussions/17)
🟣 [FacebookAI/roberta-large#6](https://huggingface.co/FacebookAI/roberta-large/discussions/6)
🟣 [FacebookAI/roberta-large-mnli#8](https://huggingface.co/FacebookAI/roberta-large-mnli/discussions/8)
🟣 [openai-community/roberta-large-openai-detector#5](https://huggingface.co/openai-community/roberta-large-openai-detector/discussions/5)
🟣 [FacebookAI/xlm-roberta-base#27](https://huggingface.co/FacebookAI/xlm-roberta-base/discussions/27)
🟣 [FacebookAI/xlm-roberta-large#17](https://huggingface.co/FacebookAI/xlm-roberta-large/discussions/17)
🟣 [FacebookAI/xlm-roberta-large-finetuned-conll02-dutch#3](https://huggingface.co/FacebookAI/xlm-roberta-large-finetuned-conll02-dutch/discussions/3)
🟣 [FacebookAI/xlm-roberta-large-finetuned-conll02-spanish#3](https://huggingface.co/FacebookAI/xlm-roberta-large-finetuned-conll02-spanish/discussions/3)
🟣 [FacebookAI/xlm-roberta-large-finetuned-conll03-english#11](https://huggingface.co/FacebookAI/xlm-roberta-large-finetuned-conll03-english/discussions/11)
🟣 [FacebookAI/xlm-roberta-large-finetuned-conll03-german#4](https://huggingface.co/FacebookAI/xlm-roberta-large-finetuned-conll03-german/discussions/4)

## Non-canonical models

🟣 [facebook/m2m100_418M#16](https://huggingface.co/facebook/m2m100_418M/discussions/16)
🟣 [openai/clip-vit-base-patch32#13](https://huggingface.co/openai/clip-vit-base-patch32/discussions/13)
🟣 [google/bigbird-roberta-large#2](https://huggingface.co/google/bigbird-roberta-large/discussions/2)
🟣 [google/bigbird-base-trivia-itc#2](https://huggingface.co/google/bigbird-base-trivia-itc/discussions/2)
🟢 [google/rembert#2](https://huggingface.co/google/rembert/discussions/2)
🟢 [YituTech/conv-bert-base#4](https://huggingface.co/YituTech/conv-bert-base/discussions/4)
🟢 [YituTech/conv-bert-medium-small#2](https://huggingface.co/YituTech/conv-bert-medium-small/discussions/2)
🟢 [YituTech/conv-bert-small#2](https://huggingface.co/YituTech/conv-bert-small/discussions/2)
🔴 [facebook/wav2vec2-lv-60-espeak-cv-ft#5](https://huggingface.co/facebook/wav2vec2-lv-60-espeak-cv-ft/discussions/5)
🟣 [facebook/blenderbot_small-90M#5](https://huggingface.co/facebook/blenderbot_small-90M/discussions/5)
🟣 [funnel-transformer/small#2](https://huggingface.co/funnel-transformer/small/discussions/2)
🟣 [funnel-transformer/small-base#2](https://huggingface.co/funnel-transformer/small-base/discussions/2)
🟣 [funnel-transformer/medium#2](https://huggingface.co/funnel-transformer/medium/discussions/2)
🟣 [funnel-transformer/medium-base#1](https://huggingface.co/funnel-transformer/medium-base/discussions/1)
🟣 [funnel-transformer/intermediate#2](https://huggingface.co/funnel-transformer/intermediate/discussions/2)
🟣 [funnel-transformer/intermediate-base#2](https://huggingface.co/funnel-transformer/intermediate-base/discussions/2)
🟣 [funnel-transformer/large#3](https://huggingface.co/funnel-transformer/large/discussions/3)
🟣 [funnel-transformer/large-base#1](https://huggingface.co/funnel-transformer/large-base/discussions/1)
🟣 [funnel-transformer/xlarge#2](https://huggingface.co/funnel-transformer/xlarge/discussions/2)
🟣 [funnel-transformer/xlarge-base#2](https://huggingface.co/funnel-transformer/xlarge-base/discussions/2)
🟣 [flaubert/flaubert_small_cased#2](https://huggingface.co/flaubert/flaubert_small_cased/discussions/2)
🟣 [flaubert/flaubert_base_uncased#2](https://huggingface.co/flaubert/flaubert_base_uncased/discussions/2)
🟣 [flaubert/flaubert_base_cased#2](https://huggingface.co/flaubert/flaubert_base_cased/discussions/2)
🟣 [flaubert/flaubert_large_cased#2](https://huggingface.co/flaubert/flaubert_large_cased/discussions/2)
🟣 [google/realm-cc-news-pretrained-embedder#1](https://huggingface.co/google/realm-cc-news-pretrained-embedder/discussions/1)
🟣 [google/realm-cc-news-pretrained-encoder#1](https://huggingface.co/google/realm-cc-news-pretrained-encoder/discussions/1)
🟣 [google/realm-cc-news-pretrained-scorer#1](https://huggingface.co/google/realm-cc-news-pretrained-scorer/discussions/1)
🟣 [google/realm-cc-news-pretrained-openqa#1](https://huggingface.co/google/realm-cc-news-pretrained-openqa/discussions/1)
🟣 [google/realm-orqa-nq-openqa#1](https://huggingface.co/google/realm-orqa-nq-openqa/discussions/1)
🟣 [google/realm-orqa-nq-reader#1](https://huggingface.co/google/realm-orqa-nq-reader/discussions/1)
🟣 [google/realm-orqa-wq-openqa#1](https://huggingface.co/google/realm-orqa-wq-openqa/discussions/1)
🟣 [google/realm-orqa-wq-reader#1](https://huggingface.co/google/realm-orqa-wq-reader/discussions/1)
🟣 [google/fnet-base#1](https://huggingface.co/google/fnet-base/discussions/1)
🟣 [google/fnet-large#1](https://huggingface.co/google/fnet-large/discussions/1)
🟣 [microsoft/mpnet-base#4](https://huggingface.co/microsoft/mpnet-base/discussions/4)
🟣 [google/reformer-crime-and-punishment#2](https://huggingface.co/google/reformer-crime-and-punishment/discussions/2)
🟢 [facebook/s2t-wav2vec2-large-en-de#3](https://huggingface.co/facebook/s2t-wav2vec2-large-en-de/discussions/3)
🟢 [allenai/longformer-base-4096#6](https://huggingface.co/allenai/longformer-base-4096/discussions/6)
🟢 [allenai/longformer-large-4096#3](https://huggingface.co/allenai/longformer-large-4096/discussions/3)
🟢 [allenai/longformer-large-4096-finetuned-triviaqa#4](https://huggingface.co/allenai/longformer-large-4096-finetuned-triviaqa/discussions/4)
🟢 [allenai/longformer-base-4096-extra.pos.embd.only#2](https://huggingface.co/allenai/longformer-base-4096-extra.pos.embd.only/discussions/2)
🟢 [allenai/longformer-large-4096-extra.pos.embd.only#2](https://huggingface.co/allenai/longformer-large-4096-extra.pos.embd.only/discussions/2)
🟣 [cl-tohoku/bert-base-japanese#2](https://huggingface.co/tohoku-nlp/bert-base-japanese/discussions/2)
🟣 [cl-tohoku/bert-base-japanese-whole-word-masking#3](https://huggingface.co/tohoku-nlp/bert-base-japanese-whole-word-masking/discussions/3)
🟣 [cl-tohoku/bert-base-japanese-char#2](https://huggingface.co/tohoku-nlp/bert-base-japanese-char/discussions/2)
🟣 [cl-tohoku/bert-base-japanese-char-whole-word-masking#2](https://huggingface.co/tohoku-nlp/bert-base-japanese-char-whole-word-masking/discussions/2)
🟣 [google/electra-small-generator#4](https://huggingface.co/google/electra-small-generator/discussions/4)
🟣 [google/electra-base-generator#2](https://huggingface.co/google/electra-base-generator/discussions/2)
🟣 [google/electra-large-generator#2](https://huggingface.co/google/electra-large-generator/discussions/2)
🟣 [google/electra-small-discriminator#1](https://huggingface.co/google/electra-small-discriminator/discussions/1)
🟣 [google/electra-base-discriminator#3](https://huggingface.co/google/electra-base-discriminator/discussions/3)
🟣 [google/electra-large-discriminator#1](https://huggingface.co/google/electra-large-discriminator/discussions/1)
🟢 [microsoft/layoutlmv2-base-uncased#5](https://huggingface.co/microsoft/layoutlmv2-base-uncased/discussions/5)
🟢 [microsoft/layoutlmv2-large-uncased#3](https://huggingface.co/microsoft/layoutlmv2-large-uncased/discussions/3)
🟢 [microsoft/deberta-v2-xlarge#3](https://huggingface.co/microsoft/deberta-v2-xlarge/discussions/3)
🟢 [microsoft/deberta-v2-xxlarge#4](https://huggingface.co/microsoft/deberta-v2-xxlarge/discussions/4)
🟣 [microsoft/deberta-v2-xlarge-mnli#2](https://huggingface.co/microsoft/deberta-v2-xlarge-mnli/discussions/2)
🟢 [microsoft/deberta-v2-xxlarge-mnli#2](https://huggingface.co/microsoft/deberta-v2-xxlarge-mnli/discussions/2)
🟢 [vinai/bartpho-syllable#3](https://huggingface.co/vinai/bartpho-syllable/discussions/3)
🟢 [Helsinki-NLP/opus-mt-en-de#6](https://huggingface.co/Helsinki-NLP/opus-mt-en-de/discussions/6)
🟢 [facebook/bart-base#4](https://huggingface.co/facebook/bart-base/discussions/4)
🟢 [facebook/bart-large-cnn#71](https://huggingface.co/facebook/bart-large-cnn/discussions/71)
🟢 [yjernite/bart_eli5#2](https://huggingface.co/yjernite/bart_eli5/discussions/2)
🟢 [microsoft/layoutlm-base-uncased#3](https://huggingface.co/microsoft/layoutlm-base-uncased/discussions/3)
🟢 [microsoft/layoutlm-large-uncased#1](https://huggingface.co/microsoft/layoutlm-large-uncased/discussions/1)
🟢 [junnyu/roformer_chinese_char_small#1](https://huggingface.co/junnyu/roformer_chinese_char_small/discussions/1)
🟢 [junnyu/roformer_chinese_char_base#1](https://huggingface.co/junnyu/roformer_chinese_char_base/discussions/1)
🟢 [junnyu/roformer_small_discriminator#2](https://huggingface.co/junnyu/roformer_small_discriminator/discussions/2)
🟢 [junnyu/roformer_small_generator#2](https://huggingface.co/junnyu/roformer_small_generator/discussions/2)
🟢 [uclanlp/plbart-base#6](https://huggingface.co/uclanlp/plbart-base/discussions/6)
🟢 [uclanlp/plbart-c-cpp-defect-detection#4](https://huggingface.co/uclanlp/plbart-c-cpp-defect-detection/discussions/4)
🟢 [uclanlp/plbart-cs-java#1](https://huggingface.co/uclanlp/plbart-cs-java/discussions/1)
🟢 [uclanlp/plbart-en_XX-java#2](https://huggingface.co/uclanlp/plbart-en_XX-java/discussions/2)
🟢 [uclanlp/plbart-go-en_XX#1](https://huggingface.co/uclanlp/plbart-go-en_XX/discussions/1)
🟢 [uclanlp/plbart-java-clone-detection#2](https://huggingface.co/uclanlp/plbart-java-clone-detection/discussions/2)
🟢 [uclanlp/plbart-java-cs#3](https://huggingface.co/uclanlp/plbart-java-cs/discussions/3)
🟢 [uclanlp/plbart-java-en_XX#2](https://huggingface.co/uclanlp/plbart-java-en_XX/discussions/2)
🟢 [uclanlp/plbart-javascript-en_XX#2](https://huggingface.co/uclanlp/plbart-javascript-en_XX/discussions/2)
🟢 [uclanlp/plbart-php-en_XX#2](https://huggingface.co/uclanlp/plbart-php-en_XX/discussions/2)
🟢 [uclanlp/plbart-python-en_XX#2](https://huggingface.co/uclanlp/plbart-python-en_XX/discussions/2)
🟢 [uclanlp/plbart-refine-java-medium#1](https://huggingface.co/uclanlp/plbart-refine-java-medium/discussions/1)
🟢 [uclanlp/plbart-refine-java-small#2](https://huggingface.co/uclanlp/plbart-refine-java-small/discussions/2)
🟢 [uclanlp/plbart-ruby-en_XX#1](https://huggingface.co/uclanlp/plbart-ruby-en_XX/discussions/1)
🟢 [openbmb/cpm-ant-10b#2](https://huggingface.co/openbmb/cpm-ant-10b/discussions/2)
🟢 [microsoft/deberta-base#5](https://huggingface.co/microsoft/deberta-base/discussions/5)
🟢 [microsoft/deberta-large#4](https://huggingface.co/microsoft/deberta-large/discussions/4)
🟢 [microsoft/deberta-xlarge#2](https://huggingface.co/microsoft/deberta-xlarge/discussions/2)
🟢 [microsoft/deberta-base-mnli#1](https://huggingface.co/microsoft/deberta-base-mnli/discussions/1)
🟢 [microsoft/deberta-large-mnli#1](https://huggingface.co/microsoft/deberta-large-mnli/discussions/1)
🟢 [microsoft/deberta-xlarge-mnli#5](https://huggingface.co/microsoft/deberta-xlarge-mnli/discussions/5)
🟢 [vinai/bertweet-base#5](https://huggingface.co/vinai/bertweet-base/discussions/5)
🟢 [AI-Sweden-Models/gpt-sw3-126m#5](https://huggingface.co/AI-Sweden-Models/gpt-sw3-126m/discussions/5)
🟢 [AI-Sweden-Models/gpt-sw3-356m#4](https://huggingface.co/AI-Sweden-Models/gpt-sw3-356m/discussions/4)
🟢 [AI-Sweden-Models/gpt-sw3-1.3b#4](https://huggingface.co/AI-Sweden-Models/gpt-sw3-1.3b/discussions/4)
🟢 [AI-Sweden-Models/gpt-sw3-6.7b#5](https://huggingface.co/AI-Sweden-Models/gpt-sw3-6.7b/discussions/5)
🟢 [AI-Sweden-Models/gpt-sw3-6.7b-v2#4](https://huggingface.co/AI-Sweden-Models/gpt-sw3-6.7b-v2/discussions/4)
🟢 [AI-Sweden-Models/gpt-sw3-20b#6](https://huggingface.co/AI-Sweden-Models/gpt-sw3-20b/discussions/6)
🟣 [AI-Sweden-Models/gpt-sw3-40b#4](https://huggingface.co/AI-Sweden-Models/gpt-sw3-40b/discussions/4)
🟢 [facebook/mbart-large-50-one-to-many-mmt#7](https://huggingface.co/facebook/mbart-large-50-one-to-many-mmt/discussions/7)
🟢 [facebook/xglm-564M#7](https://huggingface.co/facebook/xglm-564M/discussions/7)
🟢 [facebook/blenderbot-3B#7](https://huggingface.co/facebook/blenderbot-3B/discussions/7)
🟢 [microsoft/prophetnet-large-uncased#2](https://huggingface.co/microsoft/prophetnet-large-uncased/discussions/2)
🟢 [microsoft/xprophetnet-large-wiki100-cased#1](https://huggingface.co/microsoft/xprophetnet-large-wiki100-cased/discussions/1)
🟢 [squeezebert/squeezebert-uncased#3](https://huggingface.co/squeezebert/squeezebert-uncased/discussions/3)
🟢 [squeezebert/squeezebert-mnli#2](https://huggingface.co/squeezebert/squeezebert-mnli/discussions/2)
🟢 [squeezebert/squeezebert-mnli-headless#2](https://huggingface.co/squeezebert/squeezebert-mnli-headless/discussions/2)
🟢 [abeja/gpt-neox-japanese-2.7b#5](https://huggingface.co/abeja/gpt-neox-japanese-2.7b/discussions/5)
🟢 [microsoft/biogpt#26](https://huggingface.co/microsoft/biogpt/discussions/26)
🔴 [facebook/wav2vec2-base-960h#11](https://huggingface.co/facebook/wav2vec2-base-960h/discussions/11)
🟣 [moussaKam/mbarthez#2](https://huggingface.co/moussaKam/mbarthez/discussions/2)
🟣 [moussaKam/barthez#3](https://huggingface.co/moussaKam/barthez/discussions/3)
🟣 [moussaKam/barthez-orangesum-title#2](https://huggingface.co/moussaKam/barthez-orangesum-title/discussions/2)
🟢 [vinai/phobert-base#5](https://huggingface.co/vinai/phobert-base/discussions/5)
🟢 [vinai/phobert-large#2](https://huggingface.co/vinai/phobert-large/discussions/2)
🟢 [facebook/mbart-large-en-ro#3](https://huggingface.co/facebook/mbart-large-en-ro/discussions/3)
🟢 [facebook/mbart-large-cc25#5](https://huggingface.co/facebook/mbart-large-cc25/discussions/5)



The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29112). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Updated the base to move some of the refactor to that PR: https://github.com/huggingface/transformers/pull/29534

why deprecate vs remove?

Deprecate for 2 months and then simply `DEL` a file and it's removed

It should be good for review for the brave one @amyeroberts @ArthurZucker.

Failing tests are unrelated and also failing on `main`.