Model resources contribution
Author: stevhliu
Pull request: no
Labels: name: Help wanted

name: Good First Issue

name: Good First Documentation Issue

Hi friends! 👋

There are a lot of cool existing resources for how to do *x* with *x* model, and we’d like to showcase and aggregate these resources on a model’s documentation. This’ll help users see how they can get started with a model for their own tasks since we know a lot of users check out the model documentation first. Take a look at a completed [resource section](https://huggingface.co/docs/transformers/main/en/model_doc/distilbert#resources) for DistilBERT as an example.

I’ve identified the top 20 models by pageviews, and now I’d like to open it up to the community if anyone is interested in helping!

Anyone can contribute; you just need to comment and claim one of the models on this [list](https://github.com/huggingface/transformers/issues/19848). Contributing is super easy:

1. Once you've claimed a model from the list, collect the existing resources from:
- the Hugging Face [blog](https://huggingface.co/blog)
- relevant materials from the 🤗 Hugging Face [Course](https://huggingface.co/course/chapter1/1)
- the Hugging Face [example scripts](https://github.com/huggingface/transformers/tree/main/examples) and [notebooks](https://github.com/huggingface/transformers/tree/main/notebooks)
- @NielsRogge's Transformers Tutorials [repository](https://github.com/NielsRogge/Transformers-Tutorials)
- @philschmid's [blog](https://www.philschmid.de/)
- [notebooks](https://huggingface.co/docs/transformers/community) from the community ❤️

2. Organize the resources by model tasks or applications (like inference or deployment):
- Use the corresponding icons for each task (you can find the names for each icon [here](https://github.com/huggingface/doc-builder/blob/19ba9da2556294f1777c865793d13e9ea47f8716/kit/src/lib/PipelineIcon.svelte#L42-L71)):
   ```
   <PipelineTag pipeline=”name-of-task”/>
   ```
- For certain categories, you can just do: 🚀 Deploy, ⚡️ Inference, or ⚗️ Optimization, etc.
- For community resources, add the 🌎 emoji at the end to indicate it’s not an official Hugging Face resource.
- Use this DistilBERT [file](https://github.com/huggingface/transformers/pull/19930/files) as a template. You can copy and paste the intro text and just replace DistilBERT with the name of the model you're working on.
    
3. Open a Pull Request with the new resources for your chosen model and ping me for a review (if you’re just getting started with contributing to an open-source project, check out @merveenoyan's awesome [GitHub Contribution Guide](https://www.notion.so/19411c29298644df8e9656af45a7686d)).

4. Congratulations, you just merged a PR into 🤗 Transformers, and your contribution will now help anyone who is looking at the model docs! 🎉

If you have any questions or need any help, don’t hesitate to ping me! 🤗❤️

Comments:
Hi @stevhliu, I want to work on OpenAI GPT!

Awesome! I'm looking forward to your contribution, and feel free to ping me if you have any questions! 🤗

@stevhliu 
I have a question. Is there a good way to search GitHub and blog posts? I tried to find related repos and blog posts with the word `OpenAI GPT` but I couldn't find them because search function doesn't seem to work well... Should I search one by one repo or post?

I made a draft pull request although it doesn't have links of GitHub and blog. You can check it to see if my research has been good or not
https://github.com/huggingface/transformers/pull/20084

Hey @shogohida, thanks for starting on this! 

The easiest way I've found for searching the blog posts is to go to the blog [repo](https://github.com/huggingface/blog) and search for mentions of `GPT` inside the repo. Then you can take a look at the results and see what's relevant!

For GitHub materials, you only have to look at the example scripts, and notebooks and *see what task* your model can be applied to. For example, `OpenAI GPT` is a casual language model, so you can link to example scripts for [causal language modeling](https://github.com/huggingface/transformers/tree/main/examples/pytorch/language-modeling#gpt-2gpt-and-causal-language-modeling) and also [text generation](https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-generation#language-generation). You can link the equivalent scripts in [TensorFlow](https://github.com/huggingface/transformers/tree/main/examples/tensorflow) and [Flax](https://github.com/huggingface/transformers/tree/main/examples/flax) if they're available.

After the scripts, you can hop over to the [notebooks](https://github.com/huggingface/transformers/tree/main/notebooks) and see what task your model can be applied to (language modeling, generate text) and do the same thing for the [community notebooks](https://huggingface.co/docs/transformers/community)!

@stevhliu 
Thanks for your comment! It will take a lot of time to collect resources from scripts and notebooks because I'm not very familiar with OpenAI GPT but I'll do my best. I'll let you know if I have another question

Hi, I would like to take CLIP from the list you have mentioned. :)

That's great @ambujpawar! I'm looking forward to your contribution, and feel free to ping me if you have any questions! 🤗



@stevhliu I would like to work on DeBERTa

Great, thanks for taking on DeBERTa @Saad135! 🤗

Hello, do you mind if  I can tackle on ALBERT model? @stevhliu 

For sure, looking forward to your contribution @JuheonChu! 🤗

Hi! Could I try ViT? It might take me some time though as have some work projects to complete too.

Hi, I would like to work on XLM-RoBERTa! @stevhliu

Hey @stanleycai95, that would be great! Feel free to work on it when you have the time :)

Awesome, XLM-RoBERTa is all yours @hazrulakmal!

Hi, I would like to work on GPT-J! @stevhliu 

Yay thanks for taking on GPTJ @adit299! Let me know if you have any questions or need any help 🤗 

Hi, could I work on OPT? :) @stevhliu

OPT is all yours @alissadb! 🤩 

Let me round out the list @stevhliu . TrOCR

Awesome, thanks for finishing this off @Laxmaan! 🎉 

Hello @stevhliu . I'd love to contribute in documentation. I see all models are assigned, is there any other I can help with?
Thank you 😊

Hi @elabongaatuo, sorry for the late reply and thanks for your enthusiasm! 

I think we are good with the model resource contributions for now. If you're looking for ways to contribute to the docs, feel free to open an issue for improving the docs (content that is unclear, missing, or inaccurate or fixing typos) and we can review it there. For more info about getting started with contributing, take a look at this [guide](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md)! 🤗

Hello @stevhliu . Thanks for getting back to me. I'll be on the lookout for docs that need improving. 

Hi @JuheonChu and @Laxmaan, I wanted to check and see if you're still interested in making a model contribution. Totally cool if you aren't available anymore, I'll unassign the models you claimed and let others take a shot at it. Thanks! 

Hi @stevhliu, I'd like to take a shot at one of the models if one of them becomes unassigned. Please let me know!

Thanks for the interest; TrOCR, LayoutLMV2, and ALBERT are now available!

Hello @stevhliu. I'd like to take up  ALBERT.

> Thanks for the interest; TrOCR, LayoutLMV2, and ALBERT are now available!

I’d like to take TrOCR!

All yours! Happy contributing and feel free to let me know if you have any questions! 🤗

> Thanks for the interest; TrOCR, LayoutLMV2, and ALBERT are now available!

Hello!! @stevhliu I don't have any option I guess 😅. LayoutLMV2 for me then 🌏.

Hi @subham73, LayoutLMv2 is actually [done](https://github.com/huggingface/transformers/issues/19848) haha! 

Hi @stevhliu are there any open issues to work on :)

Hi, thanks for your interest @Girish16! 

Feel free to browse [Good First Issues](https://github.com/huggingface/transformers/issues?q=is%3Aopen+is%3Aissue+label%3A%22Good+First+Issue%22) for open issues to work on, and you can also check out the [Contribution](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#contribute-to--transformers) guide for more ways to contribute! 🤗 

Hi. Is ALBERT still available? 

Hi @ENate, ALBERT is currently being worked on in #23685. If the original contributor is no longer interested in working on it, I'll let you know! 😄  

No worries thanks :) . 

@stevhliu hello, @ENate can take it up. 😊

Okay then. Will proceed using the guidelines provided by @stevhliu and the example for DIstilBERT.

@stevhliu - I saw that there is a resource for ALBERT at:
```
https://huggingface.co/docs/transformers/main/en/model_doc/albert
```

which is similar to the resources for DistilBERT you mentioned in the guidelines above at:

```
https://huggingface.co/docs/transformers/main/en/model_doc/distilbert#resources 
``` 


Yeah ALBERT only has the task guides, and it doesn't go quite as in-depth as DistilBERT. For example, DistilBERT includes links to the course, notebooks, and scripts. You can probably just copy over most of the content from DistilBERT that is relevant to ALBERT (in other words, replace `DistilBERTForX` with `ALBERTForX`)!

Thanks :) @stevhliu 

Hello @stevhliu is Jukebox still available? 

Feel free to open a PR for Jukebox @daniela-basurto! 🤗 

Hello @stevhliu may I please take up **whisper** with a few of the OSSCA mentees?

Cc: tysm @ArthurZucker for the pointer! We'll start compiling models with incomplete resource tabs so our mentees can work on them.

Yes absolutely, thanks for your interest @wonhyeongseo!

I ran a simple `grep -wL * -e "## Resources"` command, and a total of **150 out of 222 documents** would benefit from this issue. I'm not sure if all of these are open for contributions though.
Below is the todo list with contributors I saw recently.

<summary>

Model Docs in need of resources [Updated 23-07-22]:

</summary>

<details>

- [ ] albert
- [ ] altclip
- [ ] bark
- [ ] barthez
- [ ] bartpho
- [ ] bert-generation
- [ ] bert-japanese
- [ ] bertweet
- [ ] big_bird
- [ ] bigbird_pegasus
- [ ] biogpt
- [ ] blenderbot-small
- [ ] blenderbot
- [ ] bridgetower
- [ ] byt5
- [ ] camembert
- [ ] canine
- [ ] chinese_clip
- [ ] clap
- [ ] codegen
- [ ] conditional_detr
- [ ] convbert
- [ ] cpm
- [ ] cpmant
- [ ] ctrl
- [ ] deberta-v2
- [ ] decision_transformer
- [ ] deplot
- [ ] dialogpt
- [ ] dinov2
- [ ] donut
- [ ] dpr
- [ ] efficientformer
- [ ] efficientnet
- [ ] electra
- [ ] encodec
- [ ] encoder-decoder
- [ ] ernie
- [ ] ernie_m
- [ ] esm
- [ ] flan-t5
- [ ] flan-ul2
- [ ] flaubert
- [ ] flava
- [ ] fnet
- [ ] focalnet
- [ ] fsmt
- [ ] funnel
- [ ] gpt-sw3
- [ ] gpt_bigcode
- [ ] gpt_neo
- [ ] gpt_neox
- [ ] gpt_neox_japanese
- [ ] gptsan-japanese
- [ ] graphormer
- [ ] herbert
- [ ] hubert
- [ ] ibert
- [ ] instructblip
- [ ] jukebox @daniela-basurto
- [ ] layoutxlm
- [ ] led
- [ ] llama @wonhyeongseo and OSSCA
- [ ] llama2
- [ ] longformer
- [ ] longt5
- [ ] luke
- [ ] lxmert
- [ ] m2m_100
- [ ] marian
- [ ] markuplm
- [ ] matcha
- [ ] mbart
- [ ] mega
- [ ] megatron-bert
- [ ] megatron_gpt2
- [ ] mgp-str
- [ ] mluke
- [ ] mms
- [ ] mobilebert
- [ ] mobilevitv2
- [ ] mpnet
- [ ] mra
- [ ] mt5
- [ ] musicgen
- [ ] mvp
- [ ] nezha
- [ ] nllb-moe
- [ ] nllb
- [ ] nystromformer
- [ ] owlvit
- [ ] pegasus
- [ ] pegasus_x
- [ ] perceiver
- [ ] phobert
- [ ] plbart
- [ ] prophetnet
- [ ] qdqbert
- [ ] rag
- [ ] realm
- [ ] reformer
- [ ] rembert
- [ ] roberta-prelayernorm
- [ ] roc_bert
- [ ] roformer
- [ ] rwkv
- [ ] sam
- [ ] sew-d
- [ ] sew
- [ ] speech-encoder-decoder
- [ ] speech_to_text
- [ ] speech_to_text_2
- [ ] speecht5
- [ ] splinter
- [ ] squeezebert
- [ ] swiftformer
- [ ] t5v1.1
- [ ] tapas
- [ ] timesformer
- [ ] todo
- [ ] tvlt
- [ ] ul2
- [ ] umt5
- [ ] unispeech-sat
- [ ] unispeech
- [ ] vilt
- [ ] vision-encoder-decoder
- [ ] vision-text-dual-encoder
- [ ] visual_bert
- [ ] vivit
- [ ] wav2vec2-conformer
- [ ] wav2vec2_phoneme
- [ ] wavlm
- [ ] whisper @wonhyeongseo and OSSCA
- [ ] xglm
- [ ] xlm-prophetnet
- [ ] xlm-roberta-xl
- [ ] xlm-v
- [ ] xlm
- [ ] xlnet
- [ ] xls_r
- [ ] xlsr_wav2vec2
- [ ] yoso

</details>

Ok.

> I'm not sure if all of these are open for contributions though.

Thanks for checking @wonhyeongseo! I think it would be nice to eventually have Resources for all the models, so if you see other ones you're interested in contributing to, feel free to open a PR! I would focus on the more high-impact models first (like LLaMA) that get more pageviews/usage. For certain models (like [BORT](https://huggingface.co/docs/transformers/model_doc/bort)) that are in maintenance mode, we can skip those entirely.

Awesome @stevhliu , thank you so much for your warm reception. 

- **May we please reserve LLaMA as well for the OSSCA team?**
- **In your opinion, when is the ideal time to start gathering resources after a model's release?**
  For LLaMA2, since it's relatively new, there might not be many official resources yet. It will depend on a models impact as you described, but a rule of thumb would be useful.
- **Although I think this is already the case, would it be possible for you to sort these incomplete models and provide the top 20 sorted by impact or page views as of recent advances?**
  I'm sure some will pique the interest of my team and our mentees.

<summary>

Thank you for the heads up for files under maintenance! I've deleted 8 of those from the above https://github.com/huggingface/transformers/issues/20055#issuecomment-1645188309 list by `grep -LZ "## Resources" * | xargs -0 grep -l "<Tip warning={true}>"` :

</summary>
<details>

```js
auto.md
bort.md
mctct.md
open-llama.md
retribert.md
tapex.md
trajectory_transformer.md
transfo-xl.md
```

</details>

Thank you so much for your support @stevhliu .
Hope you have a wonderful weekend!

Best regards,
Won Seo

> May we please reserve LLaMA as well for the OSSCA team?

For sure! 👍

> In your opinion, when is the ideal time to start gathering resources after a model's release?

I think maybe whenever you see some content, you can open a PR to add it to the model page. It's ok if it's just one guide/tutorial/blog post; we can gradually add to it as more content and resources get created. For example, Philipp has a [blog post](https://www.philschmid.de/sagemaker-llama2-qlora) about fine-tuning LLaMA 2 on SageMaker here that can be added :)

> Although I think this is already the case, would it be possible for you to sort these incomplete models and provide the top 20 sorted by impact or page views as of recent advances?

By downloads, here are the next top 20 models (its okay to skip some of the models if there aren't any available resources for them):

<details>

BART
CLIPSeg
Marian
MPNet
ELECTRA
ResNet
CamemBERT
HuBERT
LLaMA
Longformer
VisionEncoderDecoder
GPT NeoX
EnCodec
ConvBERT
mBART
GPT Neo
FNet
YOLOS
BLIP
BEiT

</details>

ok

@stevhliu 
Hello, I would like to put together some resources for Longformer, willing to look into CamemBERT as well if Longformer has already been taken. 

Hi @stevhliu  can I work on LLaMA, can you please assign me?

Hi @debrupf2946, we already have a [resource section](https://huggingface.co/docs/transformers/main/en/model_doc/llama#resources) for Llama. Feel free to work on another model if you're interested!

Hi @stevhliu Can I work on ALBERT, which is the only left model in the list? 

Hi @siddharth1012 I am almost done with ALBERT and about to open a PR. Issue is due to a doc-builder issue I encountered. Also the problem on Cython version.

So I am working on ALBERT thanks

Hi, is there anything that i can help with? i can see lot of diff models are being talked above, where can find open list ?
thanks,

Hi, thanks for your interest @jaykhatri0875! The open model list is [here](https://github.com/huggingface/transformers/issues/19848) but I think we're close to completing it. If you're interested in making other contributions, feel free to check out the [Good First Issues](https://github.com/huggingface/transformers/issues?q=is:open+is:issue+label:%22Good+First+Issue%22) 🤗 

All finished now! 🥳