Transformers documentation

๐Ÿค— Transformers

You are viewing v4.44.2 version. A newer version v4.46.3 is available.
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

๐Ÿค— Transformers

PyTorch, TensorFlow, JAXใฎใŸใ‚ใฎๆœ€ๅ…ˆ็ซฏๆฉŸๆขฐๅญฆ็ฟ’ใ€‚

๐Ÿค— Transformers ใฏๆœ€ๅ…ˆ็ซฏใฎๅญฆ็ฟ’ๆธˆใฟใƒขใƒ‡ใƒซใ‚’็ฐกๅ˜ใซใƒ€ใ‚ฆใƒณใƒญใƒผใƒ‰ใ—ใฆๅญฆ็ฟ’ใ™ใ‚‹APIใจใƒ„ใƒผใƒซใ‚’ๆไพ›ใ—ใพใ™ใ€‚ๅญฆ็ฟ’ๆธˆใฟใƒขใƒ‡ใƒซใ‚’ไฝฟ็”จใ™ใ‚‹ใ“ใจใง่จˆ็ฎ—ใ‚ณใ‚นใƒˆใจไบŒ้…ธๅŒ–็‚ญ็ด ใฎๆŽ’ๅ‡บ้‡ใ‚’ๅ‰Šๆธ›ใงใใ€ใพใŸใ‚ผใƒญใ‹ใ‚‰ใƒขใƒ‡ใƒซใ‚’ๅญฆ็ฟ’ใ™ใ‚‹ใŸใ‚ใซ่ฆๆฑ‚ใ•ใ‚Œใ‚‹ๆ™‚้–“ใจใƒชใ‚ฝใƒผใ‚นใ‚’็ฏ€็ด„ใ™ใ‚‹ใ“ใจใŒใงใใพใ™ใ€‚ ใ“ใ‚Œใ‚‰ใฎใƒขใƒ‡ใƒซใฏไปฅไธ‹ใฎใ‚ˆใ†ใช็•ฐใชใ‚‹ใƒขใƒ€ใƒชใƒ†ใ‚ฃใซใŠใ‘ใ‚‹ไธ€่ˆฌ็š„ใชใ‚ฟใ‚นใ‚ฏใ‚’ใ‚ตใƒใƒผใƒˆใ—ใพใ™:

๐Ÿ“ ่‡ช็„ถ่จ€่ชžๅ‡ฆ็†: ใƒ†ใ‚ญใ‚นใƒˆๅˆ†้กžใ€ ๅ›บๆœ‰่กจ็พๆŠฝๅ‡บใ€ ่ณชๅ•ๅฟœ็ญ”ใ€ ่จ€่ชžใƒขใƒ‡ใƒชใƒณใ‚ฐใ€ ๆ–‡็ซ ่ฆ็ด„ใ€ ๆฉŸๆขฐ็ฟป่จณใ€ ่ค‡ๆ•ฐ้ธๆŠžใ€ใƒ†ใ‚ญใ‚นใƒˆ็”Ÿๆˆใ€‚
๐Ÿ–ผ๏ธ ใ‚ณใƒณใƒ”ใƒฅใƒผใ‚ฟใƒ“ใ‚ธใƒงใƒณ: ็”ปๅƒๅˆ†้กžใ€ ็‰ฉไฝ“ๆคœๅ‡บใ€ ใ‚ปใ‚ฐใƒกใƒณใƒ†ใƒผใ‚ทใƒงใƒณใ€‚
๐Ÿ—ฃ๏ธ ้Ÿณๅฃฐ: ่‡ชๅ‹•้Ÿณๅฃฐ่ช่ญ˜ใ€้Ÿณๅฃฐๅˆ†้กžใ€‚
๐Ÿ™ ใƒžใƒซใƒใƒขใƒผใƒ€ใƒซ: ใƒ†ใƒผใƒ–ใƒซ่ณชๅ•ๅฟœ็ญ”ใ€ ๅ…‰ๅญฆๆ–‡ๅญ—่ช่ญ˜(OCR)ใ€ ใ‚นใ‚ญใƒฃใƒณใ•ใ‚ŒใŸใƒ‰ใ‚ญใƒฅใƒกใƒณใƒˆใ‹ใ‚‰ใฎๆƒ…ๅ ฑๆŠฝๅ‡บใ€ ๅ‹•็”ปๅˆ†้กžใ€ visual question answering(่ฆ–่ฆš็š„่ณชๅ•ๅฟœ็ญ”)ใ€‚

๐Ÿค— Transformers ใฏPyTorch, TensorFlow, JAX้–“ใฎใƒ•ใƒฌใƒผใƒ ใƒฏใƒผใ‚ฏ็›ธไบ’้‹็”จๆ€งใ‚’ใ‚ตใƒใƒผใƒˆใ—ใฆใ„ใพใ™ใ€‚ ใ“ใ‚Œใฏใƒขใƒ‡ใƒซใฎๅ„ๆฎต้šŽใง็•ฐใชใ‚‹ใƒ•ใƒฌใƒผใƒ ใƒฏใƒผใ‚ฏใ‚’ไฝฟใ†ใŸใ‚ใฎๆŸ”่ปŸๆ€งใ‚’ๆไพ›ใ—ใพใ™ใ€‚ใ‚ใ‚‹ใƒ•ใƒฌใƒผใƒ ใƒฏใƒผใ‚ฏใง3่กŒใฎใ‚ณใƒผใƒ‰ใงใƒขใƒ‡ใƒซใ‚’ๅญฆ็ฟ’ใ—ใ€ๅˆฅใฎใƒ•ใƒฌใƒผใƒ ใƒฏใƒผใ‚ฏใงๆŽจ่ซ–ใฎใŸใ‚ใซใƒขใƒ‡ใƒซใ‚’ใƒญใƒผใƒ‰ใ™ใ‚‹ใ“ใจใŒๅฏ่ƒฝใงใ™ใ€‚ใพใŸใ€ๆœฌ็•ช็’ฐๅขƒใฎใƒ‡ใƒ—ใƒญใ‚คใฎใŸใ‚ใซใƒขใƒ‡ใƒซใ‚’ONNXใ‚„TorchScriptใฎใ‚ˆใ†ใชๅฝขๅผใงใ‚จใ‚ฏใ‚นใƒใƒผใƒˆใ™ใ‚‹ใ“ใจใ‚‚ๅฏ่ƒฝใงใ™ใ€‚

Hub, forum, Discordใงๆˆ้•ทไธญใฎใ‚ณใƒŸใƒฅใƒ‹ใƒ†ใ‚ฃใซไปŠๆ—ฅๅ‚ๅŠ ใ—ใพใ—ใ‚‡ใ†๏ผ

Hugging Faceใƒใƒผใƒ ใซใ‚ˆใ‚‹ใ‚ซใ‚นใ‚ฟใƒ ใ‚ตใƒใƒผใƒˆใ‚’ใ”ๅธŒๆœ›ใฎๅ ดๅˆ

HuggingFace Expert Acceleration Program

็›ฎๆฌก

ใƒ‰ใ‚ญใƒฅใƒกใƒณใƒˆใฏไปฅไธ‹ใฎ5ใคใฎใ‚ปใ‚ฏใ‚ทใƒงใƒณใงๆง‹ๆˆใ•ใ‚Œใฆใ„ใพใ™:

  • ใฏใ˜ใ‚ใซ ใฏใ€ใƒฉใ‚คใƒ–ใƒฉใƒชใฎใ‚ฏใ‚คใƒƒใ‚ฏใƒ„ใ‚ขใƒผใจใƒฉใ‚คใƒ–ใƒฉใƒชใ‚’ไฝฟใ„ๅง‹ใ‚ใ‚‹ใŸใ‚ใฎใ‚คใƒณใ‚นใƒˆใƒผใƒซๆ‰‹้ †ใ‚’ๆไพ›ใ—ใฆใ„ใพใ™ใ€‚

  • ใƒใƒฅใƒผใƒˆใƒชใ‚ขใƒซ ใฏใ€ๅˆๅฟƒ่€…ใŒๅง‹ใ‚ใ‚‹ใฎใซๆœ€้ฉใชๅ ดๆ‰€ใงใ™ใ€‚ใ“ใฎใ‚ปใ‚ฏใ‚ทใƒงใƒณใงใฏใ€ใƒฉใ‚คใƒ–ใƒฉใƒชใ‚’ไฝฟใ„ๅง‹ใ‚ใ‚‹ใŸใ‚ใซๅฟ…่ฆใชๅŸบๆœฌ็š„ใชใ‚นใ‚ญใƒซใ‚’็ฟ’ๅพ—ใงใใพใ™ใ€‚

  • HOW-TOใ‚ฌใ‚คใƒ‰ ใฏใ€่จ€่ชžใƒขใƒ‡ใƒชใƒณใ‚ฐใฎใŸใ‚ใซๅญฆ็ฟ’ๆธˆใฟใƒขใƒ‡ใƒซใ‚’finetuningใ™ใ‚‹ใ“ใจใ‚„ใ‚ซใ‚นใ‚ฟใƒ ใƒขใƒ‡ใƒซใฎไฝœๆˆใจๅ…ฑๆœ‰ใฎๆ–นๆณ•ใชใฉใจใ„ใฃใŸ็‰นๅฎšใฎ็›ฎๆจ™ใ‚’้”ๆˆใ™ใ‚‹ใŸใ‚ใฎๆ–นๆณ•ใ‚’็คบใ—ใฆใ„ใพใ™ใ€‚

  • ใ‚ณใƒณใ‚ปใƒ—ใƒˆใ‚ฌใ‚คใƒ‰ ใฏใ€ใƒขใƒ‡ใƒซใ‚„ใ‚ฟใ‚นใ‚ฏใ€ใใ—ใฆ ๐Ÿค— Transformersใฎ่จญ่จˆๆ€ๆƒณใฎ่ƒŒๆ™ฏใซใ‚ใ‚‹ๅŸบๆœฌ็š„ใซใ‚ณใƒณใ‚ปใƒ—ใƒˆใ‚„่€ƒใˆๆ–นใซใคใ„ใฆใ‚ˆใ‚Šๆทฑใ่€ƒๅฏŸใ—่งฃ่ชฌใ—ใฆใ„ใพใ™ใ€‚

  • API ๅ…จใฆใฎใ‚ฏใƒฉใ‚นใจ้–ขๆ•ฐใ‚’่ชฌๆ˜Žใ—ใพใ™:

    • MAIN CLASSES ใฏใ€configuration, model, tokenizer, pipelineใจใ„ใฃใŸๆœ€ใ‚‚้‡่ฆใชใ‚ฏใƒฉใ‚นใซใคใ„ใฆ่ฉณ็ดฐใซ่ชฌๆ˜Žใ—ใฆใ„ใพใ™ใ€‚
    • MODELS ใฏใ€ใƒฉใ‚คใƒ–ใƒฉใƒชใงๅฎŸ่ฃ…ใ•ใ‚Œใฆใ„ใ‚‹ใใ‚Œใžใ‚Œใฎใƒขใƒ‡ใƒซใซ้–ข้€ฃใ—ใŸใ‚ฏใƒฉใ‚นใจ้–ขๆ•ฐใ‚’่ฉณ็ดฐใซ่ชฌๆ˜Žใ—ใฆใ„ใพใ™ใ€‚
    • INTERNAL HELPERS ใฏใ€ๅ†…้ƒจใงไฝฟ็”จใ•ใ‚Œใฆใ„ใ‚‹ใƒฆใƒผใƒ†ใ‚ฃใƒชใƒ†ใ‚ฃใ‚ฏใƒฉใ‚นใ‚„้–ขๆ•ฐใ‚’่ฉณ็ดฐใซ่ชฌๆ˜Žใ—ใฆใ„ใพใ™ใ€‚

ใ‚ตใƒใƒผใƒˆใ•ใ‚Œใฆใ„ใ‚‹ใƒขใƒ‡ใƒซ

  1. ALBERT (Google Research and the Toyota Technological Institute at Chicago ใ‹ใ‚‰) Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, Radu Soricut ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
  2. AltCLIP (BAAI ใ‹ใ‚‰) Chen, Zhongzhi and Liu, Guang and Zhang, Bo-Wen and Ye, Fulong and Yang, Qinghong and Wu, Ledell ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities
  3. Audio Spectrogram Transformer (MIT ใ‹ใ‚‰) Yuan Gong, Yu-An Chung, James Glass ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: AST: Audio Spectrogram Transformer
  4. BART (Facebook ใ‹ใ‚‰) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov and Luke Zettlemoyer ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
  5. BARThez (ร‰cole polytechnique ใ‹ใ‚‰) Moussa Kamal Eddine, Antoine J.-P. Tixier, Michalis Vazirgiannis ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: BARThez: a Skilled Pretrained French Sequence-to-Sequence Model
  6. BARTpho (VinAI Research ใ‹ใ‚‰) Nguyen Luong Tran, Duong Minh Le and Dat Quoc Nguyen ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese
  7. BEiT (Microsoft ใ‹ใ‚‰) Hangbo Bao, Li Dong, Furu Wei ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: BEiT: BERT Pre-Training of Image Transformers
  8. BERT (Google ใ‹ใ‚‰) Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  9. BERT For Sequence Generation (Google ใ‹ใ‚‰) Sascha Rothe, Shashi Narayan, Aliaksei Severyn ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Leveraging Pre-trained Checkpoints for Sequence Generation Tasks
  10. BERTweet (VinAI Research ใ‹ใ‚‰) Dat Quoc Nguyen, Thanh Vu and Anh Tuan Nguyen ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: BERTweet: A pre-trained language model for English Tweets
  11. BigBird-Pegasus (Google Research ใ‹ใ‚‰) Manzil Zaheer, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, Amr Ahmed ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Big Bird: Transformers for Longer Sequences
  12. BigBird-RoBERTa (Google Research ใ‹ใ‚‰) Manzil Zaheer, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, Amr Ahmed ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Big Bird: Transformers for Longer Sequences
  13. BioGpt (Microsoft Research AI4Science ใ‹ใ‚‰) Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon and Tie-Yan Liu ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: BioGPT: generative pre-trained transformer for biomedical text generation and mining
  14. BiT (Google AI ใ‹ใ‚‰) Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, Neil ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Big Transfer (BiT)Houlsby.
  15. Blenderbot (Facebook ใ‹ใ‚‰) Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M. Smith, Y-Lan Boureau, Jason Weston ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Recipes for building an open-domain chatbot
  16. BlenderbotSmall (Facebook ใ‹ใ‚‰) Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M. Smith, Y-Lan Boureau, Jason Weston ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Recipes for building an open-domain chatbot
  17. BLIP (Salesforce ใ‹ใ‚‰) Junnan Li, Dongxu Li, Caiming Xiong, Steven Hoi ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
  18. BLOOM (BigScience workshop ใ‹ใ‚‰) BigScience Workshop ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚Œใพใ—ใŸ.
  19. BORT (Alexa ใ‹ใ‚‰) Adrian de Wynter and Daniel J. Perry ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Optimal Subarchitecture Extraction For BERT
  20. ByT5 (Google Research ใ‹ใ‚‰) Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: ByT5: Towards a token-free future with pre-trained byte-to-byte models
  21. CamemBERT (Inria/Facebook/Sorbonne ใ‹ใ‚‰) Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suรกrez*, Yoann Dupont, Laurent Romary, ร‰ric Villemonte de la Clergerie, Djamรฉ Seddah and Benoรฎt Sagot ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: CamemBERT: a Tasty French Language Model
  22. CANINE (Google Research ใ‹ใ‚‰) Jonathan H. Clark, Dan Garrette, Iulia Turc, John Wieting ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation
  23. Chinese-CLIP (OFA-Sys ใ‹ใ‚‰) An Yang, Junshu Pan, Junyang Lin, Rui Men, Yichang Zhang, Jingren Zhou, Chang Zhou ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese
  24. CLIP (OpenAI ใ‹ใ‚‰) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Learning Transferable Visual Models From Natural Language Supervision
  25. CLIPSeg (University of Gรถttingen ใ‹ใ‚‰) Timo Lรผddecke and Alexander Ecker ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Image Segmentation Using Text and Image Prompts
  26. CodeGen (Salesforce ใ‹ใ‚‰) Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, Caiming Xiong ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: A Conversational Paradigm for Program Synthesis
  27. Conditional DETR (Microsoft Research Asia ใ‹ใ‚‰) Depu Meng, Xiaokang Chen, Zejia Fan, Gang Zeng, Houqiang Li, Yuhui Yuan, Lei Sun, Jingdong Wang ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Conditional DETR for Fast Training Convergence
  28. ConvBERT (YituTech ใ‹ใ‚‰) Zihang Jiang, Weihao Yu, Daquan Zhou, Yunpeng Chen, Jiashi Feng, Shuicheng Yan ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: ConvBERT: Improving BERT with Span-based Dynamic Convolution
  29. ConvNeXT (Facebook AI ใ‹ใ‚‰) Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, Saining Xie ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: A ConvNet for the 2020s
  30. ConvNeXTV2 (from Facebook AI) released with the paper ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders by Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, Saining Xie.
  31. CPM (Tsinghua University ใ‹ใ‚‰) Zhengyan Zhang, Xu Han, Hao Zhou, Pei Ke, Yuxian Gu, Deming Ye, Yujia Qin, Yusheng Su, Haozhe Ji, Jian Guan, Fanchao Qi, Xiaozhi Wang, Yanan Zheng, Guoyang Zeng, Huanqi Cao, Shengqi Chen, Daixuan Li, Zhenbo Sun, Zhiyuan Liu, Minlie Huang, Wentao Han, Jie Tang, Juanzi Li, Xiaoyan Zhu, Maosong Sun ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: CPM: A Large-scale Generative Chinese Pre-trained Language Model
  32. CTRL (Salesforce ใ‹ใ‚‰) Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong and Richard Socher ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: CTRL: A Conditional Transformer Language Model for Controllable Generation
  33. CvT (Microsoft ใ‹ใ‚‰) Haiping Wu, Bin Xiao, Noel Codella, Mengchen Liu, Xiyang Dai, Lu Yuan, Lei Zhang ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: CvT: Introducing Convolutions to Vision Transformers
  34. Data2Vec (Facebook ใ‹ใ‚‰) Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, Michael Auli ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Data2Vec: A General Framework for Self-supervised Learning in Speech, Vision and Language
  35. DeBERTa (Microsoft ใ‹ใ‚‰) Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: DeBERTa: Decoding-enhanced BERT with Disentangled Attention
  36. DeBERTa-v2 (Microsoft ใ‹ใ‚‰) Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: DeBERTa: Decoding-enhanced BERT with Disentangled Attention
  37. Decision Transformer (Berkeley/Facebook/Google ใ‹ใ‚‰) Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, Igor Mordatch ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Decision Transformer: Reinforcement Learning via Sequence Modeling
  38. Deformable DETR (SenseTime Research ใ‹ใ‚‰) Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, Jifeng Dai ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Deformable DETR: Deformable Transformers for End-to-End Object Detection
  39. DeiT (Facebook ใ‹ใ‚‰) Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, Hervรฉ Jรฉgou ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Training data-efficient image transformers & distillation through attention
  40. DETR (Facebook ใ‹ใ‚‰) Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: End-to-End Object Detection with Transformers
  41. DialoGPT (Microsoft Research ใ‹ใ‚‰) Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation
  42. DiNAT (SHI Labs ใ‹ใ‚‰) Ali Hassani and Humphrey Shi ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Dilated Neighborhood Attention Transformer
  43. DistilBERT (HuggingFace ใ‹ใ‚‰), Victor Sanh, Lysandre Debut and Thomas Wolf. ๅŒใ˜ๆ‰‹ๆณ•ใง GPT2, RoBERTa ใจ Multilingual BERT ใฎๅœง็ธฎใ‚’่กŒใ„ใพใ—ใŸ.ๅœง็ธฎใ•ใ‚ŒใŸใƒขใƒ‡ใƒซใฏใใ‚Œใžใ‚Œ DistilGPT2ใ€DistilRoBERTaใ€DistilmBERT ใจๅไป˜ใ‘ใ‚‰ใ‚Œใพใ—ใŸ. ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
  44. DiT (Microsoft Research ใ‹ใ‚‰) Junlong Li, Yiheng Xu, Tengchao Lv, Lei Cui, Cha Zhang, Furu Wei ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: DiT: Self-supervised Pre-training for Document Image Transformer
  45. Donut (NAVER ใ‹ใ‚‰), Geewook Kim, Teakgyu Hong, Moonbin Yim, Jeongyeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, Seunghyun Park ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: OCR-free Document Understanding Transformer
  46. DPR (Facebook ใ‹ใ‚‰) Vladimir Karpukhin, Barlas OฤŸuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Dense Passage Retrieval for Open-Domain Question Answering
  47. DPT (Intel Labs ใ‹ใ‚‰) Renรฉ Ranftl, Alexey Bochkovskiy, Vladlen Koltun ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Vision Transformers for Dense Prediction
  48. EfficientNet (from Google Research) released with the paper EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks by Mingxing Tan and Quoc V. Le.
  49. ELECTRA (Google Research/Stanford University ใ‹ใ‚‰) Kevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. Manning ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: ELECTRA: Pre-training text encoders as discriminators rather than generators
  50. EncoderDecoder (Google Research ใ‹ใ‚‰) Sascha Rothe, Shashi Narayan, Aliaksei Severyn ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Leveraging Pre-trained Checkpoints for Sequence Generation Tasks
  51. ERNIE (Baidu ใ‹ใ‚‰) Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, Hua Wu ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: ERNIE: Enhanced Representation through Knowledge Integration
  52. ESM (Meta AI ใ‹ใ‚‰) ใฏใƒˆใƒฉใƒณใ‚นใƒ•ใ‚ฉใƒผใƒžใƒผใƒ—ใƒญใƒ†ใ‚คใƒณ่จ€่ชžใƒขใƒ‡ใƒซใงใ™. ESM-1b ใฏ Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C. Lawrence Zitnick, Jerry Ma, and Rob Fergus ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. ESM-1v ใฏ Joshua Meier, Roshan Rao, Robert Verkuil, Jason Liu, Tom Sercu and Alexander Rivesใ€€ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Language models enable zero-shot prediction of the effects of mutations on protein function. ESM-2 ใจใ€€ESMFold ใฏ Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, Alexander Rives ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Language models of protein sequences at the scale of evolution enable accurate structure prediction
  53. FLAN-T5 (Google AI ใ‹ใ‚‰) Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸใƒฌใƒใ‚ธใƒˆใƒชใƒผ google-research/t5x Le, and Jason Wei
  54. FlauBERT (CNRS ใ‹ใ‚‰) Hang Le, Loรฏc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoรฎt Crabbรฉ, Laurent Besacier, Didier Schwab ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: FlauBERT: Unsupervised Language Model Pre-training for French
  55. FLAVA (Facebook AI ใ‹ใ‚‰) Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guillaume Couairon, Wojciech Galuba, Marcus Rohrbach, and Douwe Kiela ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: FLAVA: A Foundational Language And Vision Alignment Model
  56. FNet (Google Research ใ‹ใ‚‰) James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, Santiago Ontanon ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: FNet: Mixing Tokens with Fourier Transforms
  57. Funnel Transformer (CMU/Google Brain ใ‹ใ‚‰) Zihang Dai, Guokun Lai, Yiming Yang, Quoc V. Le ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing
  58. GIT (Microsoft Research ใ‹ใ‚‰) Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, Lijuan Wang. ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡ GIT: A Generative Image-to-text Transformer for Vision and Language
  59. GLPN (KAIST ใ‹ใ‚‰) Doyeon Kim, Woonghyun Ga, Pyungwhan Ahn, Donggyu Joo, Sehwan Chun, Junmo Kim ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepth
  60. GPT (OpenAI ใ‹ใ‚‰) Alec Radford, Karthik Narasimhan, Tim Salimans and Ilya Sutskever ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Improving Language Understanding by Generative Pre-Training
  61. GPT Neo (EleutherAI ใ‹ใ‚‰) Sid Black, Stella Biderman, Leo Gao, Phil Wang and Connor Leahy ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸใƒฌใƒใ‚ธใƒˆใƒชใƒผ : EleutherAI/gpt-neo
  62. GPT NeoX (EleutherAI ใ‹ใ‚‰) Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, Samuel Weinbach ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: GPT-NeoX-20B: An Open-Source Autoregressive Language Model
  63. GPT NeoX Japanese (ABEJA ใ‹ใ‚‰) Shinya Otani, Takayoshi Makabe, Anuj Arora, and Kyo Hattori ใ‹ใ‚‰ใƒชใƒชใƒผใ‚น.
  64. GPT-2 (OpenAI ใ‹ใ‚‰) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei and Ilya Sutskever ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Language Models are Unsupervised Multitask Learners
  65. GPT-J (EleutherAI ใ‹ใ‚‰) Ben Wang and Aran Komatsuzaki ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸใƒฌใƒใ‚ธใƒˆใƒชใƒผ kingoflolz/mesh-transformer-jax
  66. GPT-Sw3 (AI-Sweden ใ‹ใ‚‰) Ariel Ekgren, Amaru Cuba Gyllensten, Evangelia Gogoulou, Alice Heiman, Severine Verlinden, Joey ร–hman, Fredrik Carlsson, Magnus Sahlgren ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Lessons Learned from GPT-SW3: Building the First Large-Scale Generative Language Model for Swedish
  67. GroupViT (UCSD, NVIDIA ใ‹ใ‚‰) Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, Xiaolong Wang ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: GroupViT: Semantic Segmentation Emerges from Text Supervision
  68. Hubert (Facebook ใ‹ใ‚‰) Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
  69. I-BERT (Berkeley ใ‹ใ‚‰) Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: I-BERT: Integer-only BERT Quantization
  70. ImageGPT (OpenAI ใ‹ใ‚‰) Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Generative Pretraining from Pixels
  71. Jukebox (OpenAI ใ‹ใ‚‰) Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Jukebox: A Generative Model for Music
  72. LayoutLM (Microsoft Research Asia ใ‹ใ‚‰) Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: LayoutLM: Pre-training of Text and Layout for Document Image Understanding
  73. LayoutLMv2 (Microsoft Research Asia ใ‹ใ‚‰) Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
  74. LayoutLMv3 (Microsoft Research Asia ใ‹ใ‚‰) Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, Furu Wei ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
  75. LayoutXLM (Microsoft Research Asia ใ‹ใ‚‰) Yiheng Xu, Tengchao Lv, Lei Cui, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Furu Wei ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding
  76. LED (AllenAI ใ‹ใ‚‰) Iz Beltagy, Matthew E. Peters, Arman Cohan ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Longformer: The Long-Document Transformer
  77. LeViT (Meta AI ใ‹ใ‚‰) Ben Graham, Alaaeldin El-Nouby, Hugo Touvron, Pierre Stock, Armand Joulin, Hervรฉ Jรฉgou, Matthijs Douze ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: LeViT: A Vision Transformer in ConvNetโ€™s Clothing for Faster Inference
  78. LiLT (South China University of Technology ใ‹ใ‚‰) Jiapeng Wang, Lianwen Jin, Kai Ding ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding
  79. Longformer (AllenAI ใ‹ใ‚‰) Iz Beltagy, Matthew E. Peters, Arman Cohan ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Longformer: The Long-Document Transformer
  80. LongT5 (Google AI ใ‹ใ‚‰) Mandy Guo, Joshua Ainslie, David Uthus, Santiago Ontanon, Jianmo Ni, Yun-Hsuan Sung, Yinfei Yang ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: LongT5: Efficient Text-To-Text Transformer for Long Sequences
  81. LUKE (Studio Ousia ใ‹ใ‚‰) Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda, Yuji Matsumoto ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
  82. LXMERT (UNC Chapel Hill ใ‹ใ‚‰) Hao Tan and Mohit Bansal ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: LXMERT: Learning Cross-Modality Encoder Representations from Transformers for Open-Domain Question Answering
  83. M-CTC-T (Facebook ใ‹ใ‚‰) Loren Lugosch, Tatiana Likhomanenko, Gabriel Synnaeve, and Ronan Collobert ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Pseudo-Labeling For Massively Multilingual Speech Recognition
  84. M2M100 (Facebook ใ‹ใ‚‰) Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Edouard Grave, Michael Auli, Armand Joulin ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Beyond English-Centric Multilingual Machine Translation
  85. MarianMT Jรถrg Tiedemann ใ‹ใ‚‰. OPUS ใ‚’ไฝฟใ„ใชใŒใ‚‰ๅญฆ็ฟ’ใ•ใ‚ŒใŸ โ€œMachine translationโ€ (ใƒžใ‚ทใƒณใƒˆใƒฉใƒณใ‚นใƒฌใƒผใ‚ทใƒงใƒณ) ใƒขใƒ‡ใƒซ. Marian Framework ใฏMicrosoft Translator Teamใ€€ใŒ็พๅœจ้–‹็™บไธญใงใ™.
  86. MarkupLM (Microsoft Research Asia ใ‹ใ‚‰) Junlong Li, Yiheng Xu, Lei Cui, Furu Wei ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understanding
  87. Mask2Former (FAIR and UIUC ใ‹ใ‚‰) Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexander Kirillov, Rohit Girdhar. ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡ Masked-attention Mask Transformer for Universal Image Segmentation
  88. MaskFormer (Meta and UIUC ใ‹ใ‚‰) Bowen Cheng, Alexander G. Schwing, Alexander Kirillov ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Per-Pixel Classification is Not All You Need for Semantic Segmentation
  89. mBART (Facebook ใ‹ใ‚‰) Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, Luke Zettlemoyer ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Multilingual Denoising Pre-training for Neural Machine Translation
  90. mBART-50 (Facebook ใ‹ใ‚‰) Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, Angela Fan ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Multilingual Translation with Extensible Multilingual Pretraining and Finetuning
  91. Megatron-BERT (NVIDIA ใ‹ใ‚‰) Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper and Bryan Catanzaro ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
  92. Megatron-GPT2 (NVIDIA ใ‹ใ‚‰) Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper and Bryan Catanzaro ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
  93. mLUKE (Studio Ousia ใ‹ใ‚‰) Ryokan Ri, Ikuya Yamada, and Yoshimasa Tsuruoka ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models
  94. MobileBERT (CMU/Google Brain ใ‹ใ‚‰) Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, and Denny Zhou ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
  95. MobileNetV1 (Google Inc. ใ‹ใ‚‰) Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, Hartwig Adam ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
  96. MobileNetV2 (Google Inc. ใ‹ใ‚‰) Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, Liang-Chieh Chen ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: MobileNetV2: Inverted Residuals and Linear Bottlenecks
  97. MobileViT (Apple ใ‹ใ‚‰) Sachin Mehta and Mohammad Rastegari ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer
  98. MPNet (Microsoft Research ใ‹ใ‚‰) Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, Tie-Yan Liu ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: MPNet: Masked and Permuted Pre-training for Language Understanding
  99. MT5 (Google AI ใ‹ใ‚‰) Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: mT5: A massively multilingual pre-trained text-to-text transformer
  100. MVP (RUC AI Box ใ‹ใ‚‰) Tianyi Tang, Junyi Li, Wayne Xin Zhao and Ji-Rong Wen ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: MVP: Multi-task Supervised Pre-training for Natural Language Generation
  101. NAT (SHI Labs ใ‹ใ‚‰) Ali Hassani, Steven Walton, Jiachen Li, Shen Li, and Humphrey Shi ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Neighborhood Attention Transformer
  102. Nezha (Huawei Noahโ€™s Ark Lab ใ‹ใ‚‰) Junqiu Wei, Xiaozhe Ren, Xiaoguang Li, Wenyong Huang, Yi Liao, Yasheng Wang, Jiashu Lin, Xin Jiang, Xiao Chen and Qun Liu ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: NEZHA: Neural Contextualized Representation for Chinese Language Understanding
  103. NLLB (Meta ใ‹ใ‚‰) the NLLB team ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: No Language Left Behind: Scaling Human-Centered Machine Translation
  104. Nystrรถmformer (the University of Wisconsin - Madison ใ‹ใ‚‰) Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, Mingxing Tan, Glenn Fung, Yin Li, Vikas Singh ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Nystrรถmformer: A Nystrรถm-Based Algorithm for Approximating Self-Attention
  105. OneFormer (SHI Labs ใ‹ใ‚‰) Jitesh Jain, Jiachen Li, MangTik Chiu, Ali Hassani, Nikita Orlov, Humphrey Shi ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: OneFormer: One Transformer to Rule Universal Image Segmentation
  106. OPT (Meta AI ใ‹ใ‚‰) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen et al ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: OPT: Open Pre-trained Transformer Language Models
  107. OWL-ViT (Google AI ใ‹ใ‚‰) Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, Xiao Wang, Xiaohua Zhai, Thomas Kipf, and Neil Houlsby ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Simple Open-Vocabulary Object Detection with Vision Transformers
  108. Pegasus (Google ใ‹ใ‚‰) Jingqing Zhang, Yao Zhao, Mohammad Saleh and Peter J. Liu ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
  109. PEGASUS-X (Google ใ‹ใ‚‰) Jason Phang, Yao Zhao, and Peter J. Liu ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Investigating Efficiently Extending Transformers for Long Input Summarization
  110. Perceiver IO (Deepmind ใ‹ใ‚‰) Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch, Catalin Ionescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, Olivier Hรฉnaff, Matthew M. Botvinick, Andrew Zisserman, Oriol Vinyals, Joรฃo Carreira ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Perceiver IO: A General Architecture for Structured Inputs & Outputs
  111. PhoBERT (VinAI Research ใ‹ใ‚‰) Dat Quoc Nguyen and Anh Tuan Nguyen ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: PhoBERT: Pre-trained language models for Vietnamese
  112. PLBart (UCLA NLP ใ‹ใ‚‰) Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Unified Pre-training for Program Understanding and Generation
  113. PoolFormer (Sea AI Labs ใ‹ใ‚‰) Yu, Weihao and Luo, Mi and Zhou, Pan and Si, Chenyang and Zhou, Yichen and Wang, Xinchao and Feng, Jiashi and Yan, Shuicheng ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: MetaFormer is Actually What You Need for Vision
  114. ProphetNet (Microsoft Research ใ‹ใ‚‰) Yu Yan, Weizhen Qi, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang and Ming Zhou ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
  115. QDQBert (NVIDIA ใ‹ใ‚‰) Hao Wu, Patrick Judd, Xiaojie Zhang, Mikhail Isaev and Paulius Micikevicius ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation
  116. RAG (Facebook ใ‹ใ‚‰) Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kรผttler, Mike Lewis, Wen-tau Yih, Tim Rocktรคschel, Sebastian Riedel, Douwe Kiela ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  117. REALM (Google Research ใ‹ใ‚‰) Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat and Ming-Wei Chang ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: REALM: Retrieval-Augmented Language Model Pre-Training
  118. Reformer (Google Research ใ‹ใ‚‰) Nikita Kitaev, ลukasz Kaiser, Anselm Levskaya ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Reformer: The Efficient Transformer
  119. RegNet (META Platforms ใ‹ใ‚‰) Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, Piotr Dollรกr ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Designing Network Design Space
  120. RemBERT (Google Research ใ‹ใ‚‰) Hyung Won Chung, Thibault Fรฉvry, Henry Tsai, M. Johnson, Sebastian Ruder ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Rethinking embedding coupling in pre-trained language models
  121. ResNet (Microsoft Research ใ‹ใ‚‰) Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Deep Residual Learning for Image Recognition
  122. RoBERTa (Facebook ใ‹ใ‚‰), Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: RoBERTa: A Robustly Optimized BERT Pretraining Approach
  123. RoBERTa-PreLayerNorm (Facebook ใ‹ใ‚‰) Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, Michael Auli ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: fairseq: A Fast, Extensible Toolkit for Sequence Modeling
  124. RoCBert (WeChatAI ใ‹ใ‚‰) HuiSu, WeiweiShi, XiaoyuShen, XiaoZhou, TuoJi, JiaruiFang, JieZhou ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: RoCBert: Robust Chinese Bert with Multimodal Contrastive Pretraining
  125. RoFormer (ZhuiyiTechnology ใ‹ใ‚‰), Jianlin Su and Yu Lu and Shengfeng Pan and Bo Wen and Yunfeng Liu ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: RoFormer: Enhanced Transformer with Rotary Position Embedding
  126. SegFormer (NVIDIA ใ‹ใ‚‰) Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M. Alvarez, Ping Luo ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
  127. SEW (ASAPP ใ‹ใ‚‰) Felix Wu, Kwangyoun Kim, Jing Pan, Kyu Han, Kilian Q. Weinberger, Yoav Artzi ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition
  128. SEW-D (ASAPP ใ‹ใ‚‰) Felix Wu, Kwangyoun Kim, Jing Pan, Kyu Han, Kilian Q. Weinberger, Yoav Artzi ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition
  129. SpeechToTextTransformer (Facebook ใ‹ใ‚‰), Changhan Wang, Yun Tang, Xutai Ma, Anne Wu, Dmytro Okhonko, Juan Pino ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: fairseq S2T: Fast Speech-to-Text Modeling with fairseq
  130. SpeechToTextTransformer2 (Facebook ใ‹ใ‚‰), Changhan Wang, Anne Wu, Juan Pino, Alexei Baevski, Michael Auli, Alexis Conneau ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Large-Scale Self- and Semi-Supervised Learning for Speech Translation
  131. Splinter (Tel Aviv University ใ‹ใ‚‰), Ori Ram, Yuval Kirstain, Jonathan Berant, Amir Globerson, Omer Levy ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Few-Shot Question Answering by Pretraining Span Selection
  132. SqueezeBERT (Berkeley ใ‹ใ‚‰) Forrest N. Iandola, Albert E. Shaw, Ravi Krishna, and Kurt W. Keutzer ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: SqueezeBERT: What can computer vision teach NLP about efficient neural networks?
  133. Swin Transformer (Microsoft ใ‹ใ‚‰) Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, Baining Guo ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
  134. Swin Transformer V2 (Microsoft ใ‹ใ‚‰) Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, Furu Wei, Baining Guo ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Swin Transformer V2: Scaling Up Capacity and Resolution
  135. Swin2SR (University of Wรผrzburg ใ‹ใ‚‰) Marcos V. Conde, Ui-Jin Choi, Maxime Burchi, Radu Timofte ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration
  136. SwitchTransformers (Google ใ‹ใ‚‰) William Fedus, Barret Zoph, Noam Shazeer ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
  137. T5 (Google AI ใ‹ใ‚‰) Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
  138. T5v1.1 (Google AI ใ‹ใ‚‰) Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸใƒฌใƒใ‚ธใƒˆใƒชใƒผ google-research/text-to-text-transfer-transformer
  139. Table Transformer (Microsoft Research ใ‹ใ‚‰) Brandon Smock, Rohith Pesala, Robin Abraham ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: PubTables-1M: Towards Comprehensive Table Extraction From Unstructured Documents
  140. TAPAS (Google AI ใ‹ใ‚‰) Jonathan Herzig, Paweล‚ Krzysztof Nowak, Thomas Mรผller, Francesco Piccinno and Julian Martin Eisenschlos ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: TAPAS: Weakly Supervised Table Parsing via Pre-training
  141. TAPEX (Microsoft Research ใ‹ใ‚‰) Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, Jian-Guang Lou ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: TAPEX: Table Pre-training via Learning a Neural SQL Executor
  142. Time Series Transformer (HuggingFace ใ‹ใ‚‰).
  143. TimeSformer (Facebook ใ‹ใ‚‰) Gedas Bertasius, Heng Wang, Lorenzo Torresani ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Is Space-Time Attention All You Need for Video Understanding?
  144. Trajectory Transformer (the University of California at Berkeley ใ‹ใ‚‰) Michael Janner, Qiyang Li, Sergey Levine ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Offline Reinforcement Learning as One Big Sequence Modeling Problem
  145. Transformer-XL (Google/CMU ใ‹ใ‚‰) Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, Ruslan Salakhutdinov ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
  146. TrOCR (Microsoft ใ‹ใ‚‰), Minghao Li, Tengchao Lv, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhoujun Li, Furu Wei ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models
  147. UL2 (Google Research ใ‹ใ‚‰) Yi Tay, Mostafa Dehghani, Vinh Q ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Unifying Language Learning Paradigms Tran, Xavier Garcia, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Neil Houlsby, Donald Metzler
  148. UniSpeech (Microsoft Research ใ‹ใ‚‰) Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, Xuedong Huang ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
  149. UniSpeechSat (Microsoft Research ใ‹ใ‚‰) Sanyuan Chen, Yu Wu, Chengyi Wang, Zhengyang Chen, Zhuo Chen, Shujie Liu, Jian Wu, Yao Qian, Furu Wei, Jinyu Li, Xiangzhan Yu ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: UNISPEECH-SAT: UNIVERSAL SPEECH REPRESENTATION LEARNING WITH SPEAKER AWARE PRE-TRAINING
  150. UPerNet (Peking University ใ‹ใ‚‰) Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, Jian Sun. ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡ Unified Perceptual Parsing for Scene Understanding
  151. VAN (Tsinghua University and Nankai University ใ‹ใ‚‰) Meng-Hao Guo, Cheng-Ze Lu, Zheng-Ning Liu, Ming-Ming Cheng, Shi-Min Hu ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Visual Attention Network
  152. VideoMAE (Multimedia Computing Group, Nanjing University ใ‹ใ‚‰) Zhan Tong, Yibing Song, Jue Wang, Limin Wang ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
  153. ViLT (NAVER AI Lab/Kakao Enterprise/Kakao Brain ใ‹ใ‚‰) Wonjae Kim, Bokyung Son, Ildoo Kim ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision
  154. Vision Transformer (ViT) (Google AI ใ‹ใ‚‰) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  155. VisualBERT (UCLA NLP ใ‹ใ‚‰) Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, Kai-Wei Chang ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: VisualBERT: A Simple and Performant Baseline for Vision and Language
  156. ViT Hybrid (Google AI ใ‹ใ‚‰) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  157. ViTMAE (Meta AI ใ‹ใ‚‰) Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollรกr, Ross Girshick ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Masked Autoencoders Are Scalable Vision Learners
  158. ViTMSN (Meta AI ใ‹ใ‚‰) Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Florian Bordes, Pascal Vincent, Armand Joulin, Michael Rabbat, Nicolas Ballas ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Masked Siamese Networks for Label-Efficient Learning
  159. Wav2Vec2 (Facebook AI ใ‹ใ‚‰) Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, Michael Auli ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
  160. Wav2Vec2-Conformer (Facebook AI ใ‹ใ‚‰) Changhan Wang, Yun Tang, Xutai Ma, Anne Wu, Sravya Popuri, Dmytro Okhonko, Juan Pino ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: FAIRSEQ S2T: Fast Speech-to-Text Modeling with FAIRSEQ
  161. Wav2Vec2Phoneme (Facebook AI ใ‹ใ‚‰) Qiantong Xu, Alexei Baevski, Michael Auli ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Simple and Effective Zero-shot Cross-lingual Phoneme Recognition
  162. WavLM (Microsoft Research ใ‹ใ‚‰) Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, Furu Wei ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
  163. Whisper (OpenAI ใ‹ใ‚‰) Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, Ilya Sutskever ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Robust Speech Recognition via Large-Scale Weak Supervision
  164. X-CLIP (Microsoft Research ใ‹ใ‚‰) Bolin Ni, Houwen Peng, Minghao Chen, Songyang Zhang, Gaofeng Meng, Jianlong Fu, Shiming Xiang, Haibin Ling ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Expanding Language-Image Pretrained Models for General Video Recognition
  165. XGLM (From Facebook AI) Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian Oโ€™Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov, Xian Li ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Few-shot Learning with Multilingual Language Models
  166. XLM (Facebook ใ‹ใ‚‰) Guillaume Lample and Alexis Conneau ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Cross-lingual Language Model Pretraining
  167. XLM-ProphetNet (Microsoft Research ใ‹ใ‚‰) Yu Yan, Weizhen Qi, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang and Ming Zhou ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
  168. XLM-RoBERTa (Facebook AI ใ‹ใ‚‰), Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmรกn, Edouard Grave, Myle Ott, Luke Zettlemoyer and Veselin Stoyanov ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Unsupervised Cross-lingual Representation Learning at Scale
  169. XLM-RoBERTa-XL (Facebook AI ใ‹ใ‚‰), Naman Goyal, Jingfei Du, Myle Ott, Giri Anantharaman, Alexis Conneau ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Larger-Scale Transformers for Multilingual Masked Language Modeling
  170. XLNet (Google/CMU ใ‹ใ‚‰) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, Quoc V. Le ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: โ€‹XLNet: Generalized Autoregressive Pretraining for Language Understanding
  171. XLS-R (Facebook AI ใ‹ใ‚‰) Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick von Platen, Yatharth Saraf, Juan Pino, Alexei Baevski, Alexis Conneau, Michael Auli ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
  172. XLSR-Wav2Vec2 (Facebook AI ใ‹ใ‚‰) Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, Michael Auli ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: Unsupervised Cross-Lingual Representation Learning For Speech Recognition
  173. YOLOS (Huazhong University of Science & Technology ใ‹ใ‚‰) Yuxin Fang, Bencheng Liao, Xinggang Wang, Jiemin Fang, Jiyang Qi, Rui Wu, Jianwei Niu, Wenyu Liu ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection
  174. YOSO (the University of Wisconsin - Madison ใ‹ใ‚‰) Zhanpeng Zeng, Yunyang Xiong, Sathya N. Ravi, Shailesh Acharya, Glenn Fung, Vikas Singh ใ‹ใ‚‰ๅ…ฌ้–‹ใ•ใ‚ŒใŸ็ ”็ฉถ่ซ–ๆ–‡: You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli Sampling

ใ‚ตใƒใƒผใƒˆใ•ใ‚Œใฆใ„ใ‚‹ใƒ•ใƒฌใƒผใƒ ใƒฏใƒผใ‚ฏ

ไปฅไธ‹ใฎใƒ†ใƒผใƒ–ใƒซใฏใใ‚Œใžใ‚Œใฎใƒขใƒ‡ใƒซใงใ‚ตใƒใƒผใƒˆใ•ใ‚Œใฆใ„ใ‚‹ใƒฉใ‚คใƒ–ใƒฉใƒชใ‚’็คบใ—ใฆใ„ใพใ™ใ€‚โ€œslowโ€ใจๅ‘ผใฐใ‚Œใ‚‹Pythonใƒˆใƒผใ‚ฏใƒŠใ‚คใ‚ถใƒผใ€๐Ÿค— Tokenizers ใƒฉใ‚คใƒ–ใƒฉใƒชใซใ‚ˆใ‚‹โ€fastโ€ใƒˆใƒผใ‚ฏใƒŠใ‚คใ‚ถใƒผใ€PyTorch, TensorFlow, Flaxใฎ5ใคใฎใใ‚Œใžใ‚ŒใŒใ‚ตใƒใƒผใƒˆใ•ใ‚Œใฆใ„ใ‚‹ใ‹ใ‚’็คบใ—ใฆใ„ใพใ™ใ€‚

Model Tokenizer slow Tokenizer fast PyTorch support TensorFlow support Flax Support
ALBERT โœ… โœ… โœ… โœ… โœ…
AltCLIP โŒ โŒ โœ… โŒ โŒ
Audio Spectrogram Transformer โŒ โŒ โœ… โŒ โŒ
BART โœ… โœ… โœ… โœ… โœ…
BEiT โŒ โŒ โœ… โŒ โœ…
BERT โœ… โœ… โœ… โœ… โœ…
Bert Generation โœ… โŒ โœ… โŒ โŒ
BigBird โœ… โœ… โœ… โŒ โœ…
BigBird-Pegasus โŒ โŒ โœ… โŒ โŒ
BioGpt โœ… โŒ โœ… โŒ โŒ
BiT โŒ โŒ โœ… โŒ โŒ
Blenderbot โœ… โœ… โœ… โœ… โœ…
BlenderbotSmall โœ… โœ… โœ… โœ… โœ…
BLIP โŒ โŒ โœ… โŒ โŒ
BLOOM โŒ โœ… โœ… โŒ โŒ
CamemBERT โœ… โœ… โœ… โœ… โŒ
CANINE โœ… โŒ โœ… โŒ โŒ
Chinese-CLIP โŒ โŒ โœ… โŒ โŒ
CLIP โœ… โœ… โœ… โœ… โœ…
CLIPSeg โŒ โŒ โœ… โŒ โŒ
CodeGen โœ… โœ… โœ… โŒ โŒ
Conditional DETR โŒ โŒ โœ… โŒ โŒ
ConvBERT โœ… โœ… โœ… โœ… โŒ
ConvNeXT โŒ โŒ โœ… โœ… โŒ
CTRL โœ… โŒ โœ… โœ… โŒ
CvT โŒ โŒ โœ… โœ… โŒ
Data2VecAudio โŒ โŒ โœ… โŒ โŒ
Data2VecText โŒ โŒ โœ… โŒ โŒ
Data2VecVision โŒ โŒ โœ… โœ… โŒ
DeBERTa โœ… โœ… โœ… โœ… โŒ
DeBERTa-v2 โœ… โœ… โœ… โœ… โŒ
Decision Transformer โŒ โŒ โœ… โŒ โŒ
Deformable DETR โŒ โŒ โœ… โŒ โŒ
DeiT โŒ โŒ โœ… โœ… โŒ
DETR โŒ โŒ โœ… โŒ โŒ
DiNAT โŒ โŒ โœ… โŒ โŒ
DistilBERT โœ… โœ… โœ… โœ… โœ…
DonutSwin โŒ โŒ โœ… โŒ โŒ
DPR โœ… โœ… โœ… โœ… โŒ
DPT โŒ โŒ โœ… โŒ โŒ
ELECTRA โœ… โœ… โœ… โœ… โœ…
Encoder decoder โŒ โŒ โœ… โœ… โœ…
ERNIE โŒ โŒ โœ… โŒ โŒ
ESM โœ… โŒ โœ… โœ… โŒ
FairSeq Machine-Translation โœ… โŒ โœ… โŒ โŒ
FlauBERT โœ… โŒ โœ… โœ… โŒ
FLAVA โŒ โŒ โœ… โŒ โŒ
FNet โœ… โœ… โœ… โŒ โŒ
Funnel Transformer โœ… โœ… โœ… โœ… โŒ
GIT โŒ โŒ โœ… โŒ โŒ
GLPN โŒ โŒ โœ… โŒ โŒ
GPT Neo โŒ โŒ โœ… โŒ โœ…
GPT NeoX โŒ โœ… โœ… โŒ โŒ
GPT NeoX Japanese โœ… โŒ โœ… โŒ โŒ
GPT-J โŒ โŒ โœ… โœ… โœ…
GPT-Sw3 โœ… โœ… โœ… โœ… โœ…
GroupViT โŒ โŒ โœ… โœ… โŒ
Hubert โŒ โŒ โœ… โœ… โŒ
I-BERT โŒ โŒ โœ… โŒ โŒ
ImageGPT โŒ โŒ โœ… โŒ โŒ
Jukebox โœ… โŒ โœ… โŒ โŒ
LayoutLM โœ… โœ… โœ… โœ… โŒ
LayoutLMv2 โœ… โœ… โœ… โŒ โŒ
LayoutLMv3 โœ… โœ… โœ… โœ… โŒ
LED โœ… โœ… โœ… โœ… โŒ
LeViT โŒ โŒ โœ… โŒ โŒ
LiLT โŒ โŒ โœ… โŒ โŒ
Longformer โœ… โœ… โœ… โœ… โŒ
LongT5 โŒ โŒ โœ… โŒ โœ…
LUKE โœ… โŒ โœ… โŒ โŒ
LXMERT โœ… โœ… โœ… โœ… โŒ
M-CTC-T โŒ โŒ โœ… โŒ โŒ
M2M100 โœ… โŒ โœ… โŒ โŒ
Marian โœ… โŒ โœ… โœ… โœ…
MarkupLM โœ… โœ… โœ… โŒ โŒ
Mask2Former โŒ โŒ โœ… โŒ โŒ
MaskFormer โŒ โŒ โœ… โŒ โŒ
MaskFormerSwin โŒ โŒ โŒ โŒ โŒ
mBART โœ… โœ… โœ… โœ… โœ…
Megatron-BERT โŒ โŒ โœ… โŒ โŒ
MobileBERT โœ… โœ… โœ… โœ… โŒ
MobileNetV1 โŒ โŒ โœ… โŒ โŒ
MobileNetV2 โŒ โŒ โœ… โŒ โŒ
MobileViT โŒ โŒ โœ… โœ… โŒ
MPNet โœ… โœ… โœ… โœ… โŒ
MT5 โœ… โœ… โœ… โœ… โœ…
MVP โœ… โœ… โœ… โŒ โŒ
NAT โŒ โŒ โœ… โŒ โŒ
Nezha โŒ โŒ โœ… โŒ โŒ
Nystrรถmformer โŒ โŒ โœ… โŒ โŒ
OpenAI GPT โœ… โœ… โœ… โœ… โŒ
OpenAI GPT-2 โœ… โœ… โœ… โœ… โœ…
OPT โŒ โŒ โœ… โœ… โœ…
OWL-ViT โŒ โŒ โœ… โŒ โŒ
Pegasus โœ… โœ… โœ… โœ… โœ…
PEGASUS-X โŒ โŒ โœ… โŒ โŒ
Perceiver โœ… โŒ โœ… โŒ โŒ
PLBart โœ… โŒ โœ… โŒ โŒ
PoolFormer โŒ โŒ โœ… โŒ โŒ
ProphetNet โœ… โŒ โœ… โŒ โŒ
QDQBert โŒ โŒ โœ… โŒ โŒ
RAG โœ… โŒ โœ… โœ… โŒ
REALM โœ… โœ… โœ… โŒ โŒ
Reformer โœ… โœ… โœ… โŒ โŒ
RegNet โŒ โŒ โœ… โœ… โœ…
RemBERT โœ… โœ… โœ… โœ… โŒ
ResNet โŒ โŒ โœ… โœ… โœ…
RetriBERT โœ… โœ… โœ… โŒ โŒ
RoBERTa โœ… โœ… โœ… โœ… โœ…
RoBERTa-PreLayerNorm โŒ โŒ โœ… โœ… โœ…
RoCBert โœ… โŒ โœ… โŒ โŒ
RoFormer โœ… โœ… โœ… โœ… โœ…
SegFormer โŒ โŒ โœ… โœ… โŒ
SEW โŒ โŒ โœ… โŒ โŒ
SEW-D โŒ โŒ โœ… โŒ โŒ
Speech Encoder decoder โŒ โŒ โœ… โŒ โœ…
Speech2Text โœ… โŒ โœ… โœ… โŒ
Speech2Text2 โœ… โŒ โŒ โŒ โŒ
Splinter โœ… โœ… โœ… โŒ โŒ
SqueezeBERT โœ… โœ… โœ… โŒ โŒ
Swin Transformer โŒ โŒ โœ… โœ… โŒ
Swin Transformer V2 โŒ โŒ โœ… โŒ โŒ
Swin2SR โŒ โŒ โœ… โŒ โŒ
SwitchTransformers โŒ โŒ โœ… โŒ โŒ
T5 โœ… โœ… โœ… โœ… โœ…
Table Transformer โŒ โŒ โœ… โŒ โŒ
TAPAS โœ… โŒ โœ… โœ… โŒ
Time Series Transformer โŒ โŒ โœ… โŒ โŒ
TimeSformer โŒ โŒ โœ… โŒ โŒ
Trajectory Transformer โŒ โŒ โœ… โŒ โŒ
Transformer-XL โœ… โŒ โœ… โœ… โŒ
TrOCR โŒ โŒ โœ… โŒ โŒ
UniSpeech โŒ โŒ โœ… โŒ โŒ
UniSpeechSat โŒ โŒ โœ… โŒ โŒ
UPerNet โŒ โŒ โœ… โŒ โŒ
VAN โŒ โŒ โœ… โŒ โŒ
VideoMAE โŒ โŒ โœ… โŒ โŒ
ViLT โŒ โŒ โœ… โŒ โŒ
Vision Encoder decoder โŒ โŒ โœ… โœ… โœ…
VisionTextDualEncoder โŒ โŒ โœ… โŒ โœ…
VisualBERT โŒ โŒ โœ… โŒ โŒ
ViT โŒ โŒ โœ… โœ… โœ…
ViT Hybrid โŒ โŒ โœ… โŒ โŒ
ViTMAE โŒ โŒ โœ… โœ… โŒ
ViTMSN โŒ โŒ โœ… โŒ โŒ
Wav2Vec2 โœ… โŒ โœ… โœ… โœ…
Wav2Vec2-Conformer โŒ โŒ โœ… โŒ โŒ
WavLM โŒ โŒ โœ… โŒ โŒ
Whisper โœ… โŒ โœ… โœ… โŒ
X-CLIP โŒ โŒ โœ… โŒ โŒ
XGLM โœ… โœ… โœ… โœ… โœ…
XLM โœ… โŒ โœ… โœ… โŒ
XLM-ProphetNet โœ… โŒ โœ… โŒ โŒ
XLM-RoBERTa โœ… โœ… โœ… โœ… โœ…
XLM-RoBERTa-XL โŒ โŒ โœ… โŒ โŒ
XLNet โœ… โœ… โœ… โœ… โŒ
YOLOS โŒ โŒ โœ… โŒ โŒ
YOSO โŒ โŒ โœ… โŒ โŒ
< > Update on GitHub