transformer / transformeur
Also known as: self-attention model / modèle auto-attentif§§modèle d'auto-attention
Definition: "sequence transduction model based entirely on attention, replacing the recurrent layers most commonly used in encoder-decoder architectures with multi-headed self-attention." (Vaswani et al., 2017, p. 10).
Définition : « modèle de transduction de séquences entièrement basé sur l'attention, remplaçant les couches récurrentes les plus couramment utilisées dans les architectures encodeur-décodeur par l'auto-attention multi-têtes. » (Vaswani et al., 2017, p. 10).
Broader concept: Neural networks models / modèles de réseaux de neurones
Base of: ALBERT§§large language model / ALBERT§§grand modèle de langue
Implemented by: ALeaseBERT§§ARBERT§§AbLang§§AlBERTo§§AlephBERT§§AraBERT§§AraGPT2§§astroBERT§§AstroLLaMA§§BART§§BARThez§§BEHRT§§BERTJapanese§§BETO§§BERT-PLI§§BERT-SentiX§§BERTimbau§§BERTje§§BERTopic§§BERTurk§§BERTweet§§BanglaBERT§§BioALBERT§§BioBERT§§BioMedBERT§§BioMed-RoBERTa§§BLOOM§§BlueBERT§§CANINE§§COVID-Twitter-BERT§§CamemBERT§§CancerBERT§§CharBERT§§ChemBERTa§§ClinicalBERT§§CoText§§CodeBERT§§CodeGPT§§Czech-B§§Czert-A§§DeBERTa§§DeLFT§§DrBERT§§E5§§ERNIE§§ELECTRA§§Finnish BERT§§Flair§§FlauBERT§§GPT§§GPT-2§§GPT-3§§Galactica§§GenSLMs§§Genomic ULMFit§§German BERT§§GluonNLP§§GottBERT§§GreekBERT§§GreenBioBERT§§HateBERT§§HeBERT§§HerBERT§§IndicBERT§§IndoBERT§§KB-BERT§§KLUE-BERT§§KLUE-RoBERTa§§KM-BERT§§KR-BERT§§KeBioLM§§KeyBERT§§KoBERT§§KoGPT2§§KoreALBERT§§LaMDA§§Legal-BERT§§Legal-HeBERT§§Lemur§§Llama 2§§LongFormer§§MARBERT§§MASS§§M-BERT§§MC-BERT§§MathBERT§§MiniLM§§MobileBERT§§Mol-BERT§§MuRIL§§NetBERT§§OAG-BERT§§OPT§§OuBioBERT§§PEGAUSUS§§PLBART§§PTT5§§PaLM§§ParsBERT§§PharmBERT§§PhoBERT§§PolishBERT§§PolyNC§§ProkBERT§§ProteinBERT§§PubMedBERT§§RBERT§§RadBERT§§RoBERTa§§RobBERT§§RobeCzech§§RoBERTuito§§Romanian BERT§§ruBERT§§SBERT§§SapBERT§§SciFive§§Slavic BERT§§Spark NLP§§Spark NLP Python§§Spark NLP Scala§§StructBERT§§T5§§TOD-BERT§§TextAttack§§TinyBERT§§Tohoku-BERT§§TransPolymer§§UTH-BERT§§UmBERTo§§UmlsBERT§§UniLM§§Unicoder§§WangchanBERTa§§XLM-T§§XLNet§§XLM-RoBERTa§§araELECTRA§§byT5§§convBERT§§convBERTurk§§DistilBERT§§entityBERT§§estBERT§§golgotha§§mT5§§medBERT§§nanoBERT§§polyBERT§§sciBERT§§spanBERT§§srBERT§§text§§transformers / ALeaseBERT§§ARBERT§§AbLang§§AlBERTo§§AlephBERT§§AraBERT§§AraGPT2§§astroBERT§§AstroLLaMA§§BART§§BARThez§§BEHRT§§BERTJapanese§§BETO§§BERT-PLI§§BERT-SentiX§§BERTimbau§§BERTje§§BERTopic§§BERTurk§§BERTweet§§BanglaBERT§§BioALBERT§§BioBERT§§BioMedBERT§§BioMed-RoBERTa§§BLOOM§§BlueBERT§§CANINE§§COVID-Twitter-BERT§§CamemBERT§§CancerBERT§§CharBERT§§ChemBERTa§§ClinicalBERT§§CoText§§CodeBERT§§CodeGPT§§Czech-B§§Czert-A§§DeBERTa§§DeLFT§§DrBERT§§E5§§ERNIE§§ELECTRA§§Finnish BERT§§Flair§§FlauBERT§§GPT§§GPT-2§§GPT-3§§Galactica§§GenSLMS§§Genomic ULMFit§§German BERT§§GluonNLP§§GottBERT§§GreekBERT§§GreenBioBERT§§HateBERT§§HeBERT§§HerBERT§§IndicBERT§§IndoBERT§§KB-BERT§§KLUE-BERT§§KLUE-RoBERTa§§KM-BERT§§KR-BERT§§KeBioLM§§KeyBERT§§KoBERT§§KoGPT2§§KoreALBERT§§LaMDA§§Legal-BERT§§Legal-HeBERT§§Lemur§§Llama 2§§LongFormer§§MARBERT§§MASS§§M-BERT§§MC-BERT§§MathBERT§§MiniLM§§MobileBERT§§Mol-BERT§§MuRIL§§NetBERT§§OAG-BERT§§OPT§§OuBioBERT§§PEGAUSUS§§PLBART§§PTT5§§PaLM§§ParsBERT§§PharmBERT§§PhoBERT§§PolishBERT§§PolyNC§§ProkBERT§§ProteinBERT§§PubMedBERT§§RBERT§§RadBERT§§RoBERTa§§RobBERT§§RobeCzech§§RoBERTuito§§Romanian BERT§§ruBERT§§SBERT§§SapBERT§§SciFive§§Slavic BERT§§Spark NLP§§Spark NLP Python§§Spark NLP Scala§§StructBERT§§T5§§TOD-BERT§§TextAttack§§TinyBERT§§Tohoku-BERT§§TransPolymer§§UTH-BERT§§UmBERTo§§UmlsBERT§§UniLM§§Unicoder§§WangchanBERTa§§XLM-T§§XLNet§§XLM-RoBERTa§§araELECTRA§§byT5§§convBERT§§conBERTurk§§DistilBERT§§entityBERT§§estBERT§§golgotha§§mT5§§medBERT§§nanoBERT§§polyBERT§§sciBERT§§spanBERT§§srBERT§§text§§transformers
Reference: Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. arXiv:1706.03762 [cs]. http://arxiv.org/abs/1706.03762