Transformers
Back to all models
Model: clue/roberta_chinese_3L312_clue_tiny

Monthly model downloads

clue/roberta_chinese_3L312_clue_tiny clue/roberta_chinese_3L312_clue_tiny
- downloads
last 30 days

pytorch

tf

Contributed by

CLUE benchmark non-profit
1 team member · 11 models

How to use this model directly from the 🤗/transformers library:

			
Copy model
tokenizer = AutoTokenizer.from_pretrained("clue/roberta_chinese_3L312_clue_tiny") model = AutoModel.from_pretrained("clue/roberta_chinese_3L312_clue_tiny")

Introduction

This model was trained on TPU and the details are as follows:

Model

Model_name params size Training_corpus Vocab
RoBERTa-tiny-clue
Super_small_model
7.5M 28.3M CLUECorpus2020 CLUEVocab
RoBERTa-tiny-pair
Super_small_sentence_pair_model
7.5M 28.3M CLUECorpus2020 CLUEVocab
RoBERTa-tiny3L768-clue
small_model
38M 110M CLUECorpus2020 CLUEVocab
RoBERTa-tiny3L312-clue
small_model
<7.5M 24M CLUECorpus2020 CLUEVocab
RoBERTa-large-clue
Large_model
290M 1.20G CLUECorpus2020 CLUEVocab
RoBERTa-large-pair
Large_sentence_pair_model
290M 1.20G CLUECorpus2020 CLUEVocab

Usage

With the help ofHuggingface-Transformers 2.5.1, you could use these model as follows

tokenizer = BertTokenizer.from_pretrained("MODEL_NAME")
model = BertModel.from_pretrained("MODEL_NAME")

MODEL_NAME

Model_NAME MODEL_LINK
RoBERTa-tiny-clue clue/roberta_chinese_clue_tiny
RoBERTa-tiny-pair clue/roberta_chinese_pair_tiny
RoBERTa-tiny3L768-clue clue/roberta_chinese_3L768_clue_tiny
RoBERTa-tiny3L312-clue clue/roberta_chinese_3L312_clue_tiny
RoBERTa-large-clue clue/roberta_chinese_clue_large
RoBERTa-large-pair clue/roberta_chinese_pair_large

Details

Please read https://arxiv.org/pdf/2003.01355.

Please visit our repository: https://github.com/CLUEbenchmark/CLUEPretrainedModels.git