How to use this model directly from the
from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("chiragjn/small_bert_uncased_L-12_H-768_A-12") model = AutoModel.from_pretrained("chiragjn/small_bert_uncased_L-12_H-768_A-12")
This model was converted from the release of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) referenced in Well-Read Students Learn Better: On the Importance of Pre-training Compact Models.
Conversion was performed automatically using
transformers-cli convert as explained here
This model is also available on tfhub at
Caution: The tfhub version does not generate the same output as the released checkpoint or the converted model.
Released tf checkpoint and this converted model generate similar embeddings.