The BERT base, uncased model for Romanian, trained on a 15GB corpus, version
from transformers import AutoTokenizer, AutoModel import torch # load tokenizer and model tokenizer = AutoTokenizer.from_pretrained("dumitrescustefan/bert-base-romanian-uncased-v1", do_lower_case=True) model = AutoModel.from_pretrained("dumitrescustefan/bert-base-romanian-uncased-v1") # tokenize a sentence and run through the model input_ids = torch.tensor(tokenizer.encode("Acesta este un test.", add_special_tokens=True)).unsqueeze(0) # Batch size 1 outputs = model(input_ids) # get encoding last_hidden_states = outputs # The last hidden-state is the first element of the output tuple
Remember to always sanitize your text! Replace
t cedilla-letters to comma-letters with :
text = text.replace("ţ", "ț").replace("ş", "ș").replace("Ţ", "Ț").replace("Ş", "Ș")
because the model was NOT trained on cedilla
ts. If you don't, you will have decreased performance due to s and increased number of tokens per word.
Evaluation is performed on Universal Dependencies Romanian RRT UPOS, XPOS and LAS, and on a NER task based on RONEC. Details, as well as more in-depth tests not shown here, are given in the dedicated evaluation page.
The baseline is the Multilingual BERT model
bert-base-multilingual-(un)cased, as at the time of writing it was the only available BERT model that works on Romanian.
The model is trained on the following corpora (stats in the table below are after cleaning):
- We'd like to thank Sampo Pyysalo from TurkuNLP for helping us out with the compute needed to pretrain the v1.0 BERT models. He's awesome!
- Downloads last month
Unable to determine this model’s pipeline type. Check the docs .