| EconBERTa - RoBERTa further trained on 4GB of uncompressed text sourced from economics books. | |
| Example usage for MLM: | |
| ```python | |
| from transformers import RobertaTokenizer, RobertaForMaskedLM | |
| from transformers import pipeline | |
| tokenizer = RobertaTokenizer.from_pretrained('roberta-base') | |
| model = RobertaForMaskedLM.from_pretrained('models').cpu() | |
| model.eval() | |
| mlm = pipeline('fill-mask', model = model, tokenizer = tokenizer) | |
| test = "ECB - euro, FED - <mask>, BoJ - yen" | |
| print(mlm(test)[:2]) | |
| [{'sequence': 'ECB - euro, FED - dollar, BoJ - yen', | |
| 'score': 0.7342271208763123, | |
| 'token': 1404, | |
| 'token_str': ' dollar'}, | |
| {'sequence': 'ECB - euro, FED - dollars, BoJ - yen', | |
| 'score': 0.10828445851802826, | |
| 'token': 1932, | |
| 'token_str': ' dollars'}] | |
| ``` | |