|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
- fr |
|
- es |
|
- multilingual |
|
widget: |
|
- text: "Critical levels of out of school children were reported, with 72% of respondents pointing to moderate to high numbers of primary school age not accessing <mask>" |
|
--- |
|
|
|
# HumBert |
|
|
|
HumBert (Humanitarian Bert) is a [XLM-Roberta](https://huggingface.co/xlm-roberta-base) model trained on humanitarian texts - approximately 50 million textual examples (roughly 2 billion tokens) from public humanitarian reports, law cases and news articles. |
|
Data were collected from three main sources: [Reliefweb](https://reliefweb.int/), [UNHCR Refworld](https://www.refworld.org/) and [Europe Media Monitor News Brief](https://emm.newsbrief.eu/). |
|
Although XLM-Roberta was trained on 100 different languages, this fine-tuning was performed on three languages, English, French and Spanish, due to the impossibility of finding a good amount of such kind of humanitarian data in other languages. |
|
|
|
|
|
## Intended uses |
|
|
|
To the best of our knowledge, HumBert is the first language model adapted on humanitarian topics, which often use a very specific language, making adaptation to downstream tasks (such as dister responses text classification) more effective. |
|
This model is primarily aimed at being fine-tuned on tasks such as sequence classification or token classification. |
|
|
|
## Benchmarks |
|
|
|
Soon... |
|
|
|
## Usage |
|
|
|
Here is how to use this model to get the features of a given text in PyTorch: |
|
|
|
```python |
|
from transformers import AutoTokenizer, AutoModelForMaskedLM |
|
tokenizer = AutoTokenizer.from_pretrained('nlp-thedeep/humbert') |
|
model = AutoModelForMaskedLM.from_pretrained("nlp-thedeep/humbert") |
|
# prepare input |
|
text = "YOUR TEXT" |
|
encoded_input = tokenizer(text, return_tensors='pt') |
|
# forward pass |
|
output = model(**encoded_input) |
|
``` |
