---
language: ["ru"]
tags:
- russian
- pretraining
- Text classification
license: mit
---

# multilabel-context-russian-inapropriate-messages

[BERT classifier from Skoltech](https://huggingface.co/Skoltech/russian-inappropriate-messages), finetuned on contextual data with 4 labels.

# Training

*Skoltech/russian-inappropriate-messages* was finetuned on a multiclass data with four classes 

1) OK label -- the message is OK in context and does not intent to offend or somehow harm the reputation of a speaker.
2) Toxic label -- the message might be seen as a offensive one in given context.
3) Severe toxic label -- the message is offencive, full of anger and was written to provoke a fight or any other discomfort
4) Risks label -- the message touches on sensitive topics and can harm the reputation of the speaker (i.e. religion, politics) 

The model was finetuned on DATASET_LINK. 

# Evaluation results

Model achieves the following results:

|                         | OK - Precision | OK - Recall | OK - F1-score | TOXIC - Precision | TOXIC - Recall | TOXIC - F1-score | SEVERE TOXIC - Precision | SEVERE TOXIC - Recall | SEVERE TOXIC - F1-score | RISKS - Precision | RISKS - Recall | RISKS - F1-score |
|-------------------------|----------------|-------------|---------------|-------------------|----------------|------------------|--------------------------|-----------------------|-------------------------|-------------------|----------------|------------------|
| DATASET_TWITTER val.csv | 0.883          | 0.913       | 0.896         | 0.368             | 0.330          | 0.348            | 0.515                    | 0.468                 | 0.490                   | 0.659             | 0.535          | 0.591            |
| DATASET_GENA val.csv    | 0.953          | 0.927       | 0.940         | 0.260             | 0.343          | 0.295            | 0.666                    | 0.806                 | 0.729                   | 0.523             | 0.423          | 0.46             |

The work was done during internship at Tinkoff.