README.md

metadata

language:
  - ru
tags:
  - russian
  - pretraining
  - Text classification
license: mit

multilabel-context-russian-inapropriate-messages

BERT classifier from Skoltech, finetuned on contextual data with 4 labels.

Training

Skoltech/russian-inappropriate-messages was finetuned on a multiclass data with four classes

1. OK label -- the message is OK in context and does not intent to offend or somehow harm the reputation of a speaker.
2. Toxic label -- the message might be seen as a offensive one in given context.
3. Severe toxic label -- the message is offencive, full of anger and was written to provoke a fight or any other discomfort
4. Risks label -- the message touches on sensitive topics and can harm the reputation of the speaker (i.e. religion, politics)

The model was finetuned on DATASET_LINK.

Evaluation results

Model achieves the following results:

	OK - Precision	OK - Recall	OK - F1-score	TOXIC - Precision	TOXIC - Recall	TOXIC - F1-score	SEVERE TOXIC - Precision	SEVERE TOXIC - Recall	SEVERE TOXIC - F1-score	RISKS - Precision	RISKS - Recall	RISKS - F1-score
DATASET_TWITTER val.csv	0.883	0.913	0.896	0.368	0.330	0.348	0.515	0.468	0.490	0.659	0.535	0.591
DATASET_GENA val.csv	0.953	0.927	0.940	0.260	0.343	0.295	0.666	0.806	0.729	0.523	0.423	0.46

The work was done during internship at Tinkoff.