d.tsimerman
add tags
af6db24
|
raw
history blame
2.06 kB
metadata
language:
  - ru
tags:
  - russian
  - pretraining
  - Text classification
license: mit

multilabel-context-russian-inapropriate-messages

BERT classifier from Skoltech, finetuned on contextual data with 4 labels.

Training

Skoltech/russian-inappropriate-messages was finetuned on a multiclass data with four classes

  1. OK label -- the message is OK in context and does not intent to offend or somehow harm the reputation of a speaker.
  2. Toxic label -- the message might be seen as a offensive one in given context.
  3. Severe toxic label -- the message is offencive, full of anger and was written to provoke a fight or any other discomfort
  4. Risks label -- the message touches on sensitive topics and can harm the reputation of the speaker (i.e. religion, politics)

The model was finetuned on DATASET_LINK.

Evaluation results

Model achieves the following results:

OK - Precision OK - Recall OK - F1-score TOXIC - Precision TOXIC - Recall TOXIC - F1-score SEVERE TOXIC - Precision SEVERE TOXIC - Recall SEVERE TOXIC - F1-score RISKS - Precision RISKS - Recall RISKS - F1-score
DATASET_TWITTER val.csv 0.883 0.913 0.896 0.368 0.330 0.348 0.515 0.468 0.490 0.659 0.535 0.591
DATASET_GENA val.csv 0.953 0.927 0.940 0.260 0.343 0.295 0.666 0.806 0.729 0.523 0.423 0.46

The work was done during internship at Tinkoff.