README.md

metadata

language:
  - ru
tags:
  - russian
  - classification
  - toxicity
  - multilabel
widget:
  - text: Иди ты нафиг!

This is the cointegrated/rubert-tiny model fine-tuned for classification of toxicity and inappropriateness for short informal Russian texts, such as comments in social networks.

The problem is formulated as multilabel classification with the following classes:

• non-toxic: the text does NOT contain insults, obscenities, and threats, in the sense of the OK ML Cup competition.
• insult
• obscenity
• threat
• dangerous: the text is inappropriate, in the sense of Babakov et.al., i.e. it can harm the reputation of the speaker.

A text can be considered safe if it is BOTH non-toxic and NOT dangerous.

Usage

The function below estimates the probability that the text is either toxic OR dangerous:

# !pip install transformers sentencepiece --quiet
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_checkpoint = 'cointegrated/rubert-tiny-toxicity'
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint)
if torch.cuda.is_available():
    model.cuda()
    
def text2toxicity(text, aggregate=True):
    """ Calculate toxicity of a text (if aggregate=True) or a vector of toxicity aspects (if aggregate=False)"""
    with torch.no_grad():
        inputs = tokenizer(text, return_tensors='pt', truncation=True, padding=True).to(model.device)
        proba = torch.sigmoid(model(**inputs).logits).cpu().numpy()
    if isinstance(text, str):
        proba = proba[0]
    if aggregate:
        return 1 - proba.T[0] * (1 - proba.T[-1])
    return proba

print(text2toxicity('я люблю нигеров', True))
# 0.57240640889815

print(text2toxicity('я люблю нигеров', False))
# [9.9336821e-01 6.1555761e-03 1.2781911e-03 9.2758919e-04 5.6955177e-01]

print(text2toxicity(['я люблю нигеров', 'я люблю африканцев'], True))
# [0.5724064  0.20111847]

print(text2toxicity(['я люблю нигеров', 'я люблю африканцев'], False))
# [[9.9336821e-01 6.1555761e-03 1.2781911e-03 9.2758919e-04 5.6955177e-01]
#  [9.9828428e-01 1.1138428e-03 1.1492912e-03 4.6551935e-04 1.9974548e-01]]

Training

The model has been trained on the joint dataset of OK ML Cup and Babakov et.al. with Adam optimizer, learning rate of 1e-5, and batch size of 128 for 5 epochs. The data was not filtered in any way. A text was considered inappropriate if its inappropritateness score was higher than 0.2. The per-label ROC AUC on the dev set is:

non-toxic  : 0.9909
insult     : 0.9882
obscenity  : 0.9824
threat     : 0.9868
dangerous  : 0.7758