wyluilipe/wb-themes-classification

test_trainer

This model is a fine-tuned version of cointegrated/rubert-tiny on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7461
• Accuracy: 0.8310

How to use:

# themes = ['баги', 'открытие', 'баланс', 'рейтинг', 'ревизия', 'другое']

from transformers import AutoTokenizer, AutoModel
import torch
model_name = 'wyluilipe/wb-themes-classification'

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = BertForSequenceClassification.from_pretrained(model_name, num_labels=i+1)

text = "программа не работает"
encoded_input = tokenizer(text, return_tensors='pt')

with torch.no_grad():
   output = model(**encoded_input)
   probabilities = torch.nn.functional.softmax(output.logits, dim=-1)
   predicted_class = torch.argmax(probabilities).item()

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	60	0.7383	0.8404
No log	2.0	120	0.8743	0.7840
No log	3.0	180	0.7312	0.8169
No log	4.0	240	0.6733	0.8404
No log	5.0	300	0.7612	0.7981
No log	6.0	360	0.7671	0.8122
No log	7.0	420	0.7306	0.8263
No log	8.0	480	0.7523	0.8263
0.1118	9.0	540	0.7645	0.8263
0.1118	10.0	600	0.7461	0.8310

Framework versions

• Transformers 4.37.1
• Pytorch 2.1.2
• Datasets 2.16.1
• Tokenizers 0.15.1