Hugging Face's logo

tinkoff-ai
/
response-quality-classifier-tiny

Text Classification
Transformers
PyTorch
Russian
bert
conversational
Inference Endpoints
Model card Files Community
2
egoriya commited on
Commit
a8f8dcb
•
1 Parent(s): 799292a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -6
README.md CHANGED
@@ -13,8 +13,8 @@ This classification model is based on [cointegrated/rubert-tiny2](https://huggin
13
  The model should be used to produce relevance and specificity of the last message in the context of a dialogue.
14
 
15
  The labels explanation:
16
- - `relevance`: is the last message in the dialogue relevant in the context of the full dialogue
17
- - `specificity`: is the last message in the dialogue interesting and promotes the continuation of the dialogue
18
 
19
  It is pretrained on a large corpus of dialog data in unsupervised manner: the model is trained to predict whether last response was in a real dialog, or it was pulled from some other dialog at random.
20
 
@@ -22,8 +22,7 @@ Then it was finetuned on manually labelled examples (dataset will be posted soon
22
 
23
  The model was trained with the dialogue length 4 where the last message is needed to be estimated. Each message in the dialogue was tokenized separately with ``` max_length = max_seq_length // 4 ```.
24
 
25
- It is pretrained on corpus of dialog data and finetuned on [tinkoff-ai/context_similarity](https://huggingface.co/tinkoff-ai/context_similarity).
26
- The performance of the model on validation split (dataset will be posted soon)[tinkoff-ai/context_similarity](https://huggingface.co/tinkoff-ai/context_similarity) (with the best thresholds for validation samples):
27
 
28
 
29
  | | threshold | f0.5 | ROC AUC |
@@ -48,6 +47,6 @@ with torch.inference_mode():
48
  relevance, specificity = probas
49
  ```
50
 
51
- The [app](https://huggingface.co/spaces/tinkoff-ai/response-quality-classifiers) where you can easily evaluate this model
52
 
53
- The work was done during internship at Tinkoff by [egoriyaa](https://github.com/egoriyaa)
 
13
  The model should be used to produce relevance and specificity of the last message in the context of a dialogue.
14
 
15
  The labels explanation:
16
+ - `relevance`: is the last message in the dialogue relevant in the context of the full dialogue.
17
+ - `specificity`: is the last message in the dialogue interesting and promotes the continuation of the dialogue.
18
 
19
  It is pretrained on a large corpus of dialog data in unsupervised manner: the model is trained to predict whether last response was in a real dialog, or it was pulled from some other dialog at random.
20
 
 
22
 
23
  The model was trained with the dialogue length 4 where the last message is needed to be estimated. Each message in the dialogue was tokenized separately with ``` max_length = max_seq_length // 4 ```.
24
 
25
+ The performance of the model on validation split (dataset will be posted soon) (with the best thresholds for validation samples):
 
26
 
27
 
28
  | | threshold | f0.5 | ROC AUC |
 
47
  relevance, specificity = probas
48
  ```
49
 
50
+ The [app](https://huggingface.co/spaces/tinkoff-ai/response-quality-classifiers) where you can easily evaluate this model.
51
 
52
+ The work was done during internship at Tinkoff by [egoriyaa](https://github.com/egoriyaa).