gyeoldere commited on
Commit
0527fb8
1 Parent(s): 1c325c7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -2
README.md CHANGED
@@ -7,6 +7,9 @@ datasets:
7
  model-index:
8
  - name: DeBERTa-finetuned-SNLI2
9
  results: []
 
 
 
10
  ---
11
 
12
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -16,9 +19,23 @@ should probably proofread and complete it, then remove this comment. -->
16
 
17
  This model is a fine-tuned version of [gyeoldere/test_trainer](https://huggingface.co/gyeoldere/test_trainer) on the snli dataset.
18
 
 
 
 
 
 
 
 
19
  ## Model description
20
 
21
- More information needed
 
 
 
 
 
 
 
22
 
23
  ## Intended uses & limitations
24
 
@@ -50,4 +67,4 @@ The following hyperparameters were used during training:
50
  - Transformers 4.26.0
51
  - Pytorch 1.13.1+cu116
52
  - Datasets 2.9.0
53
- - Tokenizers 0.13.2
 
7
  model-index:
8
  - name: DeBERTa-finetuned-SNLI2
9
  results: []
10
+ metrics:
11
+ - accuracy
12
+ library_name: transformers
13
  ---
14
 
15
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
19
 
20
  This model is a fine-tuned version of [gyeoldere/test_trainer](https://huggingface.co/gyeoldere/test_trainer) on the snli dataset.
21
 
22
+ Test_trainer model is a fine-tuned version of [microsoft/deberta-base](https://huggingface.co/microsoft/deberta-base) on the snli dataset.
23
+
24
+ This model achieves the following results on the evaluation set:
25
+ - NLI accuracy : 0.86
26
+ - MLM accuracy : 0.68
27
+
28
+
29
  ## Model description
30
 
31
+ This model fine-tuned to perform 2 tasks simultaneously; NLI task and MLM task.
32
+
33
+ Output vector of DeBERTa processed through two different fc layer to predict.
34
+ I used layer structure introduced in BERT paper, which is implemented on huggingface transformers; DebertaForTokenClassification and DebertaForMaskedLM.
35
+ [https://huggingface.co/docs/transformers/index]
36
+
37
+ BinaryCrossEntrophyLoss are used for each class, and two losses are added to obtain final loss
38
+ final_loss = MLM_loss + NLI_loss
39
 
40
  ## Intended uses & limitations
41
 
 
67
  - Transformers 4.26.0
68
  - Pytorch 1.13.1+cu116
69
  - Datasets 2.9.0
70
+ - Tokenizers 0.13.2