morit commited on
Commit
e10db45
1 Parent(s): 96c92ae

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -7
README.md CHANGED
@@ -49,17 +49,19 @@ classifier(sequence_to_classify, candidate_labels, hypothesis_template=hypothesi
49
 
50
 
51
  ## Training
52
- This model was pre-trained on a set of 100 languages and follwed further training on 198M multilingual tweets as described in the original [paper](https://arxiv.org/abs/2104.12250). Further it was trained on the training set of XNLI dataset in spanish which is a machine translated version of the MNLI dataset. It was trained on 3 epochs and the following specifications
53
- - learning rate: 5e-5
 
 
54
  - batch size: 32
55
  - max sequence: length 128
56
 
57
- on one GPU (NVIDIA GeForce RTX 3090) resulting in a training time of 1h 47 mins.
58
 
59
-
60
  ## Evaluation
61
- The model was evaluated after each epoch on the eval set of the XNLI Corpus and at the end of training on the Test set of the XNLI corpus.
62
- Using the test set the model reached an accuracy of
63
  ```
64
- predict_accuracy = 78.82 %
65
  ```
 
 
49
 
50
 
51
  ## Training
52
+ This model was pre-trained on a set of 100 languages and follwed further training on 198M multilingual tweets as described in the original [paper](https://arxiv.org/abs/2104.12250). Further it was trained on the training set of XNLI dataset in spanish which is a machine translated version of the MNLI dataset. It was trained on 5 epochs of the XNLI train set and evaluated on the XNLI eval dataset at the end of every to find the best performing model. The model which had the highest accuracy on the eval set was chosen at the end.
53
+
54
+ ![Training Charts from wandb](screen_wandb.png)
55
+ - learning rate: 2e-5
56
  - batch size: 32
57
  - max sequence: length 128
58
 
59
+ using a GPU (NVIDIA GeForce RTX 3090) resulting in a training time of 1h 47 mins.
60
 
 
61
  ## Evaluation
62
+
63
+ The best performing model was evaluatated on the XNLI test set to get a comparable result
64
  ```
65
+ predict_accuracy = 79.2 %
66
  ```
67
+