ibraheemmoosa commited on
Commit
b62f5a3
1 Parent(s): f1e9838

Fix evaluation result values an add results of the ablation model.

Browse files
Files changed (1) hide show
  1. README.md +13 -13
README.md CHANGED
@@ -130,19 +130,19 @@ The model was pretrained on TPUv3-8 for 1M steps. We have checkpoints available
130
  We evaluated this model on the Indo-Aryan subset of languages (Panjabi, Oriya, Assamese, Bangla, Hindi, Marathi, Gujarati) from the [IndicGLUE](https://huggingface.co/datasets/indic_glue) benchmark dataset. We report the mean and standard deviation of nine fine-tuning runs for this model.
131
 
132
  ### IndicGLUE
133
- Task | mBERT | XLM-R | IndicBERT-Base | XLMIndic-Base-Uniscript (This Model)
134
- -----| ----- | ----- | ------ | -------
135
- Wikipedia Section Title Prediction| 71.90 | 65.45 | 69.40 | **81.78 ± 0.60**
136
- Article Genre Classification | 88.64 | 96.61 | 97.72 | **98.70 ± 0.29**
137
- Named Entity Recognition (F1-score) | 71.29 | 62.18 | 56.69 | **89.85 ± 1.14**
138
- BBC Hindi News Article Classification | 60.55 | 75.52 | 74.60 | **77.28 ± 1.50**
139
- Soham Bangla News Article Classification | 80.23 | 87.6 | 78.45 | **93.22 ± 0.49**
140
- INLTK Gujarati Headlines | - | - | **92.91** | 90.41 ± 0.69
141
- INLTK Marathi Headlines | - | - | **94.30** | 92.21 ± 0.23
142
- IITP Hindi Product Reviews Sentiment Classification | 74.57 | **78.97** | 71.32 | 76.33 ± 0.84
143
- IITP Hindi Movie Reviews Sentiment Classification | 56.77 | 61.61 | 59.03 | **65.91 ± 2.20**
144
- MIDAS Hindi Discourse Classification | 71.20 | **79.94** | 78.44 | 78.39 ± 0.33
145
- Cloze Style Question Answering | - | - | 37.16 | **41.54**
146
 
147
  ## Intended uses & limitations
148
 
 
130
  We evaluated this model on the Indo-Aryan subset of languages (Panjabi, Oriya, Assamese, Bangla, Hindi, Marathi, Gujarati) from the [IndicGLUE](https://huggingface.co/datasets/indic_glue) benchmark dataset. We report the mean and standard deviation of nine fine-tuning runs for this model.
131
 
132
  ### IndicGLUE
133
+ Task | mBERT | XLM-R | IndicBERT-Base | XLMIndic-Base-Uniscript (This Model) | XLMIndic-Base-Multiscript (Ablation Model)
134
+ -----| ----- | ----- | ------ | ------- | --------
135
+ Wikipedia Section Title Prediction| 71.90 | 65.45 | 69.40 | **81.78 ± 0.60** | 77.17 ± 0.76
136
+ Article Genre Classification | 88.64 | 96.61 | 97.72 | **98.70 ± 0.29** | 98.30 ± 0.26
137
+ Named Entity Recognition (F1-score) | 71.29 | 62.18 | 56.69 | **89.85 ± 1.14** | 83.19 ± 1.58
138
+ BBC Hindi News Article Classification | 60.55 | 75.52 | 74.60 | **79.14 ± 0.60** | 77.28 ± 1.50
139
+ Soham Bangla News Article Classification | 80.23 | 87.6 | 78.45 | **93.89 ± 0.48** | 93.22 ± 0.49
140
+ INLTK Gujarati Headlines | - | - | **92.91** | 90.73 ± 0.75 | 90.41 ± 0.69
141
+ INLTK Marathi Headlines | - | - | **94.30** | 92.04 ± 0.47 | 92.21 ± 0.23
142
+ IITP Hindi Product Reviews Sentiment Classification | 74.57 | **78.97** | 71.32 | 77.18 ± 0.77 | 76.33 ± 0.84
143
+ IITP Hindi Movie Reviews Sentiment Classification | 56.77 | 61.61 | 59.03 | **66.34 ± 0.16** | 65.91 ± 2.20
144
+ MIDAS Hindi Discourse Classification | 71.20 | **79.94** | 78.44 | 78.54 ± 0.91 | 78.39 ± 0.33
145
+ Cloze Style Question Answering | - | - | 37.16 | **41.54** | 38.21
146
 
147
  ## Intended uses & limitations
148