Huertas97 commited on
Commit
5b95be3
1 Parent(s): 5329fcf

README avg results

Browse files
Files changed (1) hide show
  1. README.md +13 -1
README.md CHANGED
@@ -79,7 +79,19 @@ print(sentence_embeddings)
79
  ## Evaluation Results
80
 
81
  <!--- Describe how your model was evaluated -->
82
- Check the test results in the Semantic Textual Similarity Tasks. The 15 languages available at the [Multilingual STSB](https://github.com/Huertas97/Multilingual-STSB) have been combined into monolingual and cross-lingual tasks, giving a total of 31 tasks. Monolingual tasks have both sentences from the same language source (e.g., Ar-Ar, Es-Es), while cross-lingual tasks have two sentences, each in a different language being one of them English (e.g., en-ar, en-es). For the sake of readability tasks have been splitted into monolingual and cross-lingual tasks.
 
 
 
 
 
 
 
 
 
 
 
 
83
 
84
  | Monolingual Task | Pearson Cosine test | Spearman Cosine test |
85
  |------------------|---------------------|-----------------------|
 
79
  ## Evaluation Results
80
 
81
  <!--- Describe how your model was evaluated -->
82
+ Check the test results in the Semantic Textual Similarity Tasks. The 15 languages available at the [Multilingual STSB](https://github.com/Huertas97/Multilingual-STSB) have been combined into monolingual and cross-lingual tasks, giving a total of 31 tasks. Monolingual tasks have both sentences from the same language source (e.g., Ar-Ar, Es-Es), while cross-lingual tasks have two sentences, each in a different language being one of them English (e.g., en-ar, en-es).
83
+
84
+
85
+ Here we compare the average multilingual semantic textual similairty capabilities between the `paraphrase-multilingual-mpnet-base-v2` based model and the `mstsb-paraphrase-multilingual-mpnet-base-v2` fine-tuned model across the 31 tasks. It is worth noting that both models are multilingual, but the second model is adjusted with multilingual data for semantic similarity. The average of correlation coefficients is computed by transforming each correlation coefficient to a Fisher's z value, averaging them, and then back-transforming to a correlation coefficient.
86
+
87
+
88
+ | Model | Average Spearman Cosine Test |
89
+ |---------------------------------------------|------------------------------|
90
+ | mstsb-paraphrase-multilingual-mpnet-base-v2 | 0.835890 |
91
+ | paraphrase-multilingual-mpnet-base-v2 | 0.818896 |
92
+
93
+
94
+ For the sake of readability tasks have been splitted into monolingual and cross-lingual tasks.
95
 
96
  | Monolingual Task | Pearson Cosine test | Spearman Cosine test |
97
  |------------------|---------------------|-----------------------|