Add info about model used for embedding examples for t-SNE plot.
Browse files
README.md
CHANGED
@@ -134,7 +134,7 @@ for config in ("random", "stepwise", "gaussian"):
|
|
134 |
<caption>Figure 6. Experimental perplexity distribution of the sampled mc4-es after applying Random sampling.</caption>
|
135 |
</figure>
|
136 |
|
137 |
-
Although this is not a comprehensive analysis, we looked into the distribution of perplexity for the training corpus. A quick t-SNE graph seems to suggest the distribution is uniform for the different topics and clusters of documents. The interactive plot (**perplexity_colored_embeddings.html**) is available in the **images** folder. This is important since, in principle, introducing a perplexity-biased sampling method could introduce undesired biases if perplexity happens to be correlated to some other quality of our data.
|
138 |
|
139 |
|
140 |
### Training details
|
|
|
134 |
<caption>Figure 6. Experimental perplexity distribution of the sampled mc4-es after applying Random sampling.</caption>
|
135 |
</figure>
|
136 |
|
137 |
+
Although this is not a comprehensive analysis, we looked into the distribution of perplexity for the training corpus. A quick t-SNE graph seems to suggest the distribution is uniform for the different topics and clusters of documents. The interactive plot (**perplexity_colored_embeddings.html**) is available in the **images** folder. This graph was generated used a distilled version of [mUSE](https://huggingface.co/sentence-transformers/distiluse-base-multilingual-cased-v1) to embed a random subset of 20,000 examples. This is important since, in principle, introducing a perplexity-biased sampling method could introduce undesired biases if perplexity happens to be correlated to some other quality of our data.
|
138 |
|
139 |
|
140 |
### Training details
|