sdadas commited on
Commit
638ecc8
1 Parent(s): c7c2948

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -48,6 +48,8 @@ Below is a summary of the Qra-7B model:
48
 
49
  In this section we compare the perplexity of Qra models on Polish texts with other Polish and English LLMs.
50
 
 
 
51
  ### PolEval-2018
52
 
53
  In 2018, the PolEval competition included a language modeling task, for which training and test sets totaling over 20 million Polish sentences were made available. We used the first 10k sentences from the test set to evaluate modern neural language models. To calculate the perplexity, we used a script from the [HuggingFace Evaluate](https://huggingface.co/spaces/evaluate-metric/perplexity) library.
 
48
 
49
  In this section we compare the perplexity of Qra models on Polish texts with other Polish and English LLMs.
50
 
51
+ Note that perplexity values between different text segmentations are not directly comparable. Therefore, we can draw conclusions based on comparisons only beetween models using the same tokenizer, such as Qra and the original LLama / TinyLLama.
52
+
53
  ### PolEval-2018
54
 
55
  In 2018, the PolEval competition included a language modeling task, for which training and test sets totaling over 20 million Polish sentences were made available. We used the first 10k sentences from the test set to evaluate modern neural language models. To calculate the perplexity, we used a script from the [HuggingFace Evaluate](https://huggingface.co/spaces/evaluate-metric/perplexity) library.