perplexity-lenses / README.md

Commit History

Run tokenizer before computing perplexity and format
7b62017

edugp commited on

Add tests and fix issue when splitting into sentences, to grab the minimum number between total sentences and sample size, rather than total original documents and sample size
d131aa3

edugp commited on

Update README
6d1a001

edugp commited on

initial commit
77d22a6

system HF staff commited on