lvwerra HF staff commited on
Commit
b2a20e2
1 Parent(s): 709b020

Update Space (evaluate main: 7fa754cc)

Browse files
Files changed (2) hide show
  1. perplexity.py +1 -0
  2. requirements.txt +1 -1
perplexity.py CHANGED
@@ -49,6 +49,7 @@ Args:
49
  add_start_token (bool): whether to add the start token to the texts,
50
  so the perplexity can include the probability of the first word. Defaults to True.
51
  device (str): device to run on, defaults to 'cuda' when available
 
52
  Returns:
53
  perplexity: dictionary containing the perplexity scores for the texts
54
  in the input list, as well as the mean perplexity. If one of the input texts is
49
  add_start_token (bool): whether to add the start token to the texts,
50
  so the perplexity can include the probability of the first word. Defaults to True.
51
  device (str): device to run on, defaults to 'cuda' when available
52
+ max_length (int): the maximum length to truncate input texts to. Should be set to the maximum length the model supports. Defaults to None.
53
  Returns:
54
  perplexity: dictionary containing the perplexity scores for the texts
55
  in the input list, as well as the mean perplexity. If one of the input texts is
requirements.txt CHANGED
@@ -1,3 +1,3 @@
1
- git+https://github.com/huggingface/evaluate@75c09ff6cd599744d0641e4ef1d4cfedbd35eec9
2
  torch
3
  transformers
1
+ git+https://github.com/huggingface/evaluate@7fa754cc38c523b69054c67b8c3b8e3f4962fbf6
2
  torch
3
  transformers