Repository: EleutherAI/lm-evaluation-harness
Issue: 0.4.8 -> 0.4.9: TypeError: argument 'add_special_tokens': 'NoneType' object cannot be converted 'PyBool'
Root cause: Interface Contract Mismatch

```
Running generate_until requests:   0%|          | 0/196 [00:00<?, ?it/s]Traceback (most recent call 
last):                                                                                              
  File "/media/jason/steering-vector-on-reasoning-model/experiments/evaluate_lm_eval/main
.py", line 132, in <module>                                                                         
    results = simple_evaluate(                                                                      
              ^^^^^^^^^^^^^^^^                                                                      
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/lm_eval/utils.py", line 4
39, in _wrapper                                                                                     
    return fn(*args, **kwargs)                                                                      
           ^^^^^^^^^^^^^^^^^^^                                                                      
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/lm_eval/evaluator.py", li
ne 342, in simple_evaluate                                                                          
    results = evaluate(
              ^^^^^^^^^
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/lm_eval/utils.py", line 4
39, in _wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/lm_eval/evaluator.py", li
ne 570, in evaluate
    resps = getattr(lm, reqtype)(cloned_reqs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/lm_eval/models/huggingfac
e.py", line 1338, in generate_until
    for chunk in chunks:
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/lm_eval/models/utils.py",
 line 429, in get_batched 
    yield from batch
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/lm_eval/models/utils.py",
 line 620, in get_chunks
    _iter = tuple(_iter)
            ^^^^^^^^^^^^
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/lm_eval/models/utils.py",
 line 518, in _reorder
    arr = sorted(arr, key=self._sort_fn)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/lm_eval/models/utils.py",
 line 377, in <lambda>
    self._sort_fn = lambda x: sort_fn(x[1])
                              ^^^^^^^^^^^^^
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/lm_eval/models/huggingfac
e.py", line 1297, in _collate
    toks = self.tok_encode(req[0])
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/lm_eval/models/huggingfac
e.py", line 822, in tok_encode
    encoding = self.tokenizer.encode(string, **special_tokens_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/transformers/tokenization
_utils_base.py", line 2654, in encode
    encoded_inputs = self.encode_plus(
                     ^^^^^^^^^^^^^^^^^
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/transformers/tokenization
_utils_base.py", line 3073, in encode_plus
    return self._encode_plus(
           ^^^^^^^^^^^^^^^^^^
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/transformers/tokenization
_utils_fast.py", line 613, in _encode_plus
    batched_output = self._batch_encode_plus(
                     ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/jason/recurrent-env/lib/python3.11/site-packages/transformers/tokenization
_utils_fast.py", line 539, in _batch_encode_plus
    encodings = self._tokenizer.encode_batch(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument 'add_special_tokens': 'NoneType' object cannot be converted to 'PyBool'

Running generate_until requests:   0%|          | 0/196 [00:00<?, ?it/s]
############################## Command 1/1 failed with return code 1. Skipping to the next experimen
t. ##############################
############################## Experiment 1 failed at command 1. ##############################


Experiment Results Summary
############################################################
Experiment 1: Failed at command 1
```

Hi! is your model non-causal? If not you can explicitly set `backend="causal"` in `model_args` right now, and I'll see about making this more robust

> Hi! is your model non-causal? If not you can explicitly set `backend="causal"` in `model_args` right now, and I'll see about making this more robust

Hi, sorry for the late respond, somehow I didn't get the email notification

The model is causal

Explicitly set `backend="causal"` in the wrapped model, e.g. `model = HuginnWrapper(..., backend="causal")` and then pass it to `simpel_evaluate(model=model)` does not fix the issue.