Repository: EleutherAI/lm-evaluation-harness
Issue: Issue with PEFT Model, Model Shape Mismatch
Root cause: Interface Contract Mismatch

My Model is a PEFT model finetuned with SFT.

Model lInk : https://huggingface.co/hvadaparty/Featherlite-Aurora-v0.3-beta

i was able to sucessfully infer from this model with tgi using the below

```python
model = AutoPeftModelForCausalLM.from_pretrained(
  peft_model_id,
  device_map="auto",
  torch_dtype=torch.float16
)
tokenizer = AutoTokenizer.from_pretrained(peft_model_id)
```

I have used the below command for eval
```
lm_eval --model hf --model_args pretrained=hvadaparty/Featherlite-Aurora-v0.3-beta,peft=true --tasks hellaswag --wandb_args project=lm-eval-harness-integration   --device cuda:0     --batch_size auto  --output_path output/featherlite-v0.3
```

this is the error

```
config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 664/664 [00:00<00:00, 3.50MB/s]
adapter_config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████| 683/683 [00:00<00:00, 3.69MB/s]
Downloading shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 18.11it/s]
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████| 8/8 [00:02<00:00,  3.31it/s]
adapter_model.safetensors: 100%|███████████████████████████████████████████████████████████████████████████| 1.87G/1.87G [00:43<00:00, 42.7MB/s]
Traceback (most recent call last):
  File "/usr/local/bin/lm_eval", line 8, in <module>
    sys.exit(cli_evaluate())
  File "/usr/local/lib/python3.10/dist-packages/lm_eval/__main__.py", line 342, in cli_evaluate
    results = evaluator.simple_evaluate(
  File "/usr/local/lib/python3.10/dist-packages/lm_eval/utils.py", line 288, in _wrapper
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/lm_eval/evaluator.py", line 164, in simple_evaluate
    lm = lm_eval.api.registry.get_model(model).create_from_arg_string(
  File "/usr/local/lib/python3.10/dist-packages/lm_eval/api/model.py", line 133, in create_from_arg_string
    return cls(**args, **args2)
  File "/usr/local/lib/python3.10/dist-packages/lm_eval/models/huggingface.py", line 201, in __init__
    self._create_model(
  File "/usr/local/lib/python3.10/dist-packages/lm_eval/models/huggingface.py", line 524, in _create_model
    self._model = self.AUTO_MODEL_CLASS.from_pretrained(
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
    return model_class.from_pretrained(
  File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 3787, in from_pretrained
    model.load_adapter(
  File "/usr/local/lib/python3.10/dist-packages/transformers/integrations/peft.py", line 206, in load_adapter
    incompatible_keys = set_peft_model_state_dict(self, processed_adapter_state_dict, adapter_name)
  File "/usr/local/lib/python3.10/dist-packages/peft/utils/save_and_load.py", line 222, in set_peft_model_state_dict
    load_result = model.load_state_dict(peft_model_state_dict, strict=False)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 2153, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for MistralForCausalLM:
        size mismatch for model.embed_tokens.weight: copying a param with shape torch.Size([32002, 4096]) from checkpoint, the shape in current model is torch.Size([32000, 4096]).
        size mismatch for lm_head.weight: copying a param with shape torch.Size([32002, 4096]) from checkpoint, the shape in current model is torch.Size([32000, 4096]).
Traceback (most recent call last):
  File "/usr/local/bin/lm_eval", line 8, in <module>
    sys.exit(cli_evaluate())
  File "/usr/local/lib/python3.10/dist-packages/lm_eval/__main__.py", line 342, in cli_evaluate
    results = evaluator.simple_evaluate(
  File "/usr/local/lib/python3.10/dist-packages/lm_eval/utils.py", line 288, in _wrapper
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/lm_eval/evaluator.py", line 164, in simple_evaluate
    lm = lm_eval.api.registry.get_model(model).create_from_arg_string(
  File "/usr/local/lib/python3.10/dist-packages/lm_eval/api/model.py", line 133, in create_from_arg_string
    return cls(**args, **args2)
  File "/usr/local/lib/python3.10/dist-packages/lm_eval/models/huggingface.py", line 201, in __init__
    self._create_model(
  File "/usr/local/lib/python3.10/dist-packages/lm_eval/models/huggingface.py", line 524, in _create_model
    self._model = self.AUTO_MODEL_CLASS.from_pretrained(
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
    return model_class.from_pretrained(
  File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 3787, in from_pretrained
    model.load_adapter(
  File "/usr/local/lib/python3.10/dist-packages/transformers/integrations/peft.py", line 206, in load_adapter
    incompatible_keys = set_peft_model_state_dict(self, processed_adapter_state_dict, adapter_name)
  File "/usr/local/lib/python3.10/dist-packages/peft/utils/save_and_load.py", line 222, in set_peft_model_state_dict
    load_result = model.load_state_dict(peft_model_state_dict, strict=False)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 2153, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for MistralForCausalLM:
        size mismatch for model.embed_tokens.weight: copying a param with shape torch.Size([32002, 4096]) from checkpoint, the shape in current model is torch.Size([32000, 4096]).
        size mismatch for lm_head.weight: copying a param with shape torch.Size([32002, 4096]) from checkpoint, the shape in current model is torch.Size([32000, 4096]).
```


Looks like the same error as #1707 ? I suspect a tokenizer misconfiguration is the culprit here.