Repository: EleutherAI/lm-evaluation-harness
Issue: Raising TypeError when using simple_evaluate()
Root cause: Documentation Deficiency

Hi, I want to test the performance of llama3.1 using lm_eval. And I using the [External Library Usage
](https://github.com/EleutherAI/lm-evaluation-harness/blob/main/docs/interface.md#external-library-usage) to achieve this, but unfortunately it raises TypeError. Here is my code:
```import lm_eval
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, BitsAndBytesConfig

bnb_config = BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_use_double_quant=True,
        bnb_4bit_quant_type="nf4",
        bnb_4bit_compute_dtype=torch.bfloat16
    )

model = AutoModelForCausalLM.from_pretrained('/public/MountData/yaolu/LLM_pretrained/pruned_model/pruned_Llama-3.1-8B-Instruct_tail/',
        trust_remote_code=True, quantization_config=bnb_config, device_map='auto'
    )
lora_model = PeftModel.from_pretrained(
        model,
        '/public/MountData/yaolu/LLM_pretrained/pruned_model/finetuned_Qlora_alpaca_lama-3.1-8B-Instruct_tail/',
        torch_dtype=torch.float16,
    )

print("Applying the LoRA")
model = lora_model.merge_and_unload()
print(model)
print(type(model))
task_manager = lm_eval.tasks.TaskManager(include_path='/public/MountData/yaolu/lm-evaluation-harness/lm_eval/tasks')

results = lm_eval.simple_evaluate( # call simple_evaluate
    model=model,
    tasks=['arc_challenge', 'arc_easy'],
    num_fewshot=0,
    task_manager=task_manager,
    batch_size='auto',
    max_batch_size=None,
    device='cuda:0',
    use_cache=None,
    limit=None,
    check_integrity=False,
    write_out=False,
    gen_kwargs=None
)```
So how can I achieve this. Thanks for your reply!


Don't click the link or run that: this user is trying to spread malware.

> Don't click the link or run that: this user is trying to spread malware.

Thanks!

(Thanks @tmcw for flagging the spam reply!)

Hi @yaolu-zjut ! 

Your script is nearly correct but is missing one expected line:

```
lora_model = PeftModel.from_pretrained(
        model,
        '/public/MountData/yaolu/LLM_pretrained/pruned_model/finetuned_Qlora_alpaca_lama-3.1-8B-Instruct_tail/',
        torch_dtype=torch.float16,
    )
print("Applying the LoRA")
model = lora_model.merge_and_unload()
print(model)
print(type(model))

# new: **We need to wrap our HF initialized model with the eval harness' HFLM class, which implements generation loops and loglikelihood collection!**
lm_obj = Your_LM(model=model, batch_size='auto')

results = lm_eval.simple_evaluate( # call simple_evaluate
    model=lm_obj # NOTE this is the `lm_obj` not `model` !
	...
```

I'll make the TypeError more descriptive in #2258 ! Please let me know if this does not answer your question.


> (Thanks @tmcw for flagging the spam reply!)
> 
> Hi @yaolu-zjut !
> 
> Your script is nearly correct but is missing one expected line:
> 
> ```
> lora_model = PeftModel.from_pretrained(
>         model,
>         '/public/MountData/yaolu/LLM_pretrained/pruned_model/finetuned_Qlora_alpaca_lama-3.1-8B-Instruct_tail/',
>         torch_dtype=torch.float16,
>     )
> print("Applying the LoRA")
> model = lora_model.merge_and_unload()
> print(model)
> print(type(model))
> 
> # new: **We need to wrap our HF initialized model with the eval harness' HFLM class, which implements generation loops and loglikelihood collection!**
> lm_obj = Your_LM(model=model, batch_size='auto')
> 
> results = lm_eval.simple_evaluate( # call simple_evaluate
>     model=lm_obj # NOTE this is the `lm_obj` not `model` !
> 	...
> ```
> 
> I'll make the TypeError more descriptive in #2258 ! Please let me know if this does not answer your question.

Hi, Thanks for your reply! May I ask what is "Your_LM". I do not find this.

I have solved. Thanks!