|
--- |
|
license: mit |
|
datasets: |
|
- truthfulqa/truthful_qa |
|
--- |
|
|
|
Have you ever wanted a more truthful LLaMA with minimum intervention? Using the inference-time intervention (ITI) method discussed in [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://arxiv.org/pdf/2306.03341.pdf) (Li et al.), we baked into various LLaMa models a lightweight intervention that improves the edited models' truthfulness scores on the TruthfulQA dataset. This model was obtained via baking in ITI with alpha=15 on the top k=48 attention heads (more information on hyperparameters in paper). |
|
|
|
Codebase: https://github.com/likenneth/honest_llama |
|
|
|
You can load and play around starting from below: |
|
|
|
```python |
|
import torch |
|
from pprint import pprint |
|
from transformers import AutoConfig, AutoTokenizer, AutoModelForCausalLM |
|
|
|
model_name_new = "jujipotle/honest_llama2_chat_70B" |
|
tokenizer_new = AutoTokenizer.from_pretrained(model_name_new, trust_remote_code=True) |
|
model_new = AutoModelForCausalLM.from_pretrained(model_name_new, low_cpu_mem_usage = True, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True) |
|
|
|
q = "I ate a cherry seed. Will a cherry tree grow in my stomach?" |
|
encoded_new = tokenizer_new(q, return_tensors = "pt")["input_ids"] |
|
generated_new = model_new.generate(encoded_new.cuda())[0, encoded_new.shape[-1]:] |
|
decoded_new = tokenizer_new.decode(generated_new, skip_special_tokens=True).strip() |
|
pprint(decoded_new) |
|
``` |
