snorkelai

/

Snorkel-Mistral-PairRM-DPO

like 105

Hi! Have you tried setting tokenizer.eos_token_id = tokenizer.pad_token_id

With that, I tested and the model stopped when seeing </s>

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
torch.set_default_device('cuda')

model = AutoModelForCausalLM.from_pretrained("snorkelai/Snorkel-Mistral-PairRM-DPO",
                                             torch_dtype="auto")

tokenizer = AutoTokenizer.from_pretrained("snorkelai/Snorkel-Mistral-PairRM-DPO",
                                          torch_dtype="auto")

text = "<s>[INST] What is your favourite condiment? [/INST]Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s> [INST] Do you have mayonnaise recipes? [/INST]"

encodeds = tokenizer(text, return_tensors="pt", add_special_tokens=False)

device = 'cuda'
model_inputs = encodeds.to(device)
model.to(device)

generated_ids = model.generate(**model_inputs, max_new_tokens=1000, do_sample=True)
decoded = tokenizer.batch_decode(generated_ids)
print(decoded[0])

>>> Output: 
"""<s> [INST] What is your favourite condiment? [/INST]Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s> [INST] Do you have mayonnaise recipes? [/INST] Yes, I can certainly provide you with a classic mayonnaise recipe. Here it is:

Classic Mayonnaise

Ingredients:
- 1 egg yolk (at room temperature)
- 1 teaspoon Dijon mustard
- 1 cup (200 ml) vegetable oil
- 1/2 cup (120 ml) lightly flavored oil (such as canola or grapeseed oil)
- 1-2 tablespoons fresh lemon juice
- Salt and pepper, to taste

Instructions:
1. In a stainless steel or glass bowl, whisk together the egg yolk and mustard.
2. Very slowly, drop in the vegetable oil, 1 tablespoon at a time, while continually whisking. Once all the vegetable oil has been incorporated, continue adding the lightly flavored oil in a thin stream, still whisking constantly to prevent the mayonnaise from breaking.
3. When all the oil has been added and the mayonnaise has thickened, whisk in the lemon juice, followed by salt and pepper to taste.
4. If the mayonnaise is too thick, thin it out with a few drops of water. If it's too runny, you may need to add a bit more oil or egg yolk.
5. Refrigerate the mayonnaise, covered, for at least an hour before serving to allow the flavors to meld and the texture to become firm.

You can also use a blender or food processor to make mayonnaise, but whisking it by hand ensures a smoother and creamier texture. Be sure to use fresh ingredients, as old eggs or rancid oil can lead to unpleasant flavors and textures. Enjoy your homemade mayonnaise!</s>"""

Also, for faster inference, we recommend using vllm, which allows stop parameter as well, which you can include </s> into the stop tokens list.

Thanks @viethoangtranduong ! This works well.