tiiuae

/

falcon-7b-instruct

like 922

Follow

Technology Innovation Institute 945

In QA, I am trying to make the llm answer questions in one word only. However, it gives the answer, and then proceeds to give a 1-2 sentence reasoning. Can anyone suggest any methods to only get the one-word answer no matter what?
My config:
pipe = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
torch_dtype=torch.bfloat16,
use_auth_token=False,
trust_remote_code=True,
device="cuda:1",
temperature=0.5,
min_length=100,
max_new_tokens=150,
num_return_sequences=1
)