working with huggingface Llama 2 13b chat hf model_kwargs value error
Author: Mrjaggu
Pull request: no

using Llama 2 13b chat hf model (https://huggingface.co/meta-llama/Llama-2-13b-chat-hf) with 4bit quantization (bitsandbytes)

getting an error in the following code.. it used to work earlier

```python
generate_text = transformers.pipeline(
model=model, tokenizer=tokenizer,
return_full_text=True, # langchain expects the full text
task='text-generation',
temperature=0.0,
max_new_tokens=2000,
repetition_penalty=1.1
)
```

ValueError: The following 'model_kwargs' are not used by the model: ['max_new_token'','repetition_policy'] (note: Typos in the generate arguments will also show up in this list)

Full code:

```python
from torch import cuda, bfloat16
import transformers

model_id = 'meta-llama/Llama-2-13b-chat-hf'

device = f'cuda:{cuda.current_device()}' if cuda.is_available() else 'cpu'
print(f"Device avialble is on {device}")

bnb_config = transformers.BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type='nf4',
bnb_4bit_use_double_quant=True,
bnb_4bit_compute_dtype=bfloat16
)

hf_auth = '####'

model_config = transformers.AutoConfig.from_pretrained(
model_id,
use_auth_token=hf_auth
)

tokenizer = transformers.AutoTokenizer.from_pretrained(
model_id,
use_auth_token=hf_auth
)

model = transformers.AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
config=model_config,
quantization_config=bnb_config,
device_map='auto',
use_auth_token=hf_auth
)

generate_text = transformers.pipeline(
model=model, tokenizer=tokenizer,
return_full_text=True, # langchain expects the full text
task='text-generation',
temperature=0.0,
max_new_tokens=2000,
repetition_penalty=1.1
)
```

Error trace:
```
warnings.warn( Loading checkpoint shards: 100%
Model loaded and downloaded succesfully
3/3 100:17 00:00, 5.86/it
/opt/data1/python3.8 virtualenv/virtualenvs/transformers/lib/python3.8/site- packages/transformers/tokenization_utils_base.py:1714: FutureWarning. The use auth token argument is deprecated and will be removed in v5 of Transformers.
warnings.wam Traceback (most recent call last):
File "inference_test.py", line 73, in <module>
res generate_text("Explain encoder decoder model"), File "/opt/data/python3 8 virtualenv/virtualenvs/transformers/lib/python3.8/site-
packages/transformers/pipelines/text generation.py", line 200, in __call_
return super call (text inputs, **kwargs)
File "/opt/data1/python3.8_virtualenv/virtualenvs/transformers/lib/python3.5/site-
packages/transformers/pipelines/base.py", line 1122, in call
return self.run_single(inputs, preprocess params, forward params, postprocess params) File "/opt/data/python3.8 virtualenv/virtualenvs/transformers/lib/python3.8/site- packages/transformers/pipelines/base.py", line 1129, in run single
model outputs self.forward(model inputs, forward_params)
File "/opt/data1/python3.8 virtualenv/virtualenvs/transformers/lib/python3.8/site-
packages/transformers/pipelines/base.py", line 1028, in forward
model outputs = self forward(model inputs, forward_params)
File "/opt/data1/python3.8 virtualenv/virtualenvs/transformers/lib/python3.8/site
packages/transformers/pipelines/text generation.py", line 261, in forward
generated sequence = self.model.generate(input_ids-input ids, attention mask-attention_mask, **generate_kwargs) File "/opt/data1/python3.8_virtualenv/virtualenvs/transformers/lib/python3.8/site- packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/opt/data1/python3.8_virtualenv/virtualenvs/transformers/lib/python3.8/site- packages/transformers/generation/utils.py, line 1282, in generate
self validate model kwargs(model_kwargs.copy())
File "/opt/data1/python3.8 virtualenv/virtualenvs/transformers/lib/python3.5/site packages/transformers/generation/utils.py", line 1155, in _validate model kwargs
raise ValueError(
ValueError. The following model kwargs are not used by the model: I'max_new_token',"repetition penality'] (note: typos in the generate arguments will also show up in this list)

```

Comments:
Thanks for reporting but you are not giving us the full script. The traceback shows that the errors appears at: 
```python 
generate_text("Explain encoder decoder model")
```
which I don't have access to. 
The traceback seems a bit strange, but anyway. I think that some of the generation kwargs were not passed as you are showing it: `max_new_token` is missing a `s` and 'repetition penality' has an extra space. 
Please provide a full traceback as this one seems wrong 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.