Repository: dottxt-ai/outlines

Issue:
ValueError: The following `model_kwargs` are not used by the model: ['tokenizer'] (note: typos in the generate arguments will also show up in this list)
### Describe the issue as clearly as possible:

Unable to try a simple transformers + outlines usecase

### Steps/code to reproduce the bug:

```python
import outlines
from transformers import AutoTokenizer, AutoModelForCausalLM

import torch
MODEL_NAME = "deepseek-ai/deepseek-coder-33b-instruct"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto", torch_dtype=torch.bfloat16),
    AutoTokenizer.from_pretrained(MODEL_NAME)
)

response = model("How many countries are there in the world", max_new_tokens=20)
```

### Expected result:

```shell
It should provide output without erroring out
```

### Error message:

```shell
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[43], line 1
----> 1 response = model("How many countries are there in the world", max_new_tokens=20)

File /opt/conda/lib/python3.11/site-packages/outlines/models/base.py:114, in Model.__call__(self, model_input, output_type, **inference_kwargs)
     83 """Call the model.
     84 
     85 Users can call the model directly, in which case we will create a
   (...)    110 
    111 """
    112 from outlines import Generator
--> 114 return Generator(self, output_type)(model_input, **inference_kwargs)

File /opt/conda/lib/python3.11/site-packages/outlines/generator.py:267, in SteerableGenerator.__call__(self, prompt, **inference_kwargs)
    251 def __call__(self, prompt: Any, **inference_kwargs) -> Any:
    252     """Generate a response from the model.
    253 
    254     Parameters
   (...)    265 
    266     """
--> 267     return self.model.generate(
    268         prompt, self.logits_processor, **inference_kwargs
    269     )

File /opt/conda/lib/python3.11/site-packages/outlines/models/transformers.py:289, in Transformers.generate(self, model_input, output_type, **inference_kwargs)
    286 prompts, inputs = self._prepare_model_inputs(model_input, output_type)
    287 logits_processor = self.type_adapter.format_output_type(output_type)
--> 289 generated_ids = self._generate_output_seq(
    290     prompts, inputs, logits_processor=logits_processor, **inference_kwargs
    291 )
    293 # if single str input, convert to a 1D outputt
    294 if isinstance(prompts, str):

File /opt/conda/lib/python3.11/site-packages/outlines/models/transformers.py:311, in Transformers._generate_output_seq(self, prompts, inputs, **inference_kwargs)
    309 def _generate_output_seq(self, prompts, inputs, **inference_kwargs):
    310     input_ids = inputs["input_ids"]
--> 311     output_ids = self.model.generate(
    312         **inputs,
    313         tokenizer=self.transformer_tokenizer,
    314         **inference_kwargs,
    315     )
    317     # encoder-decoder returns output_ids only, decoder-only returns full seq ids
    318     if self.model.config.is_encoder_decoder:

File /opt/conda/lib/python3.11/site-packages/torch/utils/_contextlib.py:115, in context_decorator.<locals>.decorate_context(*args, **kwargs)
    112 @functools.wraps(func)
    113 def decorate_context(*args, **kwargs):
    114     with ctx_factory():
--> 115         return func(*args, **kwargs)

File /opt/conda/lib/python3.11/site-packages/transformers/generation/utils.py:1530, in GenerationMixin.generate(self, inputs, generation_config, logits_processor, stopping_criteria, prefix_allowed_tokens_fn, synced_gpus, assistant_model, streamer, negative_prompt_ids, negative_prompt_attention_mask, **kwargs)
   1528 model_kwargs = generation_config.update(**kwargs)  # All unused kwargs must be model kwargs
   1529 generation_config.validate()
-> 1530 self._validate_model_kwargs(model_kwargs.copy())
   1532 # 2. Set generation parameters if not already defined
   1533 logits_processor = logits_processor if logits_processor is not None else LogitsProcessorList()

File /opt/conda/lib/python3.11/site-packages/transformers/generation/utils.py:1344, in GenerationMixin._validate_model_kwargs(self, model_kwargs)
   1341         unused_model_args.append(key)
   1343 if unused_model_args:
-> 1344     raise ValueError(
   1345         f"The following `model_kwargs` are not used by the model: {unused_model_args} (note: typos in the"
   1346         " generate arguments will also show up in this list)"
   1347     )

ValueError: The following `model_kwargs` are not used by the model: ['tokenizer'] (note: typos in the generate arguments will also show up in this list)
```

### Outlines/Python version information:

Version information
<details>
```
outlines 1.0.3
Python 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0]
transformers==4.50.0
```
</details>


### Context for the issue:

Trying out a simple use case with outlines


Discussion:
Hi @kmehant! Thanks for opening an issue. I have opened a PR to fix this bug: #1679 
Many thanks @RobinPicard 
The fix is available in the latest release :)



Patch:
diff --git a/outlines/models/transformers.py b/outlines/models/transformers.py
index 19350de52..a247c7e56 100644
--- a/outlines/models/transformers.py
+++ b/outlines/models/transformers.py
@@ -220,7 +220,6 @@ def __init__(
 
         tokenizer.padding_side = "left"
         self.model = model
-        self.transformer_tokenizer = tokenizer
         self.tokenizer = TransformerTokenizer(tokenizer)
         self.type_adapter = TransformersTypeAdapter()
 
@@ -332,9 +331,9 @@ def generate_stream(self, model_input, output_type, **inference_kwargs):
 
     def _generate_output_seq(self, prompts, inputs, **inference_kwargs):
         input_ids = inputs["input_ids"]
+
         output_ids = self.model.generate(
             **inputs,
-            tokenizer=self.transformer_tokenizer,
             **inference_kwargs,
         )
 


Test patch:
diff --git a/tests/v0_legacy/models/test_transformers_legacy.py b/tests/v0_legacy/models/test_transformers_legacy.py
index 2c34c2517..0388a4348 100644
--- a/tests/v0_legacy/models/test_transformers_legacy.py
+++ b/tests/v0_legacy/models/test_transformers_legacy.py
@@ -168,5 +168,6 @@ def test_transformers_legacy_call_generation():
         "foo",
         2,
         length_penalty=0.5,
+        tokenizer=model.tokenizer.tokenizer,
     )
     assert isinstance(result, str)
diff --git a/tests/v0_legacy/models/test_transformers_vision_legacy.py b/tests/v0_legacy/models/test_transformers_vision_legacy.py
index bcaf4ea3e..cd7324555 100644
--- a/tests/v0_legacy/models/test_transformers_vision_legacy.py
+++ b/tests/v0_legacy/models/test_transformers_vision_legacy.py
@@ -106,5 +106,6 @@ def test_transformers_vision_legacy_call_generation():
         "foo",
         2,
         length_penalty=0.5,
+        tokenizer=model.tokenizer.tokenizer,
     )
     assert isinstance(result, str)