databricks

/

dolly-v2-3b

like 282

pad_token_id no longer needs to be passed explicitly. generate_kwargs dictionary already contains it. Python does not know which one to pick.

TypeError: transformers.generation.utils.GenerationMixin.generate() got multiple values for keyword argument 'pad_token_id'

Removing the attribute override fixes the error.

Script

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from instruct_pipeline import InstructionTextGenerationPipeline

model_id = "databricks/dolly-v2-3b"

tokenizer = AutoTokenizer.from_pretrained(model_id, padding_side="left")
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", torch_dtype=torch.bfloat16)

generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)

res = generate_text("Explain to me the difference between nuclear fission and fusion.")

Error

Traceback (most recent call last):
  File "/home/alexis/Documents/python/LLM/dolly/script.py", line 12, in <module>
    res = generate_text("Explain to me the difference between nuclear fission and fusion.")
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/alexis/.conda/envs/llm/lib/python3.12/site-packages/transformers/pipelines/base.py", line 1242, in __call__
    return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/alexis/.conda/envs/llm/lib/python3.12/site-packages/transformers/pipelines/base.py", line 1249, in run_single
    model_outputs = self.forward(model_inputs, **forward_params)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/alexis/.conda/envs/llm/lib/python3.12/site-packages/transformers/pipelines/base.py", line 1149, in forward
    model_outputs = self._forward(model_inputs, **forward_params)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/alexis/Documents/python/LLM/dolly/instruct_pipeline.py", line 125, in _forward
    generated_sequence = self.model.generate(
                         ^^^^^^^^^^^^^^^^^^^^
TypeError: transformers.generation.utils.GenerationMixin.generate() got multiple values for keyword argument 'pad_token_id'