google

/

flan-t5-large

like 460

Hi @namantjeaswi
Thanks for the issue ! can you share a reproducible snippet of the error?

Hello, Thank you for your response

Here is the code and I have also included my system information.

from transformers import pipeline
from transformers import AutoTokenizer, AutoModelForCausalLM, AutoModelForSeq2SeqLM
from langchain.llms import HuggingFacePipeline



#tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-large")
#model = AutoModelForSeq2SeqLM.from_pretrained("google/flan-t5-large") 


#save the model

save_directory = "flant-t5-large"
#tokenizer.save_pretrained(save_directory)    
#model.save_pretrained(save_directory)

#load the model
tokenizer = AutoTokenizer.from_pretrained(save_directory, local_files_only=True)
model = AutoModelForSeq2SeqLM.from_pretrained(save_directory, local_files_only=True)



pipe = pipeline(task="text2text-generation", 
                model=model,
                tokenizer=tokenizer,
                device='cuda'
                )   


local_llm = HuggingFacePipeline(pipeline=pipe)

res = local_llm("tell me about the paper attention is all you need")
print(res)

Error
File "C:\Users\Naman\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\transformers\generation\utils.py", line 1121, in _validate_model_kwargs
raise ValueError(
ValueError: The following model_kwargs are not used by the model: ['return_full_text'] (note: typos in the generate arguments will also show up in this list)

Fix I used: #commented line 1325 i.e. self._validate_model_kwargs(model_kwargs.copy()) in utils.py of transformers library to avoid error

System Information

• transformers version: 4.39.3
• Platform: Windows-11-10.0.22631-SP0
• Python version: 3.12.2
• Huggingface_hub version: 0.21.4
• Safetensors version: 0.4.2
• Accelerate version: 0.29.1
• Accelerate config: not found
• PyTorch version (GPU?): 2.2.2+cu118 (True)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?: Yes
• Using distributed or parallel set-up in script?: No

Thanks @namantjeaswi !
Hmmm this indicates the issue in on langchain side, I don't think the fix should go on transformers as the pipeline is correctly initialized
Would you mind opening an issue there? You can also ping me so that i can follow the issue and the potential fix

Okay I will open an issue with langchain on github and share it with you.

Thanks