Hello @mrizwanse!

Could you please provide the code you are using to load the model?

Regards,
Gustavo.

Importing Python packages

from environs import Env

Importing HuggingFace packages

from transformers import TFAutoModelForCausalLM, AutoTokenizer, pipeline

Importing LangChain packages

from langchain.llms.huggingface_pipeline import HuggingFacePipeline
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain

---------------------------------------------------------------------------

Read environment variables from .env file

env = Env()
env.read_env(".env")

HuggingFace Hub API token

HUGGINGFACEHUB_API_TOKEN = env.str("HUGGINGFACEHUB_API_TOKEN")

---------------------------------------------------------------------------

1. Image to Text (Image Captioning)

def img2txt(img_url):
image_to_text = pipeline(
task="image-to-text", model="Salesforce/blip-image-captioning-base"
)

text = image_to_text(img_url)[0]["generated_text"]

# print("\n\n", text)
return text

2. LLM to generate story

def generate_story(scenario):
template = """
You are a story teller;
You can generate a short story based on a simple narrative, the story should be no more than 100 words;

CONTEXT: {scenario}
STORY:
"""

prompt = PromptTemplate(template=template, input_variables=["scenario"])

model_id = "microsoft/phi-2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = TFAutoModelForCausalLM.from_pretrained(
    model_id, trust_remote_code=True
)
pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    max_new_tokens=70,
)
story_llm = HuggingFacePipeline(pipeline=pipe)

llm_chain = LLMChain(prompt=prompt, llm=story_llm, verbose=True)

story = llm_chain.run(scenario)

print("\n\n", story)
return story

generated_text = img2txt("images/rizwan.jpg")
generated_story = generate_story(generated_text)

Python

• 3.11.4

Python pip

• 23.3.1

TensorFlow

• 2.15.0

Trransformers

• 4.35.2

Environs

• 9.5.0

PIL

• 10.1.0

LangChain

• 0.0.345

There is an ongoing PR which should mitigate such an issue: https://github.com/huggingface/transformers/pull/28163.

We will use it to update Phi-2's code.