Multi-turn Conversations with MiniCPM-V: Exploring Single Image Understanding

#3
by JamePeng2023 - opened

Hello openbmb team, can MiniCPM-V engage in multi-turn conversations about a single image?

OpenBMB org
edited Apr 16
# First round chat 
msgs = [{"role": "user", "content": "Where should I go to buy a camera?"}]
res, context, _ = model.chat(
    image=image,
    msgs=msgs,
    tokenizer=tokenizer
)
print(res)

# Second round chat ,  append history context to msgs
msgs.append({"role": "assistant", "content": res})
msgs.append({"role": "user", "content": "Where is this store in the image?"})

res, context, _ = model.chat(
    image=image,
    msgs=msgs,
    tokenizer=tokenizer
)
print(res)

It's really a very interesting method. I get the last picture from the chat history and transfer it to the conversation with the big model.

OpenBMB org

A bug fixed, msgs would be changed after calling model.chat(), please pull the latest file modeling_minicpmv.py

Sign up or log in to comment