---
license: apache-2.0
pipeline_tag: text-generation
---


<p align="center">
    <img src="logo.png" width="400"/>
<p>

<p align="center">
    <b><font size="6">InternLM-XComposer2</font></b> 
<p>

<div align="center">

[💻Github Repo](https://github.com/InternLM/InternLM-XComposer)

</div>

**InternLM-XComposer2** is a vision-language large model (VLLM) based on [InternLM2](https://github.com/InternLM/InternLM) for advanced text-image comprehension and composition. 

We release InternLM-XComposer2 series in two versions:

- InternLM-XComposer2-VL: The pretrained VLLM model with InternLM2 as the initialization of the LLM, achieving strong performance on various multimodal benchmarks.
- InternLM-XComposer2: The finetuned VLLM for *Free-from Interleaved Text-Image Composition*.

### Import from Transformers
To load the InternLM-XComposer2-7B model using Transformers, use the following code:
```python
import torch
from PIL import Image
from transformers import AutoTokenizer, AutoModelForCausalLM
ckpt_path = "internlm/internlm-xcomposer2-7b"
tokenizer = AutoTokenizer.from_pretrained(ckpt_path, trust_remote_code=True).cuda()
# Set `torch_dtype=torch.float16` to load model in float16, otherwise it will be loaded as float32 and might cause OOM Error.
model = AutoModelForCausalLM.from_pretrained(ckpt_path, torch_dtype=torch.float16, trust_remote_code=True).cuda()
model = model.eval()
img_path_list = [
    './panda.jpg',
    './bamboo.jpeg',
]
images = []
for img_path in img_path_list:
    image = Image.open(img_path).convert("RGB")
    image = model.vis_processor(image)
    images.append(image)
image = torch.stack(images)
query = '<ImageHere> <ImageHere>please write an article based on the images. Title: my favorite animal.'
with torch.cuda.amp.autocast():
    response, history = model.chat(tokenizer, query=query, image=image, history=[], do_sample=False)
print(response)
""""
# My favorite animal is the panda. Pandas are one of the most popular animals in the world, and for good reason. They are adorable, cuddly creatures that have captured the hearts of people all over the globe.
Pandas are native to China and can be found in the wild in a few specific regions. However, they are also very popular in captivity, with many zoos around the world housing pandas as part of their exhibits. I have been fortunate enough to see pandas up close at several different zoos, and each time it was an amazing experience.
One thing that always strikes me about pandas is how much they love to eat bamboo. In fact, pandas spend almost all of their waking hours eating bamboo. This may not seem like a lot of fun, but pandas actually enjoy munching on this tough plant. It's fascinating to watch them chew through the tough stalks and leaves, and then lick their lips in satisfaction.
Another thing that I find interesting about pandas is their black and white fur. The combination of these two colors creates a striking contrast that makes pandas instantly recognizable. In addition, the black patches on their face give them a unique expression that seems to convey both playfulness and seriousness.
Despite their popularity, pandas do face some challenges. Their habitat is being destroyed by human activities such as logging and agriculture, which has led to a decline in their population. Additionally, pandas are considered endangered due to factors such as low reproductive rates and limited genetic diversity.
However, there are efforts underway to protect pandas and their habitats. Many organizations work to raise awareness about the importance of preserving these beautiful creatures, and governments in countries where pandas live are taking steps to conserve their natural environment.
In conclusion, pandas are truly remarkable animals that deserve our admiration and protection. With their distinctive appearance, playful personalities, and love of bamboo, it's no wonder that pandas have become so beloved around the world. Let's do what we can to ensure that future generations can continue to appreciate these wonderful creatures.
"""
```

### 通过 Transformers 加载
通过以下的代码加载 InternLM-XComposer2-7B 模型

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
ckpt_path = "internlm/internlm-xcomposer2-7b"
tokenizer = AutoTokenizer.from_pretrained(ckpt_path, trust_remote_code=True).cuda()
# `torch_dtype=torch.float16` 可以令模型以 float16 精度加载，否则 transformers 会将模型加载为 float32，导致显存不足
model = AutoModelForCausalLM.from_pretrained(ckpt_path, torch_dtype=torch.float16, trust_remote_code=True).cuda()
model = model.eval() 
img_path_list = [
    './panda.jpg',
    './bamboo.jpeg',
]
images = []
for img_path in img_path_list:
    image = Image.open(img_path).convert("RGB")
    image = model.vis_processor(image)
    images.append(image)
image = torch.stack(images)
query = '<ImageHere> <ImageHere>请根据图片写一篇作文：我最喜欢的小动物。要求：选准角度，确定立意，明确文体，自拟标题；不要套作，不得抄袭；不得泄露个人信息。'
with torch.cuda.amp.autocast():
    response, history = model.chat(tokenizer, query=query, image=image, history=[], do_sample=False)
print(response)
"""
# 我最喜欢的小动物
我喜欢许多小动物，有活泼可爱的小狗、美丽高贵的孔雀……但我最喜欢的是那一只憨态可掬的大熊猫。
大熊猫的毛色黑白分明，身体胖乎乎的，四肢短小，走起路来摇摇晃晃，样子十分可爱。它的头又圆又大，耳朵小小的，眼睛黑黑的，像两颗闪闪发光的黑宝石。它还有一张大大的嘴巴，里面长着锋利的牙齿，看起来非常凶猛。但是，别看它长得这么凶猛，其实它很胆小，一听到一点声音就会吓得躲到树后面去。
大熊猫是国宝，也是我们中国的象征。它们喜欢吃竹子，每天要吃30多斤的竹子，所以人们给它取了一个外号叫“食竹动物”。
大熊猫不仅爱吃竹子，还爱睡觉。一天中，除了吃就是睡，吃饱了就睡，睡醒了再吃，真是一个名副其实的“大懒虫”。
这就是我喜欢的动物——大熊猫，你们喜欢吗？
"""
```