JeasLee commited on
Commit
5a2ffc5
·
verified ·
1 Parent(s): 368caf0

Delete README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +0 -85
README.md DELETED
@@ -1,85 +0,0 @@
1
- ---
2
- license: apache-2.0
3
- base_model:
4
- - Qwen/Qwen2.5-VL-3B-Instruct
5
- - Qwen/Qwen2.5-VL-7B-Instruct
6
- - lmms-lab/llava-onevision-qwen2-7b-ov
7
- tags:
8
- - robotics
9
- - vision-language-action-model
10
- - vision-language-model
11
- library_name: transformers
12
-
13
- # Collection Metadata (Referencing InternRobotics/VLN-PE style)
14
- repo: InternRobotics/RoboInter-VLM
15
- type: "checkpoint-collection"
16
- description: "Collection of RoboInterVLM checkpoints and configs fine-tuned on RoboInter-VQA."
17
- checkpoints:
18
- - name: RoboInterVLM_qwenvl25_3b
19
- path: RoboInterVLM_qwenvl25_3b/
20
- notes: "Lightweight Qwen2.5-VL model"
21
- - name: RoboInterVLM_qwenvl25_7b
22
- path: RoboInterVLM_qwenvl25_7b/
23
- notes: "Stronger performance Qwen2.5-VL backbone"
24
- - name: RoboInterVLM_llava_one_vision_7B
25
- path: RoboInterVLM_llava_one_vision_7B/
26
- notes: "LLaVA-OneVision (SigLIP + Qwen2) backbone"
27
- ---
28
-
29
- # RoboInterVLM: Vision-Language Model Checkpoints for RoboInter Manipulation Suite
30
-
31
- Model checkpoints of **RoboInterVLM**, developed as part of the [RoboInter](https://github.com/InternRobotics/RoboInter) project. These models are fine-tuned on the [RoboInter-VQA](https://huggingface.co/datasets/InternRobotics/RoboInter-VQA) dataset for intermediate representation understanding and generation in robotic manipulation.
32
-
33
- ## Available Checkpoints
34
-
35
- | Checkpoint | Base Model | Architecture | Parameters | Description |
36
- |---|---|---|---|---|
37
- | `RoboInterVLM_qwenvl25_3b` | [Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct) | Qwen2.5-VL | ~3B | Lightweight Qwen2.5VL model, suitable for efficient deployment |
38
- | `RoboInterVLM_qwenvl25_7b` | [Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct) | Qwen2.5-VL | ~7B | Larger Qwen2.5-VL backbone for stronger performance |
39
- | `RoboInterVLM_llava_one_vision_7B` | [LLaVA-OneVision-Qwen2-7B](https://huggingface.co/lmms-lab/llava-onevision-qwen2-7b-ov) | LLaVA-OneVision (SigLIP + Qwen2) | ~7B | LLaVA-OneVision backbone with SigLIP vision encoder |
40
-
41
- All checkpoints are stored in `safetensors` format with `bfloat16` precision.
42
-
43
- ## Supported Tasks
44
-
45
- These models are jointly trained on general VQA and three categories of our curated VQA tasks:
46
-
47
- - **Generation**: Predicting intermediate representations such as trajectory waypoints, gripper bounding boxes, contact points/boxes, object bounding boxes (current & final), etc.
48
- - **Understanding**: Multiple-choice visual reasoning about contact states, grasp poses, object grounding, trajectory selection, movement directions, etc.
49
- - **Task Planning**: High-level task planning including next-step prediction, action primitive recognition, success determination, etc.
50
-
51
- ## Usage
52
-
53
- ### Qwen2.5-VL Checkpoints
54
- For loading and inference with the Qwen2.5-VL checkpoint, please refer to the [RoboInterVLM-QwenVL](https://github.com/InternRobotics/RoboInter/tree/main/RoboInterVLM/RoboInterVLM-QwenVL) codebase. We provide a fast loading example below:
55
-
56
- ```python
57
- from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
58
-
59
- model_path = "InternRobotics/RoboInterVLM_qwenvl25_3b" # or RoboInterVLM_qwenvl25_7b
60
- model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
61
- model_path, torch_dtype="auto", device_map="auto"
62
- )
63
- processor = AutoProcessor.from_pretrained(model_path)
64
- ```
65
-
66
- ### LLaVA-OneVision Checkpoint
67
-
68
- For loading and inference with the LLaVA-OneVision checkpoint, please refer to the [RoboInterVLM-LLaVAOV](https://github.com/InternRobotics/RoboInter/tree/main/RoboInterVLM/RoboInterVLM-LLaVAOV) codebase, as it requires custom model classes.
69
-
70
- ### Training & Evaluation
71
-
72
- For full training and evaluation pipelines, please refer to:
73
-
74
- - **Qwen2.5-VL models**: [RoboInterVLM-QwenVL](https://github.com/InternRobotics/RoboInter/tree/main/RoboInterVLM/RoboInterVLM-QwenVL)
75
- - **LLaVA-OneVision model**: [RoboInterVLM-LLaVAOV](https://github.com/InternRobotics/RoboInter/tree/main/RoboInterVLM/RoboInterVLM-LLaVAOV)
76
- - **VQA Dataset**: [RoboInter-VQA](https://huggingface.co/datasets/InternRobotics/RoboInter-VQA)
77
-
78
- ## Related Resources
79
-
80
- - **Project**: [RoboInter](https://github.com/InternRobotics/RoboInter)
81
- - **Annotation Data**: [RoboInter-Data](https://huggingface.co/datasets/InternRobotics/RoboInter-Data)
82
- - **VQA Dataset**: [RoboInter-VQA](https://huggingface.co/datasets/InternRobotics/RoboInter-VQA)
83
- ## License
84
-
85
- Please refer to the original licenses of [RoboInter](https://github.com/InternRobotics/RoboInter), [Qwen2.5-VL](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct), and [LLaVA-OneVision](https://huggingface.co/lmms-lab/llava-onevision-qwen2-7b-ov).