Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
@@ -54,10 +54,12 @@ This project introduces **General-Level** and **General-Bench**.
|
|
54 |
- [π Overall Leaderboard](#leaderboard)
|
55 |
- [π General-Level](#level)
|
56 |
- [π General-Bench](#bench)
|
|
|
57 |
|
58 |
---
|
59 |
|
60 |
-
|
|
|
61 |
|
62 |
<div align="center">
|
63 |
<img src='https://cdn-uploads.huggingface.co/production/uploads/647773a1168cb428e00e9a8f/s1Q7t6Nmtnmv3bSvkquT0.png' width=1200px>
|
@@ -66,8 +68,8 @@ This project introduces **General-Level** and **General-Bench**.
|
|
66 |
|
67 |
---
|
68 |
|
69 |
-
|
70 |
-
|
71 |
**A 5-scale level evaluation system with a new norm for assessing the multimodal generalists (multimodal LLMs/agents).
|
72 |
The core is the use of <b style="color:red">synergy</b> as the evaluative criterion, categorizing capabilities based on whether MLLMs preserve synergy across comprehension and generation, as well as across multimodal interactions.**
|
73 |
|
@@ -86,7 +88,8 @@ The core is the use of <b style="color:red">synergy</b> as the evaluative criter
|
|
86 |
|
87 |
---
|
88 |
|
89 |
-
|
|
|
90 |
|
91 |
**A companion massive multimodal benchmark dataset, encompasses a broader spectrum of skills, modalities, formats, and capabilities, including over 700 tasks and 325K instances.**
|
92 |
|
@@ -97,7 +100,7 @@ We set two dataset types according to the use purpose:
|
|
97 |
|
98 |
|
99 |
<div align="center">
|
100 |
-
<img src='https://cdn-uploads.huggingface.co/production/uploads/647773a1168cb428e00e9a8f/d4TIWw3rlWuxpBCEpHYJB.jpeg'>
|
101 |
</div>
|
102 |
|
103 |
|
@@ -105,6 +108,27 @@ We set two dataset types according to the use purpose:
|
|
105 |
|
106 |
|
107 |
<div align="center">
|
108 |
-
<img src='https://cdn-uploads.huggingface.co/production/uploads/647773a1168cb428e00e9a8f/qkD43ne58w31Z7jpkTKjr.jpeg'>
|
109 |
</div>
|
110 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
54 |
- [π Overall Leaderboard](#leaderboard)
|
55 |
- [π General-Level](#level)
|
56 |
- [π General-Bench](#bench)
|
57 |
+
- [π Citation](#cite)
|
58 |
|
59 |
---
|
60 |
|
61 |
+
<h1 style="font-weight: bold; text-decoration: none;"> πππ Overall Leaderboard <a name="leaderboard" /> </a> </h1>
|
62 |
+
|
63 |
|
64 |
<div align="center">
|
65 |
<img src='https://cdn-uploads.huggingface.co/production/uploads/647773a1168cb428e00e9a8f/s1Q7t6Nmtnmv3bSvkquT0.png' width=1200px>
|
|
|
68 |
|
69 |
---
|
70 |
|
71 |
+
<h1 style="font-weight: bold; text-decoration: none;"> πππ General-Level <a name="level" /> </a> </h1>
|
72 |
+
|
73 |
**A 5-scale level evaluation system with a new norm for assessing the multimodal generalists (multimodal LLMs/agents).
|
74 |
The core is the use of <b style="color:red">synergy</b> as the evaluative criterion, categorizing capabilities based on whether MLLMs preserve synergy across comprehension and generation, as well as across multimodal interactions.**
|
75 |
|
|
|
88 |
|
89 |
---
|
90 |
|
91 |
+
<h1 style="font-weight: bold; text-decoration: none;"> πππ General-Bench <a name="bench" /> </a> </h1>
|
92 |
+
|
93 |
|
94 |
**A companion massive multimodal benchmark dataset, encompasses a broader spectrum of skills, modalities, formats, and capabilities, including over 700 tasks and 325K instances.**
|
95 |
|
|
|
100 |
|
101 |
|
102 |
<div align="center">
|
103 |
+
<img src='https://cdn-uploads.huggingface.co/production/uploads/647773a1168cb428e00e9a8f/d4TIWw3rlWuxpBCEpHYJB.jpeg' width=1000px>
|
104 |
</div>
|
105 |
|
106 |
|
|
|
108 |
|
109 |
|
110 |
<div align="center">
|
111 |
+
<img src='https://cdn-uploads.huggingface.co/production/uploads/647773a1168cb428e00e9a8f/qkD43ne58w31Z7jpkTKjr.jpeg' width=900px>
|
112 |
</div>
|
113 |
|
114 |
+
|
115 |
+
|
116 |
+
---
|
117 |
+
|
118 |
+
<h1 style="font-weight: bold; text-decoration: none;"> πππ Citation <a name="cite" /> </a> </h1>
|
119 |
+
|
120 |
+
If you find this project useful to your research, please kindly cite our paper:
|
121 |
+
|
122 |
+
```bibtex
|
123 |
+
@articles{fei2025pathmultimodalgeneralistgenerallevel,
|
124 |
+
title={On Path to Multimodal Generalist: General-Level and General-Bench},
|
125 |
+
author={Hao Fei and Yuan Zhou and Juncheng Li and Xiangtai Li and Qingshan Xu and Bobo Li and Shengqiong Wu and Yaoting Wang and Junbao Zhou and Jiahao Meng and Qingyu Shi and Zhiyuan Zhou and Liangtao Shi and Minghe Gao and Daoan Zhang and Zhiqi Ge and Weiming Wu and Siliang Tang and Kaihang Pan and Yaobo Ye and Haobo Yuan and Tao Zhang and Tianjie Ju and Zixiang Meng and Shilin Xu and Liyu Jia and Wentao Hu and Meng Luo and Jiebo Luo and Tat-Seng Chua and Shuicheng Yan and Hanwang Zhang},
|
126 |
+
eprint={2505.04620},
|
127 |
+
archivePrefix={arXiv},
|
128 |
+
primaryClass={cs.CV}
|
129 |
+
url={https://arxiv.org/abs/2505.04620},
|
130 |
+
}
|
131 |
+
|
132 |
+
```
|
133 |
+
|
134 |
+
|