JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluation
Paper
•
2410.17250
•
Published
•
14
JMMMU
non-profit
AI & ML interests
None defined yet.
Recent Activity
View all activity
JMMMU's activity
yuexiang96
authored
5
papers
13 days ago
Long Context Alignment with Short Instructions and Synthesized Positions
Paper
•
2405.03939
•
Published
LIME: Less Is More for MLLM Evaluation
Paper
•
2409.06851
•
Published
Machine Unlearning of Pre-trained Large Language Models
Paper
•
2402.15159
•
Published
MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale
Paper
•
2412.05237
•
Published
•
44
yuexiang96
authored
a
paper
15 days ago
AtsuMiyai
updated
a
Space
about 2 months ago
yuexiang96
authored
a
paper
about 2 months ago
yuki-imajuku
authored
a
paper
about 2 months ago
yuki-imajuku
updated
a
dataset
about 2 months ago
ku21fan
authored
a
paper
about 2 months ago
shtapm
authored
a
paper
about 2 months ago
yuexiang96
posted
an
update
2 months ago
Post
3021
🌍 I’ve always had a dream of making AI accessible to everyone, regardless of location or language. However, current open MLLMs often respond in English, even to non-English queries!
🚀 Introducing Pangea: A Fully Open Multilingual Multimodal LLM supporting 39 languages! 🌐✨
https://neulab.github.io/Pangea/
https://arxiv.org/pdf/2410.16153
The Pangea family includes three major components:
🔥 Pangea-7B: A state-of-the-art multilingual multimodal LLM capable of 39 languages! Not only does it excel in multilingual scenarios, but it also matches or surpasses English-centric models like Llama 3.2, Molmo, and LlavaOneVision in English performance.
📝 PangeaIns: A 6M multilingual multimodal instruction tuning dataset across 39 languages. 🗂️ With 40% English instructions and 60% multilingual instructions, it spans various domains, including 1M culturally-relevant images sourced from LAION-Multi. 🎨
🏆 PangeaBench: A comprehensive evaluation benchmark featuring 14 datasets in 47 languages. Evaluation can be tricky, so we carefully curated existing benchmarks and introduced two new datasets: xChatBench (human-annotated wild queries with fine-grained evaluation criteria) and xMMMU (a meticulously machine-translated version of MMMU).
Check out more details: https://x.com/xiangyue96/status/1848753709787795679
🚀 Introducing Pangea: A Fully Open Multilingual Multimodal LLM supporting 39 languages! 🌐✨
https://neulab.github.io/Pangea/
https://arxiv.org/pdf/2410.16153
The Pangea family includes three major components:
🔥 Pangea-7B: A state-of-the-art multilingual multimodal LLM capable of 39 languages! Not only does it excel in multilingual scenarios, but it also matches or surpasses English-centric models like Llama 3.2, Molmo, and LlavaOneVision in English performance.
📝 PangeaIns: A 6M multilingual multimodal instruction tuning dataset across 39 languages. 🗂️ With 40% English instructions and 60% multilingual instructions, it spans various domains, including 1M culturally-relevant images sourced from LAION-Multi. 🎨
🏆 PangeaBench: A comprehensive evaluation benchmark featuring 14 datasets in 47 languages. Evaluation can be tricky, so we carefully curated existing benchmarks and introduced two new datasets: xChatBench (human-annotated wild queries with fine-grained evaluation criteria) and xMMMU (a meticulously machine-translated version of MMMU).
Check out more details: https://x.com/xiangyue96/status/1848753709787795679
yuexiang96
authored
6
papers
2 months ago
Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages
Paper
•
2410.16153
•
Published
•
43
MixEval-X: Any-to-Any Evaluations from Real-World Data Mixtures
Paper
•
2410.13754
•
Published
•
74
KOR-Bench: Benchmarking Language Models on Knowledge-Orthogonal Reasoning Tasks
Paper
•
2410.06526
•
Published
Harnessing Webpage UIs for Text-Rich Visual Understanding
Paper
•
2410.13824
•
Published
•
29
Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQA
Paper
•
2405.20421
•
Published
MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Tasks
Paper
•
2410.10563
•
Published
•
38
yuexiang96
authored
a
paper
4 months ago