DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curation
Paper
•
2410.18666
•
Published
•
19
Quanzeng You
Ye27
AI & ML interests
None yet
Recent Activity
upvoted
a
paper
about 2 months ago
DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe
Dataset Curation
authored
a paper
about 2 months ago
DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe
Dataset Curation
Organizations
Ye27's activity
upvoted
a
paper
about 2 months ago
authored
a
paper
about 2 months ago
authored
2
papers
3 months ago
upvoted
a
paper
3 months ago
authored
a
paper
4 months ago
upvoted
a
paper
4 months ago
upvoted
an
article
5 months ago
Article
Preference Optimization for Vision Language Models
•
53
Large value difference when comparing hidden_states with flash attention ON and OFF
#42 opened 8 months ago
by
Ye27
authored
a
paper
9 months ago
upvoted
a
paper
9 months ago
authored
a
paper
10 months ago
reacted to
xiaotianhan's
post with 🤗
11 months ago
Post
Thrilled to share some of our recent work in the field of Multimodal Large Language Models (MLLMs).
1️⃣ A Survey on Multimodal Reasoning 📚
Are you curious about the reasoning abilities of MLLMs? In our latest survey, we delve into the world of multimodal reasoning. We comprehensively review existing evaluation protocols, categorize the frontiers of MLLMs, explore recent trends in their applications for reasoning-intensive tasks, and discuss current practices and future directions. For an in-depth exploration, check out our paper: Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning (2401.06805)
2️⃣ Advancing Flamingo with InfiMM 🔥
Building upon the foundation of Flamingo, we introduce the InfiMM model series. InfiMM is a reproduction of Flamingo, enhanced with stronger Large Language Models (LLMs) such as LLaMA2-13B, Vicuna-13B, and Zephyr7B. We've meticulously filtered pre-training data and fine-tuned instructions, resulting in superior performance on recent benchmarks like MMMU, InfiMM-Eval, MM-Vet, and more. Explore the power of InfiMM on Huggingface: Infi-MM/infimm-zephyr
3️⃣ Exploring Multimodal Instruction Fine-tuning 🖼️
Visual Instruction Fine-tuning (IFT) is crucial for aligning MLLMs' output with user intentions. Our research identified challenges with models trained on the LLaVA-mix-665k dataset, particularly in multi-round dialog settings. To address this, we've created a new IFT dataset with high-quality, diverse instruction annotations and images sourced exclusively from the COCO dataset. Our experiments demonstrate that when fine-tuned with this dataset, MLLMs excel in open-ended evaluation benchmarks for both single-round and multi-round dialog settings. Dive into the details in our paper: COCO is "ALL'' You Need for Visual Instruction Fine-tuning (2401.08968)
Stay tuned for more exciting developments.
Special thanks to all our collaborators: @Ye27 @wwyssh @Yongfei @Yi-Qi638 @xudonglin @KhalilMrini @lllliuhhhhggg @Borise @Hongxia
1️⃣ A Survey on Multimodal Reasoning 📚
Are you curious about the reasoning abilities of MLLMs? In our latest survey, we delve into the world of multimodal reasoning. We comprehensively review existing evaluation protocols, categorize the frontiers of MLLMs, explore recent trends in their applications for reasoning-intensive tasks, and discuss current practices and future directions. For an in-depth exploration, check out our paper: Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning (2401.06805)
2️⃣ Advancing Flamingo with InfiMM 🔥
Building upon the foundation of Flamingo, we introduce the InfiMM model series. InfiMM is a reproduction of Flamingo, enhanced with stronger Large Language Models (LLMs) such as LLaMA2-13B, Vicuna-13B, and Zephyr7B. We've meticulously filtered pre-training data and fine-tuned instructions, resulting in superior performance on recent benchmarks like MMMU, InfiMM-Eval, MM-Vet, and more. Explore the power of InfiMM on Huggingface: Infi-MM/infimm-zephyr
3️⃣ Exploring Multimodal Instruction Fine-tuning 🖼️
Visual Instruction Fine-tuning (IFT) is crucial for aligning MLLMs' output with user intentions. Our research identified challenges with models trained on the LLaVA-mix-665k dataset, particularly in multi-round dialog settings. To address this, we've created a new IFT dataset with high-quality, diverse instruction annotations and images sourced exclusively from the COCO dataset. Our experiments demonstrate that when fine-tuned with this dataset, MLLMs excel in open-ended evaluation benchmarks for both single-round and multi-round dialog settings. Dive into the details in our paper: COCO is "ALL'' You Need for Visual Instruction Fine-tuning (2401.08968)
Stay tuned for more exciting developments.
Special thanks to all our collaborators: @Ye27 @wwyssh @Yongfei @Yi-Qi638 @xudonglin @KhalilMrini @lllliuhhhhggg @Borise @Hongxia
authored
6
papers
11 months ago
COCO is "ALL'' You Need for Visual Instruction Fine-tuning
Paper
•
2401.08968
•
Published
•
2
Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning
Paper
•
2401.06805
•
Published
•
2
Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark
Paper
•
1605.02677
•
Published
•
1
CORE-MM: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models
Paper
•
2311.11567
•
Published
•
8
Reason out Your Layout: Evoking the Layout Master from Large Language Models for Text-to-Image Synthesis
Paper
•
2311.17126
•
Published
•
1
Learning Stackable and Skippable LEGO Bricks for Efficient, Reconfigurable, and Variable-Resolution Diffusion Modeling
Paper
•
2310.06389
•
Published
•
1
upvoted
a
paper
about 1 year ago