CVPR 2024 Workshop on

Representation Learningwith Very Limited Images
Zero-shot, Unsupervised, and Synthetic Learning in the Era of Big Models

LIMIT Organization TeamOrganizers in CVPR 2024 LIMIT Workshop

1Program committee
Akira
Alina E
AnestisAthanasios
AtsushiAtsushiBalu
Chia-Hsien
Daiki

Sakai
Marcu
Kastellos
PsaltisHashimoto
Kanehira
Adsumilli
Shih
Ikami

Debaditya

RoyDragosEmil
Fabio
Hankook
HannahHoàng-Ân
Hojin
Hyung-Il
Ikuro
Jiadong
Jingjing
Jose A

CosteaSlusanschi
Ferreira
LeePinsonLêJang
Kim
SatoChenChen
Sosa MartinezFujitsu Laboratories Ltd.
University Politehnica of Bucharest
ITI/CERTHITI-CERTH, University of West Attica
OMRON SINIC X Corp.
MicrosoftYouTube/Google
University of Illinois at Urbana-Champaign
Tokyo University of Agriculture and Technology
Agency for Science, Technology and Research, A*STAR,
Singapore
University "Politehnica" of Bucharest
University "Politehnica" of Bucharest
University of Freiburg
Sungkyunkwan University (SKKU)
Eindhoven University of Technology
IRISA, University of South Brittany
Korea University
ETRI
Tokyo Institute of Technology / Denso IT Laboratory
sensetime
Fudan University
University of Luxembourg

Many thanks!
2Program committee
KenshoHara

Kota
Kuniaki
Kyungjune
LileMarek
Matthieu
Mayu
Mihai Cristian
Minho
MoyuruNaoshi
Naoya
NikolasNishant
Pavan C
Pengwan
QianruRenaudRyosukeMatsui
Saito
Baek
Cai
Wodzinski
ArmandoOtaniPîrvuPark
YamadaKanekoChiba
EbertRai
Madhusudana
Yang
Sun
Vandeghen
FurutaNational Institute of Advanced Industrial Science andTechnology (AIST)
Nagoya University
OMRON SINIC X
TwelveLabs
Institute for Infocomm Research
AGH UST
Naver Labs
CyberAgent
University "Politehnica" of Bucharest
ETRI
Fujitsu Limited
Tokyo Denki University
Tohoku University
HS Mannheim
Stanford University
Samsung Research America
University of Amsterdam
National University of Singapore
University of Liège
The University of Tokyo

Many thanks!
3

Program committee
SaidSangbeom
Sanjana

LadjalLim
Srivastava

TakumaYagi

Tommi
TomoyukiVikashXiang
Youssef
Youwei
YusukeZiad

KerolaSuzukiSehwagWang
DawoudLiang
MukutaAl-HalahTélécom Paris
Korea University
Stanford University
National Institute of Advanced Industrial Science andTechnology (AIST)
Preferred Networks, Inc.
CyberAgent, Inc.
Sony AI
SenseTime Research
Friedrich-Alexander-Universität Erlangen-Nürnberg
UC San Diego
The University of Tokyo
University of Utah

Many thanks!
4

Today’s schedule5

Paper statistics❏ 15 valid submissions❏ 10 accepts (66.6% acceptance rate)❏ 4 orals❏ 15 posters (including +5 invited posters)

6Scaling for models and datasets
ModelsVision Transformer (ViT)ViT Models
AlexNet [Krizhevsky+, NIPS12]

VGGNet [Simonyan+, ICLR15]

GoogLeNet [Szegedy+, CVPR15]

ResNet [He+, CVPR16]

Datasets[Dosovitskiy+, ICLR21][Zhai+, CVPR22] [Dehghani+, ICML23]

JFT-300M/3B/4B[Sun+, ICCV17][Zhai+, CVPR22][Dehghani+, ICML23]

Params

22B

22B

Giant

2B

Huge

632M

Large

307M

Base86M

Small

22M

Tiny

5M

Large size is‘JUSTICE’

http://www.image-net.org/[Deng+, CVPR09]

https://laion.ai/

7

Scaling laws in ‘vision’ transformers
Getting larger, getting better!

Can thegame on ly b e p
Ho w d olayed atacademspecific
ic condulabs?
ct a research?

[Zhai+, CVPR22]

8

Scaling laws in ‘vision’ transformers
Getting larger, getting better!

Can thegame on ly b e p
Ho w a cl
ayed ata d e m ic
specific
conductlabs?
a research?

A larger-curated dataset tends to occur labor/ethical concerns
less data, less computational resources, higher performance is preferable!

[Zhai+, CVPR22]

9

Scaling laws in ‘vision’ transformers[Dosovitskiy+, ICLR21]+
JFT-3B

3,000,000,000 real images
are required for ViT pre-training[Zhai+, CVPR22]

10

Breaking scaling laws in ‘vision’ transformers?

OFDB
1,000 synthetic images
are enough for ViT pre-training[Nakamura+, ICCV23]

1-video
SSL

D
eff esireici
en dcy?

1 real videois enough for ViT pre-training[Venkataramanan+, ICLR24]
Best Paper Honorable Mention[Zhai+, CVPR22]

11

More wonderful solutions for ‘LIMIT’ community!
Our community has awesome solutions

Sim-to-Real[Shrivastava+, CVPR17]

Domain Randomization[Sundermeyer+, ECCV18]

Learning to See by Looking atNoiseRepresentation Learning fromsynthetic images[Baradad+, NeurIPS21][Tian+, CVPR24]

Keyword: Synthetic Data,
Adversarial Learning, Data
Augmentation, Domain
Randomization, Formula-driven
Supervised Learning, VisualPrompt Tuning
Formula-driven Learning[Kataoka+, IJCV22][Anderson+, WACV22]

1-image SSL[Asano+, ICLR20, ICLR23]

Visual Prompt Tuning[Shtedritski+, ICCV23]
12

Invited talk 1: Phillip Isola (MIT)Title: N=0: Learning Vision with Zero Visual Data
Phillip Isola is the Class of 1948 Career Development associate professor
in EECS at MIT. He studies computer vision, machine learning, robotics,and AI. He completed his Ph.D. in Brain & Cognitive Sciences at MIT, andhas since spent time at UC Berkeley, OpenAI, and Google Research. His
work has particularly impacted generative AI and self-supervised
representation learning. Dr. Isola's research has been recognized by aGoogle Faculty Research Award, a PAMI Young Researcher Award, aSamsung AI Researcher of the Year Award, a Packard Fellowship, and aSloan Fellowship. His teaching has been recognized by the Ruth and Joel
Spira Award for Distinguished Teaching. His current research focuses ontrying to scientifically understand human-like intelligence. (Refer from:http://web.mit.edu/phillipi/www/bio.html)

13

Invited talk 2: Zeynep Akata (TUM / Helmholtz Munich)Title: Learning with Small Number of Images inMultimodal Large Language Models
Zeynep Akata is a Liesel Beckmann Distinguished professor of Computer Science at
Technical University of Munich and the director of the Institute for ExplainableMachine Learning at Helmholtz Munich. After completing her PhD at the INRIA
Rhone Alpes with Prof Cordelia Schmid (2014), she worked as a post-doctoral
researcher at the Max Planck Institute for Informatics with Prof Bernt Schiele (201417) and at University of California Berkeley with Prof Trevor Darrell (2016-17) and as
an assistant professor at the University of Amsterdam with Prof Max Welling (201719). Before moving to Munich in 2024, she was a professor of computer science(W3) within the Cluster of Excellence Machine Learning at the University ofTübingen. She received a Lise-Meitner Award for Excellent Women in Computer
Science from Max Planck Society in 2014, a young scientist honour from theWerner-von-Siemens-Ring foundation in 2019, an ERC-2019 Starting Grant from theEuropean Commission, The DAGM German Pattern Recognition Award in 2021,
The ECVA Young Researcher Award in 2022 and the Alfried Krupp Award in 2023.
Her research interests include multimodal learning and explainable AI. (Refer from:https://www.eml-unitue.de/people/zeynep-akata)14