Replacing Human Vision
with Machine
Ikuro Sato

Denso IT Laboratory, Inc. & Tokyo Institute of Technology, Japan
11/1/2022

Who is Ikuro Sato?
2005.08

Ph.D. received from University of Maryland, USA

2005.10~

Postdoc at Lawrence Berkeley National Laboratory, USA

2008.03~

Researcher at Denso IT Laboratory2020.04~

Specially Appointed Associate Professor at Tokyo Tech.
Image Recognition

Machine Learning

Field of study

2022/11/1

Ikuro Sato

Replacing HumanVision with Machines

Tokyo Tech.

2 / 15DENSO IT LAB Recognition and Learning Algorithm
Collaborative Research Chair

https://d-itlab.c.titech.ac.jp/

To conduct researches on challenging and essential problems
to bring technological breakthroughs to replace human vision with machines.
(From left)• Rio Yokota• Shunsuke Ono• Ikuro Sato
• Koichi Shinoda• Nakamasa Inoue• Akihiro Nomura• Masayuki Tanaka• Rei Kawakami• Asako Kanezaki
2022/11/1

HPC

Optimization

Multi-Media Processing

Robotics

Image Recognition
Machine Learning

Ikuro Sato

Replacing HumanVision with Machines

Tokyo Tech.

3 / 15

The World of Driving
SymbolText

Weather

RarityO. Amundarain+, Sensors2021

Z. Zou+, 2019https://trafficnews.jp

https://sirabee.com/https://kuruma-news.jp

S. Vachmanus+, SICE2020.

https://trafficnews.jp/post/54477

https://www.bengo4.com/c_23/n_9085/

Z. Zou+, 20192022/11/1

Ikuro Sato

Replacing HumanVision with Machines

Tokyo Tech.

4 / 15

The World of Driving
SymbolText

Weather

RarityO. Amundarain+, Sensors2021

Z. Zou+, 2019https://trafficnews.jp

https://sirabee.com/

https://kuruma-news.jp

S. Vachmanus+, SICE2020.

https://trafficnews.jp/post/54477

https://www.bengo4.com/c_23/n_9085/

Z. Zou+, 2019

Need to replace almost all aspects of human vision
2022/11/1

Ikuro Sato

Replacing HumanVision with Machines

Tokyo Tech.

5 / 15

Great Progress by Deep Learning
ImageNet-1K Performance vs. Yearhttps://paperswithcode.com

Florence
99.02%

Human
AlexNet84.6%

14% improvement in 10 years!

2022/11/1

Ikuro Sato

Replacing HumanVision with Machines

Tokyo Tech.

6 / 15

Great Progress by Deep Learning
ImageNet-1K Performance vs. Yearhttps://paperswithcode.com

Florence
99.02%

Human
AlexNet84.6%

14% improvement in 10 years!

Will the replacement happen along this direction?
2022/11/1

Ikuro Sato

Replacing HumanVision with Machines

Tokyo Tech.

7 / 15Tasks that brains are far better
Anomaly Detection

Domain Gap

MVTec dataset

PACS dataset, image from J. Xu, et al., 2019.

Performance gap between machines and brains
2022/11/1

Ikuro Sato

Replacing HumanVision with Machines

Tokyo Tech.

8 / 15

Differences between Machine and Brain
Inference

Learning

discrete
path

Data

2022/11/1

Task

Ikuro Sato

Replacing HumanVision with Machines

Tokyo Tech.

9 / 15

Research Theme
Replacement of Human Vision with Machines

Machine-Learning Modeling
Discrete
MultiPhysicalMemory Represent
Functionality
Consistency(Local Update)-ation2022/11/1

Ikuro Sato

GenericFeature

MultiModality

Replacing HumanVision with Machines

Tokyo Tech.

Self-Super Metricvision
Learning
10 / 15

Bayesianetc.
NNResearch Theme
Collaborators

BackgroundMemories expressed by fixed points ofReplacementof Human Vision with Machines
a closed network (Krotov & Hopfield, 2021)

T. OtaK. Ishikawa

N. Inoue R. Kawakami

M. TanakaSpecific formulation  Self-Attention Mechanism

Contribution

0.21

Ours

0.18

Embedded forgetting functionality of memories
during training improves recognition performance.

I. SatoDiscrete
MultiPhysicalMemory Represent
Functionality
Consistency(Local Update)-ation2022/11/1

Baseline

Ikuro Sato

GenericFeature

MultiModality

Replacing HumanVision with Machines

Tokyo Tech.

Error rate

Immune repertoire classification accuracy

Self-Super Metricvision
Learning
11 / 15

Bayesianetc.
NNResearch Theme
Collaborators

R. Yamadaalmost discrete

Background

Discrete representations obtained by breaking
layer-wise co-adaptation (I. Sato et al., 2019)
Contribution

Showed such an optimizer flattens loss landscape.
N. Inoue R. Kawakami

M. Tanaka

I. Sato••

Theoretical derivation
Demonstration of high recognition accuracy

Received MIRU Nagao Prize (2022).
Accepted to ICML2022.

Discrete
MultiPhysicalMemory Represent
Functionality
Consistency(Local Update)-ation2022/11/1

Ikuro Sato

GenericFeature

2D distribution of representations

Baseline

Ours
Learning Curve

MultiModality

Replacing HumanVision with Machines

Tokyo Tech.

Self-Super Metricvision
Learning
12 / 15

Bayesianetc.
NNResearch Theme
Collaborators

R. Isobe

R. Kawakami

I. SatoTask

To predict viewing direction to an object
Viewing direction prediction task

Contribution

Found that auxiliary image generation
task can enhance physical consistency.
•Demonstrated outperformanceof existing method

Discrete
MultiPhysical
Memory Represent
Functionality
Consistency(Local Update)-ation2022/11/1

Ikuro Sato

GenericFeature

Proposed modelMultiModality

Replacing HumanVision with Machines

Tokyo Tech.

Self-Super Metricvision
Learning
13 / 15

Bayesianetc.
NNResearch Theme
Collaborators

T. Shibuya R. Kawakami

N. Inoue

I. SatoBackground

Target Propagation has been proposed as a
biologically-plausible deep net training algorithm.
• Local learning rules
Contribution
Proposed a new learning rule, much simpler
than existing approaches.
• Robust to different hyperparameter settings

Received MIRU Interactive Presentation Award (2022).

Discrete
MultiPhysical
Memory Represent
Functionality
Consistency(Local Update)-ation2022/11/1

Ikuro Sato

GenericFeature

MultiModality

Replacing HumanVision with Machines

Tokyo Tech.

Proposed model

Self-Super Metricvision
Learning
14 / 15

Bayesianetc.
NNSummary• We, RLA Lab., are one of the largest research groups on recognition and learning in Japan.
• We aim to develop algorithms for replacing human vision with machines.

2022/11/1

Ikuro Sato

Replacing HumanVision with Machines

Tokyo Tech.

15 / 15