Tokyo Institute of Technology
School of Computing
Department of Computer Science

Introduction of Kanezaki Laboratory
Assoc. Prof. Asako Kanezaki

Short bio.

◉ Mar 2008 Graduated from Department of Mechanical Information Engineering, Faculty of Engineering, The University of Tokyo◉ Mar 2010 Completed master's course, Graduate School of Information Science and Technology, The University of Tokyo
2010-2011 (a half year) visiting research at Technische Universität München◉ Mar 2013 Ph.D. (Information Science and Technology), The University of Tokyo,◉ Apr 2013 Full-time employee, Research and Development Center, Toshiba Corporation◉ Dec 2013 Assistant Professor, Graduate School of Information Science and Technology, The University of Tokyo
2015.8-9 visiting research at Microsoft Research Redmond◉ Apr 2016 Researcher -> Senior Researcher, Artificial Intelligence Research Center, National Institute of Advanced Industrial
Science and Technology (AIST)◉ Apr 2020 Associate Professor, School of Computing, Tokyo Institute of Technology
2Computer
VisionMachineLearning

Robotics

ResearchMaster – Ph.D. student

AIST (last 4 years)

3D features & Object recognition

3D objectrecognition

Deep Learning
Unsupervised
Image
Segmentation

Robot Navigation

3

Computer
VisionMachineLearning

Robotics

ResearchMaster – Ph.D. student

AIST (last 4 years)

3D features & Object recognition

3D objectrecognition

Deep LearningUnsupervised
ImageSegmentation

Robot Navigation

4

CVPR 2018, accepted to TPAMI

RotationNet: Joint Object Categorization andPose Estimation Using Multiviews from
Unsupervised Viewpoints

Asako Kanezaki, Yasuyuki Matsushita, and Yoshifumi Nishida.
National Institute of Advanced Industrial Science and Technology (AIST)3D object recognition?
◉ Takes 3D data as input and outputs the probability ofobject categories (object classification)systemStargazy pie

Cf.) object detection, object retrieval, parts segmentation

3D object recognition?

Approaches:
RGBD based

Point Cloud based

Voxel basedMulti-view based

Here7

RotationNet is the best!

◉ State-of-the-art scores on the ModelNet
dataset http://modelnet.cs.princeton.edu/- ModelNet40: 40 categories- ModelNet10: 10 categories- 2020/4/25 leaderboard ⇒ModelNet40

ModelNet10First： RotationNet

First： RotationNet

Multi-view based

Multi-view based

Second： iMHL
Multi-view based

Second： SPNet
Multi-view based8Motivation◉ Object recognition by robots“Move and see” to achieve better performance
No, it’s a cup.

A pan?

【a single image input】
Not always captured from a best viewto recognize an object.
9

RotationNet (Overview)◉ Takes multi-view images as input and predicts object category and pose
Applicable to real-time applications

Improved accuracy by rotating in a directionthat is easy to recognize

10Videohttps://kanezaki.github.io/rotationnet/

11

Competition & Exhibition◉ SHREC2017 - 3D Shape Retrieval
Contest 2017
Participated in 2 out of 7 tracks and won the firstprize for both of them!

[Subject 1]

Competition to find a model similar to a query
shape from a large three-dimensional CAD
object database[Subject 2]Competition to find a CAD model similar to a real
query shape data captured by an RGB-D sensor

Demonstration of Hololens application at
CEATEC2017: 152,066 participants
12See my Japanese website: https://kanezaki.github.io/index_jp.html

Invited Talks on 3D Recognition◉ 「機械学習を用いた3次元データ認識について」，第3回情報処理学会セミナー AIと歩む未来(2)：
画像・映像処理の最前線，千代田区，2019.9◉ 「深層学習を用いた三次元物体認識」，確率場と深層学習に関する第2回CRESTシンポジウム，
早稲田大学，2018.10◉ 「3次元物体認識技術」，第112回ロボット工学セミナー ロボットのための画像処理技術，2018.5◉ 「色距離センサを用いた点群処理と三次元物体認識に関する研究紹介」，画像応用技術専門委員会2016年度第３回研究会，2016.9◉ 「Kinect等の色距離センサを用いた点群処理と3D物体認識―ベーシックな手法と最新動向・ソフトウェアの紹介―」, 第22回 画像センシングシンポジウム（SSII2016）チュートリアル講演, みなとみらい, 2016.6◉ 「RGBD画像処理と三次元物体認識の研究紹介」, 公益社団法人 精密工学会 大規模環境の３次元計測と認識・モデル化技術専門委員会 第20回定例研究会, 本郷, 2015.12◉ 「三次元画像からの物体検出と三次元特徴量」, 第9回 次世代コンピュータ支援診断ソフトウェア
臨床使用・評価プラットフォーム研究会，本郷，2015.6

etc.

13

ResearchMaster – Ph.D. student

AIST (last 4 years)

3D features & Object recognition

3D objectrecognition

Deep Learning
Unsupervised
ImageSegmentation

Robot Navigation14https://github.com/kanezaki/pytorch-unsupervised-segmentation

Unsupervised Image Segmentation using Deep Learning※Each area is shown in the same random color.
Asako Kanezaki. “Unsupervised Image Segmentation by Backpropagation.” IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.1543-1547, 2018.
15

https://github.com/kanezaki/pytorch-unsupervised-segmentation

Unsupervised Image Segmentation using Deep Learning◉ End-to-end feature learning and differentiable clustering

Asako Kanezaki. “Unsupervised Image Segmentation by Backpropagation.” IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.1543-1547, 2018.
16

ResearchMaster – Ph.D. student

AIST (last 4 years)

3D features & Object recognition

3D objectrecognition

Deep Learning
UnsupervisedImageSegmentation

Robot Navigation17

Robot navigation using deep learning

Video
https://kanezaki.github.io/goselo/A. Kanezaki+, ”GOSELO: Goal-Directed Obstacle and Self-Location Map for Robot Navigation using Reactive NeuralNetworks.” IEEE Robotics and Automation Letters (RA-L), Vol.3, Issue 2, pp.696-703, 2018. (presented in ICRA’18)

18

Robot navigation using deep learning
Key Idea: By creating a goal-directed map representation, we can learn the relationshipbetween visual patterns of the surrounding environment and movement patterns
without being constrained by the shape of a particular environment!

19

Follow-up studies（1/2）◉ Yoko Sasaki, Syusuke Matsuo, Asako Kanezaki, and Hiroshi Takemura. “A3C BasedMotion Learning for an Autonomous Mobile Robot in Crowds.” IEEE International
Conference on System Man and Cybernetics (SMC2019), pp.1046-1052, 2019.

◉ 佐々木洋子，松尾修佑，金崎朝子，竹村裕. 歩行者観測履歴を用いた深層強化学習による車輪ロボットの雑踏切り抜け動作生成. 日本機械学会ロボティクス・メカトロニクス
講演会, 2019.

◉ 渋谷 薫, 金崎 朝子, 大西 正輝.
深層学習による画像識別問
題に帰着した人の流れのシミュレーション. Meeting onImage Recognition andUnderstandings (MIRU),
2018.
Learned policy simulation

20

Follow-up studies（2/2）
Kei Ota, Yoko Sasaki, Devesh K. Jha, Yusuke Yoshiyasu, Asako Kanezaki. “Efficient
Exploration in Constrained Environments with Goal-Oriented Reference Path”, arXiv, 2020.

21

Student internships at AIST

◉ Robot navigation by reinforcement learning

理科大

Yoko Sasaki, Syusuke Matsuo, Asako Kanezaki, and Hiroshi Takemura. “A3C Based Motion Learning for an
Autonomous Mobile Robot in Crowds.” IEEE International Conference on System Man and Cybernetics(SMC2019), pp.1046-1052, 2019.

◉ Product recognition using deep learning

首都大加納諒也，和田一義，金崎朝子，冨沢哲雄，谷川民生. 深層学習による物体認識のためのマーカの検討 ～第一報：AlexNetによる認識率の比較～. 日本機械学会ロボティクス・メカトロニクス講演会, 2019.

◉ Deep Multi-modal learning

阪大

Ryohei Kuga, Asako Kanezaki, Masaki Samejima, Yusuke Sugano,
and Yasuyuki Matsushita. “Multi-modal U-Nets for Multi-task
Scene Understanding.” IEEE ICCV Workshop on Multi-Sensor
Fusion for Dynamic Scene Understanding, pp.403-411, 2017.

◉ Unsupervised image segmentation by deep learning◉ Real-time object detection by RotationNet

東工大
東工大

22

Research plan

IoP (Internet of Perception)?

◉ A robot system for active information search and database construction

23

Research topics (examples)◉ Action planning for autonomous mobile robots using reinforcement learning◉ Automatic generation of daily life search engines
Machine◉ Information collection planning for interactive robots
Learning
Computer
Vision◉ Robot intelligence with superhuman recognition technology
Robotics◉ Global and local action planning for robots
HumanComputer
Interaction
Map, GPSMulticameras

Multi-wavelength
sensing
Database

Construction of robot intelligence withsuperhuman recognition technology

Global and local movements(Manipulation, etc.)24Research topics (examples)◉ Action planning for autonomous mobile robots using reinforcement learning◉ Automatic generation of daily life search engines
Machine◉ Information collection planning for interactive robots
Learning
ComputerVision◉ Robot intelligence with superhuman recognition technology
Robotics◉ Global and local action planning for robots
HumanComputer
Interaction
Map, GPSMulticameras

multi-wavelength
sensing
Database

Construction of robot intelligence withsuperhuman recognition technology

Global and local movements(Manipulation, etc.)25

Why Kanezaki Laboratory? 😉

◉ New lab. (You will be first members!)◉ Can try new things as you wish.
◉ The infrastructure will be in place.
○○○

Use of computer clusters such as TSUBAME, AIST-ABCI
Managing source code and tips on GitHub
Sharing of library know-how such as deep learning androbot simulationHere!

◉ Ookayama Campus○

West Building 8 is conveniently located.

26

Questions?
✉ kanezaki [at] c.titech.ac.jp@kanejaki

27