Prof. Kiyoharu Aizawa
aizawa@hal.t.u-tokyo.ac.jp

Computer Vision,
Image Processing,Multimedia

Hongo Campus,
Engineering Building #2
https://www.hal.t.u-tokyo.ac.jp

Image and Multimedia Processing
Image and multimedia technology is expanding to human centric computing. Specific issues and technical problems are diverse. The below arebrief description of our current research topics. We operate jointly with Yamasaki, Yamakata and Matsui laboratories. Questions are welcome.
Send them to aizawa@hal.t.u-tokyo.ac.jp. Our Web page https://www.hal.t.u-tokyo.ac.jp

1. Fundamentals of Recognition and Learning:
Open World
Deep Learning accurately works for closed dataset
containing large number of data per class. However, inreality, unknown classes and new classes with small
amount of data frequently appear. We are investigating
identification and recognition techniques for such open
world situation. The topics are methodology for noisy
training data, out-of-distribution detection, positiveunlabeled learning, open-set data learning, new category
discovery, uncertainty estimation etc.

Sim. Opt. for Learning Noisy Data

Out of Distribution Detection

2. 360̊ Image Processing・3D: Movie Map
We are investigating 360̊ image processing. Specifically,
we build “movie map” for walkers to explore in a city.
Using 360̊ street videos, we work on many different
research issues such as hyperlaspe 360̊video, 360̊image
object detection, accurate vSLAM, intersection detection,
depth from 360̊ image, RoI detection, route view
generation, 360̊ super-resolution, detection of possible
direction of travel, avatar in 360̊ video, real-world
Metaverse etc. We are prototyping a platform of virtual
exploration.

3. Multimedia Life Log: Food Computing

Route Synthesis of 360̊video

Real World Metaverse

We have been pioneering life logging technology. To
pursue specific purpose lifelogging, we focus on researchon capture and analysis of our daily food logs (FoodLog),
Using the app we developed, food records we captured
exceeds 10 million. We are investigating various
processing of FoodLog data, such as personalized foodrecognition, recipe and food record multimodal analysis,building a new tool for athletes and dietitians, predictionof healthy index, etc.

4. Manga & Comic Computing

New FoodLog Tool

Personalized Classifier using Deep Features

Manga, our unique culture, is our research target. We havebuilt a world largest scale Manga dataset, and investigate
fundamentals of image processing techniques such as
retrieval, segmentation, recognition, colorization, creatorstyle transfer, etc. We are also investigating persons’reading behaviors.

5. Deep Image Compression, Image Generation,Scene Text Recognition
We investigate new image compression techniques using
deep learning. Although image compression is a long
traditional area of signal processing, a lots of researchissues exist. We investigate a universal deep compressionthat adapts to variety of content outside the training
domain. As for image generation, we investigate aDiffusion Model based framework that allows easy
modification of the results. We have built a dataset ofmanga onomatopoeia for the most challenging problem
for scene text recognition.

MovieMap

Manga109 Dataset

Onomatopoeia
Dataset
Scene Text
Recognition

Font Generation froma Few Samples

Universal Deep Image
Compression