Feel free to send us anemail for more details.

Okatani-Suganuma Lab.
Robotics Course

Graduate School of Information Sciences, Department of System Information Sciences

Computer vision for AI
We study computer vision for artificial intelligence.
Toward the ultimate goal of developing a computer
system that can see the world as humans do, we are
studying methods for understanding images andthose for obtaining 3D information about a scenefrom its images. We are currently conducting a wide
range of research projects, e.g., from developmentof theory of deep learning to autonomous flight ofdrones for inspecting bridges. At present, Prof.
Okatani concurrently serves as a team leader at
RIKEN center for AIP, a major center of AI research inJapan.

AI that can understand images

We are studying an AI that can interact with people using language
about the content of an image. This is the research of AI closest tohuman intelligence and will open up the frontiers of deep learning.

Medical applications
Object recognition

3D measurement

Machine vision

Recognition
Visual tracking

Measurement

Fundamental
technologies
Body measurement

HCIImage media
3D displays
View synthesis
Projectorapplications
Augmented reality
Video/game productionAnalyzing fashion by computer

nullshorts
shoes
purse
topnecklacehairskin

We are studying an AI that can learn to(a) Superpixels(b) Pose estimation(c) Predicted Clothing Parse(d) Pose rejudge whether
a given combination of fashion
Figure 2: Clothing parsing pipeline: (a) Parsing the image into Superpixels [1], (b) Original pose est
items is goodbad.mixtures of parts model [27]. (c) Precise clothing parse output by our proposed cloth
the artorflexible(note the accurate labeling of items as small as the wearer’s necklace, or as intricate as her open toed shestimate of pose using clothing estimates (note the improvement in her left arm prediction, compared toestimate down along the side of her body).

Vision for mobile robots

Recognizing
Shitsukan of objects
garment retrieval application (Fig 1).

limited situations [2]. Later work f
cal representations of clothing using
Our main contributions include:
Freifeld and Black [13] represented• A novel dataset for studying clothing parsing, consistmation from an underlying body coing of 158,235 fashion photos with associated text antraining examples using principal conotations, and web-based tools for labeling.
produce eigen-clothing. Most recent• An effective model to recognize and precisely parse
made to consider clothing items such
pictures of people into their constituent garments.
semantic attributes of a person, but on• Initial experiments on how clothing prediction might
ber of garments [4]. Different from t
improve state of the art models for pose estimation.
we consider the problem of estimatin• A prototype visual garment retrieval application thatcise region based labeling of a perso
can retrieve matches independent of pose.
images with a large number of potent
Of course, clothing estimation is a very challenging
problem. The number of garment types you might observe
Clothing items have also been us
in a day on the catwalk of a New York city street is enoridentity in surveillance scenarios [26mous. Add variations in pose, garment appearance, layimage collection of an event [11, 22,ering, and occlusion into the picture, and accurate clothpation [23], or for robot manipulatio
ing parsing becomes formidable. Therefore, we consider
approach could be useful in all of thea somewhat restricted domain, fashion photos from ChicPose Estimation: Pose estimation i
topia.com. These highly motivated users – fashionistas –studied enterprise. Some previous a
We are developing vision for mobile robots; the above shows someupload individual snapshots (often full body) of their outfits
pose estimation as a labelin
Most of what
we andperceivewithsome
ourinformationeyes cannot
besideredexpressedintothewebsite
usuallyproviderelated
most likely body parts to superpixels
results of estimation of 3D structure of a scene and the motion of atothegarments,
style,
oroccasion
fortheoutfit.
This
allowswords.
We
study
how
to
recognize
suchthings
withamachine.
regions[20]. Current approaches oft
robot from the camera mounted on it.
us to consider the clothing labeling problem in two scenara collection of small parts and mode
ios: 1) a constrained labeling problem where we take thethem, using conditional random fields
users’ noisy and perhaps incomplete tags as the list of poscriminative models [8]. Recent work
sible
garmentlabels
forparsing,and2)
whereweconsider（連絡先：Email: {okatani, suganuma}@vision.is.tohoku.ac.jp, Tel: 022-795-7014 Lab: M2-209）
to more general poselet representatio
all garment types in our collection as candidate labels.
rated mixtures of parts [27] to obtain

1.1. Related Work

Clothing recognition: Though clothing items determine
most of the surface appearance of the everyday human,
there have been relatively few attempts at computational

Our pose estimation subgoal builds on
extending the approach to incorporat
in models for pose identification.

Image Parsing: Image parsing has b