About Us
Dwango Media Village is a R&D section of DWANGO Co.,Ltd in Japan.
Dwango Media Village conducts research in the fields of machine learning,
computer vision, computer graphics, audio processing, and game AI.
WEB
https://dmv.nicoTwitter@Dwango_DMV

Research
Multiple images style transfer

Fast and memory-efficient clustering[SIGGRAPH Asia 2016][ACM Multimedia 2017]

33341043 10195867 23607472

13762409

....

....

....

....

..

..

..

“Watercolor” “Pixel Art” “Deformation”

....

....

..

..

..

....

情..

11799762778232

404415

Transfer the desired style of artwork style to an arbitrary image andlearn this style. Our method is based on the observation that the style ofartwork is not characterized by the features of one work, but rather by
the features that commonly appear within a collection of works. To learn
the features that commonly appear within a collection of works, asufficiently large dataset of images created in the same style isnecessary. We present a novel illustration dataset that contains
500,000 images mainly consisting of digital paintings, annotated withrich information such as tags, comments, etc. We utilize a feature spaceconstructed from statistical properties of CNN feature responses, andrepresent the style as a closed region within the feature space.

We propose an efficient clustering method for billion-scale featurevectors, called PQk-means. By first compressing input vectors into
short product-quantized (PQ) codes, PQk-means achieves fast andmemory-efficient clustering, even for high-dimensional vectors. Similar
to k-means, PQk-means repeats the assignment and update steps,
both of which can be performed in the PQ-code domain. Experimental
results show that even short-length (32 bit) PQ-codes can produce
competitive results compared with k-means. This result is of practicalimportance for clustering in memory-restricted environments. Using theproposed PQk-means scheme, the clustering of one billion 128D SIFTfeatures with K = 10^5 is achieved within 14 hours, using just 32 GB ofmemory consumption on a single computer.

Shading of line drawings

Two stage voice conversion [IIH-MSP 2018]

Light Map (θ°)
Shading Result

Input

ResNetEstimated Normal

Light Map (φ°)
Shading Result

Estimate normal maps to shade region segmented line drawings using
convolutional neural networks. The networks are trained by parallel
data: line drawings and geometric normals that are generated from
depth maps. The shading is applied using spherical harmonics lighting.

ProjectWe propose a voice conversion technique with two-stage conversion,
low-dimensional feature conversion and low-to-high spectrogram
mapping. Low-to-high spectrogram mapping is trained using only target
speaker data. Therefore, our method requires few parallel data ofsource speaker and target one to train low-dimensional feature
conversion. Each model is based on convolutional neural networks.
Adversarial training is used to train low-to-high spectrogram mapping
model.

Mobile Application “TekuTekuTekuTeku”

nico-opendata

PhotoCNN

Estimated Depthmap

Near Object Masking

We are providing kinds of data for research communities. One of ourdata is digital images dataset from images posted on Niconico Seiga, an
online illustration community. We also share a pre-trained neuralnetwork model for image tag prediction.

AR Results

We are developing the AR feature for the mobile game,“TekuTekuTekuTeku,” RPG using real-world location. Using CNN, thebackground region is estimated from a single picture, then gamemonsters are synthesized into the picture. This game, released in thisNovember, can be played on iOS and Android.

Horse racing prediction

GO AI “DeepZenGo”

OpenToonz

Estimate the win rate of each horse from its
race history, then decide to bet with regard toodds. Currently, our algorithm won more money
than bet!

Using our GPU farm, we have trained Deep
Learning based Computer Go AI, DeepZenGo.
This year, it defeated the world’ s best humanGO player. Winner of World AI Go Open
tournament.

We are providing plugins for 2D animationproduction as a result of CG and ML
research es. OpenT oonz is open-sourc esoftware to develop and create 2D animations
all together.