DeepWear: a Case Study ofCollaborative Design between Humanand Artificial Intelligence
Natsumi Kato*
University of Tsukuba
Hiroyuki Osone*
University of Tsukuba
Daitetsu Sato
University of Tsukuba

Abstract

Naoya Muramatsu
University of Tsukuba
Yoichi Ochiai
University of Tsukuba

Conventional Method
Input

DCGANsGenerator

Design Sketch

Patterns Making

Clothes Making

DeConv

Fashion Show

Linear&Sigmoid

BatchNorm

DeConv&ReLU&BatchNorm&Reshape

&ReLU

z

100

52428816*16*2048

32*32*1024

64*64*512

128*128*256

128*128*3

Discriminator

Conv

LeakyReLU

BatchNorm&Conv&LeakyReLU&Conv

BatchNorm&LeakyReLU&Linear

128*128*3

64*64*512

32*32*1024

16*16*2048

524288OutputFeedback

Clothes Making

Patterns Making

Generated Images

Our Contribution

Figure 1: Our system’s workflow. Conventional fashion design is afeedback loop. First, designers make their clothes design. Next,
patterners draw patterns from the design and designers instrucion.
The We use DCGANs to make new feedback loop. Fashion design
process In this process, making design inspiration easier fordesigners.
*Joint first authorship.
This is the author’s version of the work. It is posted here for your personal use. Not forredistribution.
TEI 2018, March 18–21, 2018, Stockholm, Sweden.
Copyright held by the owner/author(s)ACM ISBN 978-1-4503-5568-1$15.00
https://doi.org/10.1145/3173225.3173302

Deep neural networks (DNNs) applications are now increasingly pervasive and powerful. However, fashion designers are lagging behind in leveraging this increasingly
common technology. DNNs are not yet a standard part offashion de sign practice, either clothes patterns or prototyping tools. In this paper, we present DeepWear, a method
using deep convolutional generative adversarial networks
for clothes design. The DNNs learn the feature of specific brand clothes and generate images then patterns instructed from the images are made, and an author createsclothes based on that. We evaluated this system by evaluating the credibility of the actual sold clothes on market
with our clothes. As the result, we found it is possible to
make clothes look like actual products from the generated
images. Our findings have implications for collaborative design between machine and human intelligence.

Author Keywords
Fashion; Creativity support; DCGANs.

Introduction
Recent advances in computational fabrication have afforded
the opportunity to use automated tools and machines tosupport fashion design. However, obtaining inspiration of
fashion design is still a difficult task. To address this challenge, there are projects to improve the process by incorpo-

rating DNNs into fashion design, such as Project Muze[1]or Amazon’s AI [14]. Project Muze use DNNs to learn thetrend and design clothes. Amazon’s DNNs are GANs basedarchitecture. To internalize properties of a particular style
simply by looking at many examples and apply that style toexisting clothing. However, there are articles [22, 25] which
doubt as to whether the styles created by Project Muze are
actually that people can wear. Also, since Amazon’s AI isstill in the development stage, these projects are not stillpractical.
In this work, we present DeepWear, practical designing
clothes system use DCGANs [23] to generate images anddesigners make clothes by receiving instruction from thoseimages. State-of-the-art deep learning techniques are first
applied to the workflow of designing clothes. The systemtakes specific brand [2] clothes images as the input, learns
the feature of inputs, generates images that looks close to
the clothes, then patterns instructed from the images are
made, and an author creates clothes based on that.
In the evaluation, we conduct a content analysis about
the theme related to our system practicality by comparing
the actually sold clothes with our clothes and other brandclothes by questioning which are the actually sold the specific brand clothes. The results show that our system is possible to make clothes look like actual products from the generated images. This paper shows the implications for collaborative design between machine and human intelligence.

Related Works
Image Generation
Recently, the deep generation model has attracted a lot of
attention and the ability to learn large (unlabeled) data potential and vitality [3, 24]. In [7], the deep belief net (DBN)
using a contrast divergence algorithm was initially proposed

to efficiently training. denoising autoencoder (DAE) learns
data distribution with supervised learning method [3]. BothDBN and DAE learn the low dimensional representation(encoding) of each data instance and generate from thedecode network. In recent years, Variational AutoEncoder(VAE) combine deep learning and statistical inference forrepresent data instances in a latent [13] hidden space while
utilizing a deep neural network for nonlinear mapping isused. All of these generation models are trained by maximizing the possibility of training data i.e. generative adversarial networks (GANs) [6] are generative model which areminimax game between generator model and discriminator model. This framework avoids the difficulty of maximum
likelihood learning and has remarkable success in natural
image generation [4].
GANs show impressive results in image generation [4], image transfer [11, 30], super resolution [15] and many othergeneration tasks. DCGANs utilize deep convolutional networks architecture and batch normalization [10] to extract
the feature [23], and show significantly great output. Therefore, in this study, we used DCGANs architecture for image
generation network.
Machine Intelligence Creativity Support
In other than fashion design, there are researches to support human creative activities by incorporating machine
intelligence. AutoDraw is a new kind of drawing tool thatcombines machine learning and drawing of an artist so thatanyone can create something visual quickly [9]. User experience designers integrate machine learning services
in new apps, devices, and systems [5, 28]. Many studies
have been issued for the purpose of supporting the creation of cartoons and animation. There are systems that
inputs black-and-white line drawing manga images and automatically outputs colored images [29]. In order to support

Generator
DeConvLinear&Sigmoid

BatchNorm

DeConv&ReLU&BatchNorm&Reshape

&ReLU

z

100

52428816*16*2048

32*32*1024

64*64*512

128*128*256

128*128*3

Discriminator

Conv

LeakyReLU

BatchNorm&Conv&LeakyReLU&Conv

BatchNorm&LeakyReLU&Linear

128*128*3

64*64*512

32*32*1024

16*16*2048

Figure 2: Our Networks

character design, there is research to automatically generate facial images using GANs [12]. The method to identify
structural lines from pattern-rich cartoons without being
conscious of patterns is developed [18]. Extracting structural lines from pattern-rich manga is an important step fortransferring legacy manga to the digital domain.

tures that people actually can wear [22, 25]. Amazon’s
Project use GANs based DNNs architecture [14] internalize properties of a particular style simply by looking at many
examples and apply that style to existing clothing. Amazon’s AI is still in the development stage, and these projects
are not yet practical.

Machine Intelligence & Fashion Design
There are several works trying to support or recommend
everyday life fashion design [26]. Especially, there are many
recommendation[19, 20] or classification systems [21,27]. Heterogeneous graphs to link fashion items, make up
stylish costumes, and link items to their attributes are used[16]. In [8], a tensor decomposition approach is proposed
to recommend a set of fashion items. Rather than learning
item features based on sets, use discrete item attributes
or low-level image features. In [17], implemented a representation learning framework for fashion items that includes
latent styles in which learned expressions are shared byitems in the style set.

In summary, our approach is distinct from the above in that
we aim to make practically usable clothes from a specificbrand clothes dataset. Our approach captures fashion semantics on the style space, which can be effectively utilizedby fashion design systems. We asked patterners to write
a pattern based on the generated image, and investigatedwhether we could actually make clothes. Finally, we implemented clothes based on that pattern and user-studied thequality.

Research AI fashion design. Project Muze [1] is based on
Google’s open source TensorFlow1 . It is developed as apredictive design engine. It consists of two parts: neuralnetworks and a set of aesthetic parameters. The neural network learn the color, texture and style preferences of over
600 fashion experts. Over time, it learned to connect those
preferences to other people with similar interests. Thena set of aesthetic parameters from the Google Fashion
Trends Report and Zalando’s2 deep knowledge of fashion
trends is used to refine the designs and make sure they’refashion forward. However, there are articles doubt whether
the styles created by Project Muze are actually the struc-5242881
https://github.com/tensorflow/tensorflow (last accessed January 9,
2018)2
https://www.zalando.co.uk (last accessed January 9, 2018)

Deep Convolutional Generative Adversarial Networks
Deep Convolutional Generative Adversarial Networks (DCGANs) is a neural networks that has been popular in recent
years. This lets the network learn to generate data with thesame internal structure as other data (Figure 2). One of
the most common applications is image generation. The
Generative adversarial network consists of a generator network G and a discriminator network D. Given training datax, G will take input from random noise z and try to generate
data with distribution similar to x. The discriminator network
D receives inputs from both x and the generated from G
and estimate the probability that the sample came from thetraining data, not G. G and D are trained at the same time:
Adjust the parameters of D to maximize the probability ofassigning the correct label to both the training example and
the G sample, and adjust the parameters of G to minimizelog(1−D(G(z)). In other words, D and G play the following
two player min-max game with value function V (G, D) (1).


min max V (D, G) = Ex∼pdata (x) log D(x)GD+Ez∼pz (z) log(1 − D(G(z)))(1)

running around 43000 iteration (about 270 peoch), because
the loss has become quite small and generated images
looks good. At the 43,000 iteration, G loss is 16.1374, D
loss is 1.39158.

Clothing implementation

Figure 3: Making clothes frompatterns.

A(1)

B(1)

B(2)

Figure 4: Our system’s output
clothes.

data collection
We collected images of a specific brand announced between 2014 and 2017. We used web scraping Python code
to create the training dataset. Scraping consists of threesteps. In the first step, follow the link from the top page ofthe target website. Second, we list all HTML pages with the
URL structure as directory structures. Then, we acquire
all the image URLs specified by src of the img tag in theHTML pages detected in the second step. In the third step,
we downloaded all image URLs. In order to prevent overloading the server, we restricted the request to 10 times / 1
s. For pages that restricted crawler behavior by robots.txt,
we followed that restriction. As the result, over 1.1K images
were collected. Several steps were performed to learn thefeature of the images. We paint the background white sothat only people and clothes are cut out, and processed it
into full color image of 128 px x 128 px (resized dataset ishere.3 (last accessed January 9, 2018)).
Training
We implemented our network with Chainer4 , a deep learning framework. We followed implementation and training
procedure recent work by Radford et al. [23]. Our networkarchitecture is shown in Figure 2, where training was donewith a batch size of 7, using Adam with hyperparameters(α=0.0002, β1 = 0.5, β2 = 0.999,  = 1e − 08), and run
on an NVIDIA Titan X GPU for 1000 epochs. We stopped
3

https://drive.google.com/open?id=0BywMLxsmMEnROGNKZzAtVkR3aFE
https://github.com/chainer/chainer https://drive.google.com/open?
id=0BywMLxsmMEnROGNKZzAtVkR3aFE (last accessed January 9,
2018)
4

Draw Patterns
Seven participants(6 females and 1 male) aged between
21 and 23 years partcipated. They experienced fashion
design patterners and had clothing experience of 1 year
and a half to 5 years (Patterners are people who draw patterns of clothes based on instructions from designers). We
ask them to draw patterns based on images generated by
DCGANs. This work were done under the presence of an
author or by online calls. We set time limit 70 minutes for all
subjects. The pattern created for the first was the image in
Figure 5 A, the second pattern was asked to draw the pattern based on the image selected by each subject (shownin Figure 5 B). Subjects were asked to use writing tools
such as rulers and pencils are usually used when drawing patterns. Actually, they made a pattern of size within therequired time, not full size sufficient to make clothes. The
patterns that the patterner drew based on the image wespecified are the Figure 5 A, and drew based on the images
selected by each are the Figure 5 B. The numbers (1) - (7)are the numbers of the subjects. Based on the generated
image, the patterners drew patterns with each idea. In A, 4
out of 7 of the subjects drew a pattern of a combination of aone-piece dress and outer. Two subjects drew a one-piece
dress pattern. One person drew a pattern of tops, skirts,
and outer combinations. Six subjects drew one-piece dress
patterns.
Make Clothes from the Patterns
Based on the pattern that the patterner handwritten, tracingwith Illustrator, converting it to data, making it the size of thefull size that can be worn as clothes. Then, using two kindsof black cloth, an author made three kinds of clothes fromdrawn pattern by two subjects. The working time was about
50 hours.
We found that the silhouette considerably. Figure 4 shows
images of clothes. A (1) and B (1) are produced based on
the pattern made by subject 1 pattern and B (2) are basedon subject 2 pattern.

A

(1)

(5)

(2)

(6)

(3)

(7)

(4)

B

(1)

(2)

(3)

(5)

(6)

(7)

A (1), B (1) are clothes made using black cotton sheeting,B (2) has a shiny feeling and is made from a heavy black
cloth. Sheeting is a cloth generally used for prototypes, thedensity of weaving is low, breathable fabric and thin. B (2)was made with a heavy, shiny, soft-touching and thick cloth.
The concept of the brand used as data set of the generated
image is to break the stereotype of gender by incorporating the style of male clothes into women’s clothes. One ofits characteristics is that clothes are oversize. Looking at
the output clothes, the B (2) made of a heavy black cloth
dropped down the silhouette of the entire clothing than theA (1) and A (2) made of the sheeting. Therefore, we thinkthat making clothes with a heavy black cloth was able tocapture the characteristics of the original data brand.

(4)

Qualification & User Reaction
Figure 5: Patterns made from thepatterners. A patterns are drawn
from same image by eachpatterners, B patterns are drawn
from what patterners selected.

We conducted an experiment to evaluate whether the image generated from DNNs can be instruction sources forcreating a new clothes. The experiment is evaluating thequality of the actually sold clothes on market with our clothes
and the other brand clothes.
Participants
Thirty two people who didn’t experience clothes design (14females, 18 males) aged between 19 and 61 years (M =24.3, SD = 8.96) answered this questionnaire.

Ave

Other
DeepWear
Source brand80 (1)
121 (3)131 (2)78 (5)
87 (6)
120 (4)

97 (7)107 (8)115 (9)85
105(+23.5%)
122(+43.5%)

Table 1: The result of the questionnaire that to distinguish thesource brand clothes.

Experimental Design
We prepared images of the source brand and output of our
method and clothes images of other brands567 , respectively
three. To the subjects, six clothes of the source brand werefirst exemplified. After that, we asked clothing images one
by one in random order and evaluated whether or not thedisplayed image can be seen closer to the product of thesource brand in 7 stages of 1 (looks different) to 7 (looks
learning source brand).
Result & Discussion
An weighted averages of these results are shown in the Table 1. For the costumes designed based on our system,
they are understood that the subjects were more impressedby the learning source brand and ours than the other brandcostumes. In addition, we have shown that our output isclose enough to the learning source brand, as the output
costumes are significantly closer to the learning source
brand than other brand costumes. Looking at the answersof the subjects, the clothes that were judged to be mostsimilar to the clothes of the original brand were the output
of our system (Table 1: (3) ) and the clothes of the actual
learning source brand.
5

https://page.auctions.yahoo.co.jp/jp/auction/v499221070
https://page.auctions.yahoo.co.jp/jp/auction/260446858
7https://page.auctions.yahoo.co.jp/jp/auction/m218550387 (last accessed January 9, 2018)
6

The concept of the brand is to emphasize genderless design, however for image 6, we think that the chest race wasemphasized as a feminine design, so it doesn’t look like thebrand. It is difficult for a patterner to judge detailed details
from the generated image, and we thought that the difference has come out by the intention of the patterner entering the design. From the above, since the silhouette of theclothing is greatly different depending on the cloth used as
the material, we consider important to choose which cloth
to use in order to make clothes similar to genuine brandclothes. In addition, many of the clothing of the learningsource brand are black in color and simple design, so texture of the cloth is more emphasized. Also, garments madeentirely with large silhouettes tended to resemble real ones.
Patterners Feedback to Experiment
We took a questionnaire on how the patterner sees the answer to this experiment.
Some of a kind are quite simple clothes, but if they are notsimilar, looking at the whole and putting effort into detailsuch as frills are attached to the front neck.
(Patterner 1: Male, 21, design experience 1 and half years)

Figure 6: Wearing DeepWear.

I think that the judgment criteria of similarity or dissimilarity are mainly the length and the whole image. I thoughtthat it was judged by those with a center of gravity on theupper body were not judged to be similar, the lower body
had a center of gravity, and those with a long sense of resemblance were similar. Also, I think that it was difficult tojudge that many cases of the brand have the similarities likeoversize, straight line, and big silhouette image because thewaistline of other companies’ brand products is narrow.
(Patterner 2: Female, 22, design experience 3 years)

I think that the feature of the specific brand’s design is silhouette that is comfortable between the body and clothes,so I think that such a silhouette design was chosen as
the brand. I felt that the thing judged not to be similar was
judged that detail or material are not like the brand. I thoughtthat the brand does not have much design of material likeorganza decorated with neck tuck.
(Patterner 3: Female, 22, design experience 5 years)

Conclusion
DNNs are now a fairly established technology, and fashion
designers have begun to integrate DNNs application into
the things that they design. This paper presents a system
conducted with application DCGANs to design clothes inpractice. When asked the patterner to write a pattern basedon the generated image, a pattern which can actually makeclothes was created after obtaining a good response fromthe patterner to the work. Finally, we implemented clothes
based on that pattern and we conducted a user study onthat quality, and found that the clothes made by DeepWear
are high quality clothes. Our findings show that our systemenable collaborative design between machine and human
intelligence. We expand on these findings to present a series of challenges for DNNs and fashion design research.
Video link ( https://youtu.be/pVlLwvRVdb8 )REFERENCES
1. 2016. PROJECT MUZE. (2016). Retrieved September
15, 2017 fromhttps://projectmuze.squarespace.com.
2. 2017. FASHION PRESS. (2017). Retrieved September
15, 2017 fromhttps://megalodon.jp/2018-0108-0109-31/https://www.fashion-press.net:443/collections/brand/42.

3. Yoshua Bengio, Li Yao, Guillaume Alain, and Pascal
Vincent. 2013. Generalized denoising auto-encoders
as generative models. In Advances in Neural
Information Processing Systems. 899–907.
4. Emily L Denton, Soumith Chintala, Rob Fergus, andothers. 2015. Deep Generative Image Models using a
Laplacian Pyramid of Adversarial Networks. In
Advances in neural information processing systems.
1486–1494.
5. Graham Dove, Kim Halskov, Jodi Forlizzi, and John
Zimmerman. 2017. UX Design Innovation: Challenges
for Working with Machine Learning as a Design
Material. In Proceedings of the 2017 CHI Conferenceon Human Factors in Computing Systems. ACM,
278–288.
6. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza,
Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron
Courville, and Yoshua Bengio. 2014. Generative
adversarial nets. In Advances in neural information
processing systems. 2672–2680.
7. Geoffrey E Hinton, Simon Osindero, and Yee-WhyeTeh. 2006. A fast learning algorithm for deep belief
nets. Neural computation 18, 7 (2006), 1527–1554.
8. Yang Hu, Xi Yi, and Larry S Davis. 2015. Collaborative
fashion recommendation: A functional tensor
factorization approach. In Proceedings of the 23rd ACM
international conference on Multimedia. ACM,
129–138.
9. Google Inc. 2017. AutoDraw. (2017). Retrieved
September 18, 2017 from https://www.autodraw.com.
10. Sergey Ioffe and Christian Szegedy. 2015. Batch
normalization: Accelerating deep network training by

reducing internal covariate shift. In International
Conference on Machine Learning. 448–456.
11. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A
Efros. 2016. Image-to-image translation withconditional adversarial networks. arXiv preprint
arXiv:1611.07004 (2016).
12. Yanghua Jin, Jiakai Zhang, Minjun Li, Yingtao Tian,
Huachun Zhu, and Zhihao Fang. 2017. Towards theAutomatic Anime Characters Creation with Generative
Adversarial Networks. arXiv preprint arXiv:1708.05509(2017).
13. Diederik Kingma and Jimmy Ba. 2014. Adam: A
method for stochastic optimization. arXiv preprint
arXiv:1412.6980 (2014).
14. Will Knight. 2017. Amazon Has Developed an AI
Fashion Designer. (24 August 2017). Retrieved
September 15, 2017 fromhttps://www.technologyreview.com/s/608668/
amazon-has-developed-an-ai-fashion-designer/.
15. Christian Ledig, Lucas Theis, Ferenc Huszár, Jose
Caballero, Andrew Cunningham, Alejandro Acosta,
Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan
Wang, and others. 2016. Photo-realistic single image
super-resolution using a generative adversarial
network. arXiv preprint arXiv:1609.04802 (2016).
16. Hanbit Lee and Sang-goo Lee. 2015. Style
Recommendation for Fashion Items using
Heterogeneous Information Network.. In RecSys
Posters.
17. Hanbit Lee, Jinseok Seol, and Sang-goo Lee. 2017.
Style2Vec: Representation Learning for Fashion Items
from Style Sets. arXiv preprint arXiv:1708.04014(2017).

18. Chengze Li, Xueting Liu, and Tien-Tsin Wong. 2017.
Deep extraction of manga structural lines. ACM
Transactions on Graphics (TOG) 36, 4 (2017), 117.
19. Zequn Li, Honglei Li, and Ling Shao. 2016. Improving
Online Customer Shopping Experience with Computer
Vision and Machine Learning Methods. In International
Conference on HCI in Business, Government andOrganizations. Springer, 427–436.
20. Qiang Liu, Shu Wu, and Liang Wang. 2017. DeepStyle:Learning User Preferences for Visual
Recommendation. In Proceedings of the 40thInternational ACM SIGIR Conference on Research andDevelopment in Information Retrieval. ACM, 841–844.
21. Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, andXiaoou Tang. 2016. Deepfashion: Powering robust
clothes recognition and retrieval with rich annotations.
In Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition. 1096–1104.
22. Sarah Perez. 2016. Google’s new Project Muze proves
machines aren’t that great at fashion design. (2September 2016). Retrieved September 18, 2017 fromhttps://projectmuze.squarespace.com.

26. Haosha Wang and Khaled Rasheed. 2014. Artificial
intelligence in clothing fashion. In Proceedings on theInternational Conference on Artificial Intelligence(ICAI). The Steering Committee of The World Congressin Computer Science, Computer Engineering andApplied Computing (WorldComp), 1.
27. Lixuan Yang, Helena Rodriguez, Michel Crucianu, andMarin Ferecatu. 2017. Fully Convolutional Network with
Superpixel Parsing for Fashion Web Image
Segmentation. In International Conference on
Multimedia Modeling. Springer, 139–151.
28. Qian Yang. 2017. The Role of Design in Creating
Machine-Learning-Enhanced User Experience. (2017).
29. Taizan Yonetsuji. 2017. PaintsChainer. (2017).
Retrieved September 18, 2017 from https://paintschainer.preferred.tech/index_en.html.
30. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A
Efros. 2017. Unpaired image-to-image translation using
cycle-consistent adversarial networks. arXiv preprint
arXiv:1703.10593 (2017).

23. Alec Radford, Luke Metz, and Soumith Chintala. 2015.
Unsupervised representation learning with deep
convolutional generative adversarial networks. arXiv
preprint arXiv:1511.06434 (2015).
24. Ruslan Salakhutdinov. 2009. Learning deep generativemodels. University of Toronto.
25. Brittany Vincent. 2016. Google’s Project Muze creates
unwearable fashion pieces. (2 September 2016).
Retrieved September 18, 2017 fromhttps://www.engadget.com:443/2016/09/02/
googles-project-muze-creates-unwearable-fashion-pieces/.