2016 Nicograph International

A Mobile Application for Interactive Exploratory Search of Apparel Products
Eriko Koike, Takayuki ItohOchanomizu UniversityTokyo, Japan
Email: {aco, itot} @itolab.is.ocha.ac.jp
displayed by pressing “renew” button. The system selects
a set of icons to be displayed in the next stage by applying
an interactive evolutionary computation algorithm to reﬂect
behavior of users. Users can freely look at various products
based on their preferences or curious without making explicit
queries.
The above mentioned system is supposed to use with personal computers and sufﬁcient sizes of displays. On the other
hand, recent evolution and popularization of smartphones
extended the usage of digital applications and information
accesses. There have been many studies on differencesof information access behaviors between smartphones andpersonal computers. For example, a Web page [10] discusses
that smartphone users tend to search for information during
their spare times. On the other hand, it also mentions that
personal computers are used rather than smartphones whenusers really want to purchase something. This result suggeststhat shopping applications for smartphones have not beensatisfactory, and therefore it is important to discuss and develop them separately from personal computer applications.
Another study [1] concluded that smartphone users tend touse each of applications in a very short time. It suggests thatuser interfaces of mobile applications should be simple sothat users can master the operations in such a short time.
This paper presents a mobile application for interactive
exploratory search of apparel products. There have been
many research studies on mobile applications for shopping
as well as commercial services. Traditional menu-based userinterfaces have been often applied in early studies [3] [4],and then various methods such as text input [5] and vision[8] techniques have been then applied. Small number ofsimple ﬁnger-based operations are one of the features ofthe presented application against such existing studies onmobile shopping applications. Another feature is operations
directly connected to interactive genetic algorithm applied
to learn preferences of users.
This paper presents the user interface design and processing ﬂow of the presented application, and introduces ourexperiment using this application.

Abstract—Many people (especially women) tend to take relatively longer time for shopping, and therefore it is an interesting
topic to develop enjoyable user interface for shopping. We
discussed how to develop systems for apparel product retrieval
inspired by psychology of women’s shopping activity in ourprevious paper, and actually presented an implementation for
personal computers. This paper presents a mobile application
for apparel product shopping inheriting our previous study.
The presented mobile application supports just three singleﬁnger operations: tap, next, and shufﬂe. It shows pictures ofproducts over and over while learning preferences of users
applying interactive genetic algorithm. This paper presents
the user interface design and processing ﬂow of the presented
application, and introduces our experiments.
Keywords-Mobile application,Search, Graphical User Interface.

Interactive

Exploratory

I. I NTRODUCTION
Shopping psychology has been discussed with many studies. Economists mentioned [6] major reasons why people(especially women) spend a long time for shopping. We havebeen studying how to develop effective on-line shopping
applications while adopting such shopping psychologies.
We discussed that collaborative ﬁltering, one of the mostwell-known information recommendation techniques, does
not always work well for apparel products, in our previous
paper [2]. Instead, we mentioned that interactive preference
learning mechanisms applying relevance feedback [9] orgenetic algorithm [7] are effective for various information
recommendation problems, and should be also effective forrecommendation of apparel products. We also concluded
more visual and interactive systems are better for thispurpose, because users can enjoy a long time for shopping,
and the system can gradually learn preferences of the users
while their enjoyable time.
Based on the above discussion, we presented an interactive exploration system for apparel products [2] which aims
to solve the above drawbacks. The system ﬁrstly displays
various icons designed speciﬁc to this study, as the overlay ofimages which represent shapes, colors, and texture patternsof the products. When a user selects an interested icon, thesystem displays images of individual products. Users canpress “prefer” or “delete” buttons to input their preferences
to particular products. They can display a set of preferred
products so that they can decide which products to ﬁnallypurchase. Users can also replace a set of icons to be
978-1-5090-2305-9/16 $31.00 © 2016 IEEE
DOI 10.1109/NicoInt.2016.23

II. D ESIGN P OLICY
We supposed the following policies for mobile applications on apparel shopping, and designed the user interface
to satisfy them.
120

Policy 1: Browse products with small number of operations.
Policy 2: Implement with single-ﬁnger operations so thatusers can pick up and operate a smartphone by a single
hand.
Policy 3: Minimize the number of GUI widgets to utilize thesmall display spaces to show pictures of products as many
as possible.
Policy 4: Prompt unexpected discovery so that users do not
feel boring.
Policy 5: Support users’ input for their sudden mind change.
Policies 1 and 2 are mobile application speciﬁc. Several
previous studies concluded that many mobile applications
are often used very shortly in spare time of users, andtherefore we concluded that operations for mobile shopping
should be simple. Based on this discussion, the presented
application just supports three single-ﬁnger operations: tap,next, and shufﬂe. When the application shows a set ofpictures of apparel products, users can tap particular pictures
so that they can input their preferred products. They canalso change the displayed pictures to the next set when
they want to look at the new ones. The application usuallyshows similar sets of pictures based on preferences of users;
however, they can shufﬂe the pictures so that users cangesture to reset their preferences.
Policy 3 is also mobile application speciﬁc, and also basedon our previous discussion [2] on apparel shopping. In theprevious study, we concluded that users (especially women)
tend to look at many products simultaneously while apparel
shopping, and therefore it is better to show many picturesof products in a single display space. Therefore, we did notprefer to use GUI widgets in a small display space of mobiledevices.
Policies 4 and 5 is derived from our previous development
on shopping application. It is basically effective if theapplications suggest products while learning preferences ofusers. On the other hand, it is often boring for users if
the systems always suggest similar products, and therefore
it is often more satisfactory if they sometimes suggest
unexpected types of products. Based on this discussion,
we apply interactive genetic algorithm to learn preferencesof users and appropriately select products to show, andeffectively perform the mutation mechanism to sometimes
suggest unexpected types of products. Shufﬂe operation
controls the probability of the mutation operation so thatusers can intensively gesture to reset their preferences.

describes user interface design and processing ﬂow of thepresented application.
A. Card metaphor for user interface design
Figure 1 shows snapshots of the mobile application developed in this study. This application is based on the userinterface design with a card metaphor shown in Figure 2.
The application ﬁrstly displays pre-deﬁned number ofpictures of products. Users can keep preferred products toa favorite list, or change the set of pictures. We expect theusers can enjoy looking at many products and select favorite
products by repeating the above process.
The application just supports the following three ﬁnger
operations on the display.
Tap: Users can add their preferable products into a favorite
product list by just tapping the pictures of the products.
Next: Users can see the next set of pictures of products byjust swiping the display from left to right, as card gameplayers see the next card by picking up the top card on apile.
Shufﬂe: Users can randomize the pictures of products to see
unexpected products, as card game players mix all the cards
by shufﬂing on a table.

Figure 1.

^ǁŝƉĞƚŽƐĞĞƚŚĞŶĞǆƚĐĂƌĚSnapshot of the presented application.

^ŚƵĨĨůĞĂůůƚŚĞĐĂƌĚĞƚĞĐƚŝŽŶŽĨƐŚƵĨĨůĞŽƉĞƌĂƚŝŽŶ

III. U SER I NTERFACE D ESIGN AND P ROCESSING F LOWThis section describes technical detail of the presentedapplication. The application shows sets of pictures of products, and replaces them according to users’ operations. The
applications learns preferences of users by applying a genetic
algorithm so that it can preferentially show preferable products. Also, users can perform shufﬂe operations so that theapplication selects different kinds of products. This section

Figure 2.

121

User interface design with a card metaphor.

Figure 1 (Left) shows a snapshot of initial display, andFigure 1 (Right) shows a snapshot while shufﬂing thepictures. Figure 2 (Left) shows metaphors of next and shufﬂe
operations. Figure 2 (Right) illustrates how to detect theshufﬂe operation. Our implementation converts the touched
position from orthogonal coordinate system to polar coordinate system, and calculates the rotation angle of the ﬁnger
operation.
The application calculates the rotation angle θ illustratedin Figure 2 (Right) while shufﬂe operations are performed.
Our implementation shows animation of pictures of products: It moves the pictures to the center of the display space
while 0◦ ≤ θ ≤ 360◦, and then rotates them if 360◦ ≤ θ.
This animation looks as if the pictures are gathered into thecenter and then mixed.
The application calculates the rotation angle by the following equation:s·fθ = cos−1|s||f |c) Matching. Select one product for one child gene.
If there are two or more products corresponding
to the child gene, randomly select one of the corresponding products. If there is no corresponding
products, select a similar product.
For mutation, our implementation usually applies a constant probability (currently 0.1, and adjusts the probability
when shufﬂe operations are performed.
For matching, our implementation sets priority of categorical variables to select one corresponding product. We
set the priority as “classiﬁcation”, “color”, “brand”, and“detail” with our current dataset. If there is no corresponding
products, we select one of the products those three variables
except “detail” correspond to the current child gene.
IV. E XPERIMENT
We implemented the application and executed on SONY
NW-Z1000 with Android OS 4.0.4, and conducted a user
experiment with 6 participants who were female university
students. This experiment aimed to evaluate the effectivenessof our user interface design; especially we were interested
in how shufﬂe operation and mutation process improve thesatisfaction of users.
We prepared two implementations of the application: one
supported the shufﬂe operation, while the other did notsupport. We asked participants to play with the two implementations alternatively. Three of the participants ﬁrstly used
the implementation with shufﬂe operation, while the otherparticipants ﬁrstly used the other implementation. We thenasked them to answer the following questions in ﬁve-point
Likert scale. Figure 3 shows the statistics of answers forthese questions.
Q1: Which implementation is preferable for you?
Q2: Did you feel the difference of product selection by
the application according to the rotation angle of
shufﬂe operation?
Q3: Did you feel necessary to control the change of
product selection according to the rotation angle?
(asked to participants who positively answered toQ2.)
Q4: Is this metaphor intuitive and easy to understand?
Q5: Is this metaphor easy to operate?
Also, we asked to select one or more choices for thefollowing question. Table I shows the choices for thisquestion, and number of participants selected each of thechoices.
Q6: Which situations do you want to play with thisapplication?
Finally, we asked the participants to write down any
feedback regarding this experiment.
Figure 3 shows totally good results, except just one participant rated “2” for Q2. We heard that the participant had thesame rotation angle while multiple shufﬂe operations every(1)where s is the starting position of the shufﬂe operation,and f is the current position, supposing the origin of thecoordinate system is the center of the display space.
Our current implementation calculates the possibility ofmutation.
B. Product selection by interactive genetic algorithm
We implemented a product selection mechanism applying
a modiﬁed interactive genetic algorithm. Our implementation treats products in the favorite list as parent genes. Itgenerates children genes by crossover of the parent genes,
when a next operation is performed. Also, it applies mutation
when a shufﬂe operation is performed, while the mutation
ratio is calculated from the rotation angle of the shufﬂeoperation.
Our implementation supposes that each product has several categorical variables to specify the features of theproducts. Our current dataset has four categorical variables,“classiﬁcation”, “color”, “brand”, and “detail”. The technique simply encodes these variables to genes.
Following is the processing ﬂow of interactive productselection applying a modiﬁed interactive genetic algorithm.
1) Initialize the algorithm. Select constant number (currently 6) of products randomly, as parent genes.
2) Repeat 3) to 5).
3) Display products corresponding to the parent genes.
Await tap operations.
4) Treat constant number (currently 12) of recently favorited products as the new parent genes.
5) Repeat the following processes.
a) Crossover. Generate a child gene from a pair ofrandomly selected genes.
b) Mutation. Randomly modify values of children
genes with pre-deﬁned probability.

122interactive genetic algorithm to suggest products over andover while learning preferences of users. Our experimentdemonstrated the effectiveness and remaining issues of theuser interface design and product selection mechanism of
the presented application.
We would like to customize the implementation andparameters of interactive genetic algorithm with larger number of products, and then conduct larger scales of user
experiments.

YϭYϮYϯYϰYϱϬϭϮϱϯϰϰϯϱϲϮ

R EFERENCES
Figure 3.

Statistics of answers for the ﬁve questions.

[1] M. Bohmer, et al., Falling asleep with Angry Birds, Facebook
and Kindle: a large scale study on mobile application usage,
13th International Conference on Human Computer Interaction with Mobile Devices and Services, 47-56, 2011.

Table IC HOICES FOR Q6 AND NUMBER OF PARTICIPANTS SELECTED THE
CHOICES .
Choice(a)(b)(c)(d)(e)(f)

I want to specify clothes to put on tomorrow.
I want to purchase particular products.
I have a spare time and want to look at products
without any targets.
I want to just look at something while working
with other tasks.
I get a money and therefore want to purchasesomething.
I want to release my stress.

Number ofparticipants005[2] E. Koike, T. Itoh, An Exploratory Apparel Product SearchInterface Adopting Shopping Psychology, The Journal ofSociety for Art and Science, 14(5), 170-179, 2015.
[3] S. Kurkovsky, V. Zaney, A. Kurkovsky, SMMART: using
context-awareness in m-commerce, 7th International Conference on Human Computer Interaction with Mobile Devicesand Services, 383-384, 2005.

524[4] E. Newcomb, T. Pashley, J. Stasko, Mobile computing in
the retail arena, SIGCHI Conference on Human Factors in
Computing Systems, 337-344, 2003.

time, and therefore could not see the difference of productselections. Another participant also mentioned that it seemed
difﬁcult to understand the relationship between the rotation
angle and difference of product selection. On the other hand,
many participants mentioned that they could easily masterto use the application thanks to the simple user interface
design. Based on the results, we would like to discuss how
we can make easier to understand the mutation mechanism,
while keeping the simpleness of the user interface design.
Table I shows that many participants suggested to use
this application when users do not have particular needs or
demands to purchase products. In addition, several participants mentioned that “this application is useful to browse
many products and narrow down interest of users before
they actually purchase something” and “we can recognize
our own preference while using this application.” Thesecomments also suggest that the presented application issuitable to use when users do not want to immediately
purchase something, but just want to look for preferable
products or think of their own preferences.

[5] P. Nurmi, A. Forsblom, P. Floreen, P. Peltonen, P. Saarikko,
Predictive text input in a mobile shopping assistant: methods and interface design, 14th International Conference onIntelligent User Interfaces, 435-438, 2009.
[6] H. Oda, Why women take a long time for shopping?, ISBN978-4-569-69428-3, 2007.
[7] Y. Saito, T. Itoh, MusiCube: A Visual Music Recommendation System featuring Interactive Evolutionary Computing,Visual Information Communication - Information Symposium, 2011.
[8] Y. Xu, M. Spasojevic, J. Gao, M. Jacob, Designing a visionbased mobile interface for in-store shopping, 5th Nordic
Conference on Human-Computer Interaction, 393-402, 2008.
[9] R. Yong, T. S. Huang, M. Ortega, S. Mehrotra, Relevance
feedback: a power tool for interactive content-based image
retrieval, IEEE Transactions on Circuits and Systems forVideo Technology, 8(5), 644-655, 1998.
[10] Sendenkaigi, Difference of search and access behavior
between smartphone and personal computer users, 2013.
http://mag.sendenkaigi.com/senden/201312/smartphoneconsumption-behavior/000933.php

V. C ONCLUSION
This paper discussed what kinds of shopping applications
on mobile devices are desirable, and presented a mobile
application for apparel product shopping. This application
is based on simple user interface design supporting only
three single-ﬁnger operations and no GUI widgets, since
many people use mobile applications very shortly and therefore simple applications are often preferable. We applied123