An interactive exploratory search systemfor on-line apparel shopping
Eriko Koike

Takayuki Itoh

Ochanomizu University
2-1-1 Otsuka, Bunkyo-ku, Tokyo
+81-3-5978-5399

Ochanomizu University
2-1-1 Otsuka, Bunkyo-ku, Tokyo
+81-3-5978-5700

aco <at> itolab.is.ocha.ac.jp

itot <at> is.ocha.ac.jp

ABSTRACT

which they want to buy at the moment.

Many people (especially women) tend to take relatively longer
time for shopping. This paper presents a system for productretrieval inspired by psychology of women's shopping activity,
and an implementation of the system for apparel products. Our
study supposes products which pre-defined keywords are assigned,
and prepares icons representing the combination of the keywords.
The system intuitively displays various icons in a display space todemonstrate the diversity of the products. When a user selects aninterested icon, the system switches the display to a set of images
corresponding to the selected icon, so that the user can visuallycompare the similar products. The system also features userinterfaces to input the preference of users, and reflects the input tothe evolutionary computation which adjusts the selection of icons
to their preferences. It acts real shopping behavior because weoften firstly look over the shops to understand the diversity of
products, and then close up the particular groups of the products.
This paper introduces an experiment which demonstrates thepreferable assistance of the shopping behavior of women who areinterested in various products.

3) Customers often want to continue finding their best products
taking a long time.

Categories and Subject Descriptors
H.3.3 Information search and retrieval, H.5.2 User interface, I.3.8
Computer graphics application, K.4.4 Electric commerce

General Terms
Experimentation, Human Factors.

Keywords
Exploratory search, On-line apparel shopping, Interactive
evolutionary computing, Visualization, Icon generation.

1. INTRODUCTION
Shopping is fun for many people. Especially women tend to take along time for shopping. Economists mention [11] that thefollowing are major reasons for taking long time for shopping:
1) Requirements of consumers are often ambiguous.
2) Curiosity of consumers is not limited to the target products
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,requires prior specific permission and/or a fee. Request permissions
from permissions@acm.org.
VINCI’15, August 24–26, 2015, Tokyo, Japan.
Copyright 2015 ACM 978-1-4503-3482-2 …$15.00

DOI: http://dx.doi.org/10.1145/2801040.2801041

4) Customers often enjoy shopping processes themselves.
Electric commerce (EC) still increases their market sizes. Online shopping on the Internet is very convenient because we canpurchase various products without visiting to real shops. However,
it is often time-consuming or bothering because we need toaggressively query the products while selecting adequate
keywords or categories. It is generally easy if consumers want topurchase particular products. Otherwise, we think the on-line
shopping processes are not very enjoyable. Especially, many ofexisting on-line apparel shopping Web sites prepare large numberof categories, and therefore users often need to specify adequate
combinations of the categories to look for their demanded
products. It is often bothering and even stressful.
The concept of "Exploratory Search" [17] has been recently
proposed as a framework to support the search process for theusers who have ambiguous requirements. We expect thisframework should be useful for on-line apparel shopping. This
paper presents an interactive exploratory search technique and its
implementation for on-line apparel shopping. Based on thecomment of economists and our questionnaire results, thepresented system aims to satisfy the following three requirements:[Requirement 1:] It does not always require exact query so thatusers can enjoy shopping with ambiguous needs.
[Requirement 2:] It firstly shows groups of similar products, andthen focuses to individual products in the selected groups, as users
often look over the inside of the shops first and then focuses toparticular products while the real shopping.
[Requirement 3:] It repeats to display preferable numbers ofvarious product groups reflecting the behavior of users' browsing
processes, so that users can be interested in various products.
The system firstly displays various icons representing a certain
combination of categories. When a user selects an interested icon,
the system displays images of individual products. Users can press"prefer" or "delete" buttons to input their preferences to particular
products. They can display the set of preferred products so thatthey can decide which products to finally purchase. Users can also
replace the set of icons to be displayed by pressing "renew" button.
The system adequately selects the set of icons to be displayed in
the next stage by applying an interactive evolutionary
computation algorithm to reflect the behavior of the users. Users
can freely look at various products based on their preferences or
curious without making explicit queries. The system mainlyshows icons based on preferences of users, while showing small
number of other icons so that they can be interested in various
products.

2. PRESENTED SYSTEM
This section describes overview and processing flow of thepresented technique. The technique firstly displays a set of icons
as shown in Figure 1. Here, the icons represent certain
combinations of keywords related to the appearance andcategories of the products. Users can interchange the set of icons
to be displayed by pressing "renew" button. Repetitively
displaying the sets of adequate numbers of icons, users can enjoy
the search of their preferable products, as they enjoy real shoppingby looking at many products over and over.

Figure 2. Display of products corresponding to the selected
icon.

Tops
(T-shirt)
Figure 1. Snapshot of our technique featuring icon display andgraphical user interface.
Here, we suppose the combination of the above mentioned
various keywords are also assigned to the all products. Our
implementation divides all products into a certain number ofgroups according to the combinations of keywords, and associates
icons to the product groups one-by-one.
When a user clicks one of the icons, the technique switches todisplay images of individual products in the particular groupcorresponding to the clicked icons, as shown in Figure 2. Users
can focus to each of the images, and press "prefer" or "delete"button for the particular products.
The technique supports two display modes: "icon display mode"for overview of the variety of product groups shown in Figure 1,
and "product display mode" focusing on particular groups shownin Figure 2. We think this policy is analogous to real shopping: we
often look over the interior of shops to feel diversity of the
products, and then focus on particular groups of products tocompare the similarity.

2.1 Icon creation
We suppose owners of this system prepare a set of icon images
representing the combinations of predefined keywords. Figure 3
shows an example of icon creation process, which represents the
combination of keywords, "T-shirt", "red", and "dot". This design
policy brings unified impression though the application, andintuitive understanding of product groups corresponding to the
combinations of the keywords. We suppose that keywords aredefined mainly based on shapes, colors, and patterns of the
products, because many customers of the apparel products mainly
select the products based on their visual design.

Red

Dot

Tops/Red/Dot

Figure 3. Example of icon creation.

2.2 Icon layout
This technique displays adequate number of icons in a display
space as shown in Figure 1, where the icons are selected based onan evolutionary computation algorithm described later. We expect
users can be interested in wide range of products by looking at thevariety of icons, On the other hand, we need to keep the numberof displayed icons not too much, because users may be confusedor dithered if too many icons are displayed at the same time.
Our implementation represents the numbers of products
corresponding to the icons by their sizes. Users can intuitively
understand how many products are prepared in each of the productgroups corresponding to the icons.
Here, we suppose the following three conditions for the naturally
looking icon layout results:
1) Place similarly looking icons closer.
2) Optimize display space utility. (=Yield less gaps.)3) Yield no overlaps among the icons.
To satisfy the above conditions for the icon layout, ourimplementation applies a hybrid force-directed and space-fillingalgorithm [4], originally developed for graph visualization. The
former part of the algorithm connects icons which common
keywords are assigned by edges, and supposes attractive forces tothe edges. It also supposes repulsive forces among icons which are
not connected by the edges. As a result of iterative force
calculation, similarly looking icons are closely placed in thedisplay space. The latter part of the algorithm applies a rectangle
packing technique so that display space utility is optimized whileicons do not overlap each other.

2.3 User Interface

icon. Here, e x  1 if the x-th keyword is assigned,

Our implementation features the following user interface widgets.

otherwise e x  0 .

[Selection of the number of icons:] We featured a menu to selectthe number of icons to be displayed.
[Selection of keywords:] The right side of the window shown in
Figures 1 and 2 features a set of checkboxes to select various
attributes including gender of the user, type, price, size, andcategory of the products. The evolutionary computation algorithm
described later selects icons matched to the user-selected
keywords. If the user selects no keywords, the algorithm selects
icons to be displayed from the every icons. Even if the user does
not have any specific target products and therefore does not press
any checkboxes, he/she can enjoy the shopping because thesystem shows arbitrary sets of the icons.
[Icon selection and product display:] When a user clicks an icon
by pressing the left button of the mouse, the system displays a setof products in the group corresponding to the clicked icon, as
shown in Figure 2. The user can return to the icon display bypressing "return" button featured at the top of Figure 2.
["Prefer" and "Delete" buttons:] While displaying a set of icons asshown in Figure 1, users can select particular icons by pointing
the cursor and pressing the right button of the mouse. Similarly,
users can select particular products while displaying them asshown in Figure 2. Then, the system shows a small dialog window
featuring "prefer" and "delete" buttons. When the user presses the"prefer" button, the icons those assigned keywords are similar tothe selected products are preferentially displayed later. On theother hand, when the user presses the "delete" button, the icons
those assigned keywords are similar to the selected products are
not often displayed later.

2.4 Icon selection algorithm
This system selectively displays a set of icons when a user presses"renew" button, as above mentioned. The displayed icons areautomatically selected by the following algorithm. If a user has
certain preferences to purchase items, the algorithm mainly selects
the user-preferred icons while intensively selecting several othericons so that users can be interested in other kinds of items.
Otherwise, the algorithm gradually learns preferences of users
while displaying variety of icons to prevent users confusing due totoo much randomized selection of icons. Our technique applies aninteractive evolutionary computation algorithm to selectpreferable sets of icons. This section calls previously displayed
icons "seed icons", and selected icons to be displayed "derived
icons". Moreover, this section formalizes the algorithm with thefollowing variables:n : Number of keywords.

m : Number of icons to be displayed simultaneously.

d  {d1 ,..., d n } : n-dimensional vector consisting of binary
values representing the assignment of the keywords for a
product. Here, d x  1 if the x-th keyword is assigned,otherwise d x  0 .



e  {e1 ,..., e n } : n-dimensional vector consisting of binary
values representing the assignment of the keywords for anq  {q1 ,..., q n } : n-dimensional vector representing thepreference of a user for each keyword. This section calls thisvalue "preference vector". Here, q i denotes the value of thepreference vector q when a user pressed "renew" button itimes.



A: The set of icons currently displayed.



S: The set of preferentially displayed icons.



C: The set of seed icons.

The processing flow for the icon selection is described below.
Here, Steps 1 to 6 are executed when a user presses "renew"
button.
Step0: Initialize the preference vector q 0 by the following
procedure.


Reset values of q 0 as 0 if it is for the first time for the user
to use this system.

Inherit the previous values if the user has a history of using
this system.

Step1: Let the set of seed icons C and icons currently displayed A
empty.
If the user selected particular keywords: The system firstly selectm/3 icons which one or more user-selected selected are assigned,
and insert the selected icons to S. Also, the system add predefined values to the particular dimensions corresponding to theuser-selected keywords of the preference vector q.
Selection of the icons to be registered in A:

If ms, the number of icons registered in S is smaller than m,
the system registers all the icons in S to A. Then, the system
randomly selects (m-ms) icons and registers to A, so that the
number of icons in A gets m.



If ms is equal to m or larger than m, the system randomly
extracts m icons from S and registers to A.

Step2: Display the icons registered in A. Users can freely click
the icons, look at the groups of products corresponding to theclicked icons, and press "preferred" or "deleted" buttons for the
displayed products.
Step3: The system applies Rocchio's algorithm [13] which are
originally applied for Relevance feedback. It renews the value ofthe preference vector q i by applying the following equation,
where i denotes how many times the user pressed "renew" button.

q i  q i 1 d   d    ed  Did Die  Ei

D  i : Set of products which user pressed "preferred" button after
the user pressed "renew" button i times.

D  i : Set of products which user pressed "deleted" button after
the user pressed "renew" button i times.

E  i : Set of displayed products after the user pressed "renew"

3. EXPERIMENT

button i times.

This section introduces our user experiments which demonstrate
the effectiveness of the presented system. All participants of theexperiments were female university students.

 : Constant value (   1.0 )The system preferentially selects the icons which similar sets of
keywords are assigned as the products the user pressed "preferred"button, by renewing q i by the following equation. Conversely,
the system does not willingly selects the icons which similar setsof keywords are assigned as the products the user pressed"deleted" button.
Step4: Insert the icons, those assigned keywords are entirely sameas the products which user pressed "preferred" button, to the set of
seed icons C.
Step5: Specify n s , the number of icons derived from a seed icon
registered in C. Our current implementation specifies n s as aconstant number proportional to the number of currently displayed
icons in A.

3.1 Effectiveness of user interfaces
We asked the participants to browse the presented system and an
existing EC (electric commerce) Web site for 15 minutes
respectively. We recorded their accesses, and counted thenumbers of the browsed products. Also, we asked them to write
down how they felt during the browsing products. Figure 4 shows
the numbers of browsed products by ten participants. This result
denotes that most of participants browsed larger number of
products while using our system rather than using the existing
Web site, as we aimed to develop the system so that users canenjoy to look at many products.

Step6: Execute the following procedure for each seed icon c  C .
(1)

Copy the e x value of c, and collect the set of keywords
those corresponding values e x for c is zero.

(2)

Select the y-th keyword from the set of keywords collectedin (1), and let e y  1 . Here, let the possibility to select they-th keyword as proportional to the corresponding value ofthe preference vector q y . Then, let the values e z zero, if

Proposed
Existing
0

100

200

300

Specify the icon corresponding to e z values specified in (2).
This process specifies an icon which just one of those
assigned keywords is different from c. Register the specified
icon to the set of derived icons S.

(4)

Repeat (2) to (3) n s times, for each derived icon in S.

(5)

Repeat (1) to (4) for each seed icon in C.

This algorithm preferentially selects icons those assigned
keywords are common with the products which user pressed"prefer" button. During the repetition of evolutionary computation
processes, this system selects the icons those just one of theassigned keywords are different from currently displayed icons as
the next icons. This mechanism is close to the feeling of real
shopping, because usually apparel shops arrange similar products
closely. We continuously look at similar products which just havedifferent colors or textures in apparel shops. We expect thismechanism brings users the feeling close to the real shopping.
On the other hand, this system often mixes small number ofrandomly selected icons, if ms is smaller than m in Step 1.
Keywords of the randomly selected icons are not always quite
common with the keywords of the products which the user pressed"prefer" button. This mechanism contributes to mainly show users'
preferred groups of products, while sometimes show wide variety
of other types of products. We expect this mechanism makes users
to satisfy their capricious curiosity to the variety of products,
without losing their ambiguous preferences or targets.

500

Figure 4 Numbers of browsed products by the participants.

the z-th keyword or the y-th keyword selected in (1) is
exclusively assigned to the icons.
(3)

400

Proposed
Existing

0

5

10

15

20

Figure 5 Numbers of keywords assigned to at least one of the
products which the participants pressed "prefer" button.
We also counted the numbers of keywords assigned to at least oneof the products which the participants pressed "prefer" button.
Figure 5 shows the numbers of keywords of the products which
the ten participants pressed "prefer" button. This result denotesthat the participants are interested in more variety of products
while using the presented system rather than using the existing
Web site. Though this result demonstrates the effectiveness of thepresented system that users were interested in variety of products,
we need to discuss that the result might depend on the preferencesof icon design of the participants. We heard that several
participants clicked particular icons because they felt these werecute. On the other hand, several participants skipped to click
particular icons because they did not prefer colors of them. We
found that icon design is an important factor for the effectivenessof the presented system, and need to discuss how to customize theicon design based on the preferences of users.

3.2 Effectiveness of icon selection algorithm
Next, we prepared two versions of the presented system. One ofthem (version A) featured the icon selection algorithm aspresented in this paper. The other (version B) just randomly
selected icons, while preferentially selecting icons which one ormore user-selected keywords were assigned. We evaluated if theicon selection algorithm worked as we expected to mainly show
users' preferred products, while sometimes show wide variety ofother types of products. In this experiment we focused how
participants were interested in products corresponding to therandomly selected icons, not the total number of preferred
products, while playing with the presented system. Also, we
focused how they found larger number of preferred products ifthey did not have any particular target products.

that eight of the participants archived the better ratios while using
the version A. It demonstrates that users can find larger number ofpreferable products while applying the icon selection algorithm,
when users do not have particular target products and therefore
they do not select any keywords.
Table 1Frequency of renew and number of preferred
products while using the version A of the presented system.
personrenew

Assigned

A

3

6

3

B

1

12

2

C

2

12

1

D

3

21

3

Experiment supposing particular targets.

E

5

3

4

We asked the participants to play with the two versions of thepresented system for 15 minutes respectively, after selecting
several keywords. Also, we asked them to write down how theyfelt during the browsing products.

Table 2Frequency of renew and number of preferred
products while using the version B of the presented system.

Tables 1 and 2 show the results with the five participants. In thetables, "person" denotes IDs of the participants, "renew" denotes
the frequency of pressing "renew" button, "assigned" denotes thenumber of products which user-selected keywords are assigned
and the participant pressed "prefer" button, and "other" denotes
the number of products which no user-selected keywords are
assigned and the participant pressed "prefer" button. This result
denotes all the participants were interested in the products which
no user-selected keywords were assigned, and pressed "prefer"button with the products, while playing with the version A. From
this result, it is effective to display icons which no user-selected
keywords are assigned, because the users may be interested insuch unexpected products. On the other hand, the participant E
pressed "prefer" button for five products which no user-selected
keywords were assigned while using the version B. We need to
discuss why the icon selection algorithm was not very effective
for the participant E comparing the random icon selection.
We received many positive comments for the version A. Many
participants mentioned "I was not tired by this system because it
suggested variety of products" or "the system motivated me tobrowse products we usually did not focus on." On the other hand,a participant mentioned "I was surprised because several icons I
did not prefer were displayed". We would like to solve thisproblem by developing additional indication which suggests users
to browse products corresponding to the icons selected by thealgorithm which no user-selected keywords are assigned.
Experiment without supposing particular targets.
We also asked the participants to play with the two versions of thepresented system without selecting any keywords. Here, it isdifficult the presented system precisely selects user-preferred
icons in a short experimental test without selecting any keywords,
because this system is based on a cold start algorithm. Therefore,
we asked them to firstly play with the two versions for 15 minutes.
We recorded the preference vector q, and then asked them to playwith the versions for 15 minutes again. The system loaded thepreference vector before the participants played with it.
We counted the numbers of products which the participants
pressed "prefer" button. Figure 6 shows the ratio between thenumbers of products which ten participants pressed "prefer"button and the numbers of browsed products. This figure denotes

otherpersonrenew

assigned

other

A

1

3

0

B

1

17

0

C

2

10

0

D

4

18

0

E

6

7

5

VersionB
VersionA

0

0.02

0.04

0.06

0.08

Figure 6 Ration of numbers of products which "prefer"button was pressed by the participants.

4. Related Work
Our original motivation for this research was to satisfy thepsychology of women's shopping activities. There have been
many publications on women's shopping activities [6,8]. We
remarked the conclusions regarding the characteristics of thewomen's shopping described in [8]. The system presented in thispaper does not limit the customers to only women; however, we
believe the system is especially preferred by women.
Various issues on on-line commerce business have been discussed
as a result of recent evolution, and actually several survey papers
on these issues have been published. For example, one of thesurvey papers [13] mentions that "personalization" and "positive
experience" are important issues. On the other hand, we do notthink that development of on-line commerce systems based on
women's shopping behavior or psychology has not beensignificantly discussed. Our study presented in this paper
addresses these issues.

behavior of the users. Consequently, the system mainly suggests
groups of products based on users' preferences, while it sometimes
suggests other kinds of products to satisfy the capricious curiosityof the users. We also expect users can enjoy a long time to selecttheir favorite products while the system quickly learn theirpreferences to solve the cold start problem of the recommendation
systems.

Recommendation is an important and effective technique forelectric commerce and contents viewing. Collaborative filtering[1,7] is one of the most famous and commercially successfulrecommendation techniques. We agree that collaborative filtering
would work well for apparel shopping or some of other kinds ofcommerce rather than content- or knowledge-based
recommendation techniques. However, we do not think such
recommendation systems are always satisfactory for apparel
shopping. One reason is that lifecycle of apparel products tends tobe short, and therefore we do not think cold start systems such as
collaborative filtering do not always work well. Another reason isthat customers may not want the recommendation systems torecommend the apparel products recommended to the users
themselves to other similar customers including their friends.
Consequently, we concluded to develop a more visual andinteractive system so that users can enjoy a long time for shopping,
and the system can gradually learn the preferences of the users
while their enjoyable time.

6. REFERENCES

Interactive evolutionary computing is an effective technique toquickly learn the preferences of the users and reuse the acquired
knowledge to various applications. Interactive genetic algorithm
has been applied to online-shopping [7], collaborative filtering [3]and music recommendation [9]. On the other hand, we preferred
to apply the relevance feedback [12] with Rocchio's algorithm,
because we expect this algorithm continuously recommends
slightly different sets of apparel products over and over. We
expect this behavior look similar to that we often look similar
products over and over in our real shopping.

[4] T. Itoh, C. Muelder, K.-L. Ma, J. Sese, A hybrid space-fillingand force-directed layout method for visualizing multiplecategory graphs, IEEE Pacific Visualization Symposium,
121-128, 2009.

Icon synthesis is an important portion in this research, because we
expect users enjoy shopping while selecting their interested icons.
There have been several techniques for semantics-based icon
synthesis [11]. Our icon creation policy is quite similar; however,
it is more specific to product keyword representation.
Rectangle placement problem is important for information
visualization. Various visualization techniques actually applies
rectangle packing, including tree visualization [5] and label
placement [2]. The technique we applied for the icon layout [4]has been originally developed for graph visualization; however, italso has preferable properties for our design of icon layout.

5. Conclusion
This paper presented an interactive exploratory search system
featuring a relevance feedback algorithm, and its implementation
for apparel shopping. Our study is inspired by the research of thepsychology and behavior of women's shopping. The presented
system firstly displays a variety of icons corresponding to thegroups of similar apparel products, so that users can keep wide
range of interests. It then displays the images of the products
corresponding to the icons selected by the users. This two-stepuser interface stages diversity and similarity of the products. We
think this representation is close to our real shopping: we often
look over the shops to understand the diversity of the products,
and then close up to particular groups of similar products tonarrow down the products we want to buy. Moreover, the system
automatically selects the icons to be displayed in the next stage,
by applying a relevance feedback algorithm reflecting the[1] J. S. Breese, D. Heckerman, C. Kadir, Empirical analysis ofpredictive algorithms for collaborative filtering, Uncertainty
in Artificial Intelligence, 43-52, 1998.
[2] J.-D. Fekete, C. Plaisant, Excentric labeling; dynamic
neighborhood labeling for data visualization, SIGCHI
conference on Human Factors in Computer Systems, 512-519,
1999.
[3] T, Hiroyasu, H. Yokouchi, M. Tanaka, M. Miki, Extractionof design variables using collaborative filtering forinteractive genetic algorithms, IEEE International
Conference on Fuzzy Systems, 1579-1584, 2009.

[5] B. Johnson, B. Shneiderman, Tree-maps: a space-filling
approach to the visualization of hierarchical information
structures, IEEE Visualization, 284-291, 1991.
[6] K.-J. Kim, H. Ahn, A recommender system using GA Kmeans clustering in an online shopping market, Expert
Systems with Applications, 34(2), 1200-1209, 2008.
[7] G. Linden, B. Smith, J. York, Amazon.com
recommendations: item-to-item collaborative filtering, IEEE
Internet Computing, Vol. 7, No. 1, p.76, 2003.
[8] H. Oda, Why women take a long time for shopping?, ISBN978-4-569-69428-3, 2007.
[9] Y. Saito, T. Itoh, MusiCube: a visual music recommendation
system featuring interactive evolutionary computing, Visual
Information Communication - Information Symposium(VINCI'11), 2011.
[10] G. Saltom, The SMART retrieval system - experiments in
automatic document processing, Prentice-Hall, 1971.
[11] V. Setlur, C. Albrecht-Buehler, A. Gooch, S. Rosoff, B.
Gooch, Semanticons: Visual Metaphors as File Icons,
Computer Graphics Forum, 24(3), 647-656, 2005.
[12] R. Yong, T. S. Huang, M. Ortega, S. Mehrotra, Relevance
feedback: a power tool for interactive content-based image
retrieval, IEEE Transactions on Circuits and Systems forVideo Technology, 8(5), 644-655, 1998.
[13] L. Zhou, L. Dai, D. Zhang, Online shopping acceptancemodel - a critical survey of customer factors in online
shopping, Journal of Electronic Commerce Research, 8(1),41-62, 2007.