# Use of the Smart Store for Persuasive Marketing and Immersive Customer Experiences: A Case Study of Korean Apparel Enterprise

Authors: Hyunwoo Hwangbo; Yang Sok Kim; Kyung Jin Cha
Journal: Mobile Information Systems
 (2017)

---

## Abstract

Information technology’s introduction of online retail has deeply influenced methods of doing business. However, offline retail has not changed as radically in comparison to online retailing. Recently, studies in computer science have suggested new technology that can support offline retailers, including sensors, indoor positioning, augmented reality, vision, and interactive systems. Retailers have recently shown interest in these technologies and rapidly adopted them in order to improve operational efficiency and customer experience in their retail shops. Marketing studies also address immersive marketing that employs these technologies in order to change ways of doing offline retail business. Even though there is much discussion concerning new trends, technologies, and marketing concepts, there is, as of yet, no investigation that comprehensively explains how they can be combined together seamlessly in the real world retail environment. This paper employs the term “smart store” to indicate retail stores equipped with these new technologies and modern marketing concepts. This paper aims to summarize discussions related to smart stores and their possible applications in a real business environment. Furthermore, we present a case study of a business that applies the smart store concept to its fashion retail shops in Korea.


---

## Body

## 1. Introduction

The introduction of online retailing has deeply influenced business. In 2014, eMarketer, a marketing company, estimated that retail sales reached $22.492 trillion and will increase to $28.300 trillion in 2018. Online retail sale will account for 5.9% of the total retail market worldwide in 2014 ($1.316 trillion) and will increase significantly to 8.8% in 2018 [1]. Even though online retail has drawn attention from companies, offline retail still accounts for a large proportion of whole retail market. However, offline retail has not yet been popular area of interest for technical innovation. Recently companies and researchers have paid attention to technologies, such as sensor, indoor positioning, augmented reality, vision, and interactive interface, which helped offline retail shops to improve their service quality. In this paper, we use the term “smart store” in order to describe offline retail shops that are equipped with these technologies and that create immersive, authentic user experiences for customers. Although the idea of the “smart store” can be applied to different types of retail stores, it appears that fashion retail stores are the most appropriate place, as businesses have an interest in learning and using customers’ behaviors, while customers have an interest in having a virtual experience with fashion items before physically trying them on. For this reason, many fashion companies, such as Uniqlo, Ralph Lauren, and Nike, have introduced different types of smart store applications [2–4].Despite their importance, there are no prior studies focusing on “smart stores.” This paper presents a comprehensive survey on the smart store applications within fashion retail industry and examines how various technologies can be integrated together to provide better customer services. This consists of the following content: firstly, two different views are summarized. There are many discussions on the smart store in many academic disciplines that are largely grouped into the marketing perspective and the technology perspective. Secondly, the goals of the “smart store” are identified from the marketing perspective: customer behavior analysis and customer experience enhancement. Thirdly, the technologies that are available to support the goals of the “smart store” are examined from the marketing perspective. This paper suggests that following five techniques are essential in the smart store: sensor, indoor positioning, augmented reality, vision, and interactive interface. Fourthly, previous industry applications developed by various companies are summarized, where each application is an incidental one and not an integrated one for a business model. Fifthly, we report our experience that shows different techniques can be combined for better customer service. Finally, we suggest a framework that fits into these techniques for different business purposes.


## 2. Goals of Smart Store from Marketing Perspective

### 2.1. Customer Behavior Analysis

Most marketing scholars suggest that the goal of the “smart store” is to understand customer behaviors in brick-and-mortar stores [5]. Prior marketing studies on retail business emphasize the importance of the smart store technologies and advanced analytics as they turn in-store customer behavior data into actionable insight [6, 7]. In addition, various studies suggest that Uniqlo, Zara, and other specialty store retailers of private label apparel (SPA) brands achieve success in their business because of their manufacturing excellence, process innovation in marketing and sales, and management of consumers’ dynamics with smart retail settings [8]. Uniqlo is an excellent example that demonstrates the potential of the smart store in the brick-and-mortar world. Although Uniqlo was a runner up in its e-commerce platform, it actively applied “smart store” technology, such as an in-store television and touch screen. Using marketing insights obtained from customer behavior analysis, it was able to assist customers in making appropriate purchase decisions and achieved an impressive growth in offline business. In the case of offline stores, the perspective that analyzes customer behaviors is similar to that of e-commerce sites, which track access records, clicking history, shopping cart, and series of purchases in order to understand customer behavior. In order to implement target marketing and persuasive marketing into brick-and-mortar stores, sensors, beacons, and other Internet of Everything (IoE) devices are employed to collect customer data and combine it with the purchased data.Recently all kinds of sensors have become small and affordable; thus companies are able to use them to accumulate and collect data relevant to customer behavior in brick-and-mortar stores. Advances in mathematical modeling of face recognition and biometrics make it possible for retailers to analyze dwell time, routing, and other behavioral aspects of in-store customers and link these analytic results with purchases.From the perspective of store operation, customer behavior analysis can be used to optimize store layout. One of the most important tasks of retail firms is to understand and apply various touch points at the moment-of-truth based on the customers’ journey in the store. In relation to this, many studies from various fields have emphasized the importance of indoor location analysis [5, 9–16]. Indoor location analysis aims to provide proximity marketing because it makes it possible for retail firms to collect data of customer behaviors and in-store movements in brick-and-mortar stores, which is similar to the online web-browsing analysis in e-commerce sites. Retail firms can also use this data to conduct store visits and path analysis in order to apply analysis results to implement store layout and store management strategies. This enables retail stores to implement various layout designs that can display companies’ strategic products of customers’ best interests. In addition, store managers can allocate store staff more efficiently and consider time and location in which the customers are interested. Moreover, through dwell time analysis, companies can conduct marketing active ties that can increase purchase probability. Dwell time and in-between time of the actual purchases in stores are correlated [17], and such tendency strengthens primarily in regard to dwell time in specific store zones and purchases of the products within relevant domains. Therefore, store managers can increase purchasing probability not by actively responding to the customers from the beginning, but by reacting to the acquired information according to the dwell time analysis. In addition, companies can link purchase analysis with indoor location analysis to develop personalized marketing applications [18].


### 2.2. Customer Experiences Management (CEM)

Many scholars that investigate retail industry emphasize the importance of customer experience management (CEM) [19–21]. CEM stands for internal and subjective response, which is obtained by customers via a firm’s direct or indirect contact [22]. In the retail environment, macro and firm controlled factors, such as promotion, price, merchandise, supply chain, and location, can lead to enhanced customer satisfaction, more frequent shopping visits, purchases, and profits, through superior customer experience [20]. In addition, through market place rituals of six key language categories, celebrating/commemorating, gift giving, greeting, inciting, parting, and edifying store’s employees, can enhance customer experience [21]. However, recent studies focus on enhancing customer satisfaction by providing new shopping trip experiences or bodily experiences to customers within stores, by using smart retail settings in addition to the methods that improve customer experience, and by improving interactions between customers and store employees [21, 23, 24]. The retail industry does not have a long history in considering technology on the full-scale as elements to enhance customer experience. Parasuraman [25] adds technology as the mediator to the two-dimensional “triangle model,” which explains interactions between company, customers, and employees. This leads to the suggestion of three-dimensional “pyramid model,” after which various researchers point out that self-service technologies play influential roles to the customers’ intentions [26–28] and customer satisfaction [29, 30]. In particular, Verhoef et al. [21] suggest separating the means that retailers use to provide customer experience into employee-based service and self-service technologies based service and point out the recent increase in the provision of blended services that include a mixture of the two categories. In addition, they point out that future research should proceed with investigations about how technology based service systems influence customers’ shopping experience. Moreover, their research aligns with our research agenda as they claim that customers should experience new shopping experience through a smart retail setting in brick-and-mortar stores. In this regard, Schmitt [31] categorizes human experiences into five keywords: sense, feeling, thinking, acting, and relating, and proposes that they should be organized and implemented by managers in order to achieve marketing strategy and objectives. In the same line, Naylor et al. [19] discover that transformational appeals influence consumers’ initial experiences by upgrading hedonic and symbolic benefits, after analyzing the influence of the consumer’s actual experiences to the transformational appeals by comparing information appeals in the retail sector. Therefore, the retailers can boost their sales by adopting smart retail settings and through improved customer experience and prior purchase exchanges. In addition, the study also claims that emphasis should be put on the experience because customers are satisfied simply with a better quality of previous products. For this reason, companies’ process of product consumption should emboss pleasure or sense of accomplishment as core elements.


## 2.1. Customer Behavior Analysis

Most marketing scholars suggest that the goal of the “smart store” is to understand customer behaviors in brick-and-mortar stores [5]. Prior marketing studies on retail business emphasize the importance of the smart store technologies and advanced analytics as they turn in-store customer behavior data into actionable insight [6, 7]. In addition, various studies suggest that Uniqlo, Zara, and other specialty store retailers of private label apparel (SPA) brands achieve success in their business because of their manufacturing excellence, process innovation in marketing and sales, and management of consumers’ dynamics with smart retail settings [8]. Uniqlo is an excellent example that demonstrates the potential of the smart store in the brick-and-mortar world. Although Uniqlo was a runner up in its e-commerce platform, it actively applied “smart store” technology, such as an in-store television and touch screen. Using marketing insights obtained from customer behavior analysis, it was able to assist customers in making appropriate purchase decisions and achieved an impressive growth in offline business. In the case of offline stores, the perspective that analyzes customer behaviors is similar to that of e-commerce sites, which track access records, clicking history, shopping cart, and series of purchases in order to understand customer behavior. In order to implement target marketing and persuasive marketing into brick-and-mortar stores, sensors, beacons, and other Internet of Everything (IoE) devices are employed to collect customer data and combine it with the purchased data.Recently all kinds of sensors have become small and affordable; thus companies are able to use them to accumulate and collect data relevant to customer behavior in brick-and-mortar stores. Advances in mathematical modeling of face recognition and biometrics make it possible for retailers to analyze dwell time, routing, and other behavioral aspects of in-store customers and link these analytic results with purchases.From the perspective of store operation, customer behavior analysis can be used to optimize store layout. One of the most important tasks of retail firms is to understand and apply various touch points at the moment-of-truth based on the customers’ journey in the store. In relation to this, many studies from various fields have emphasized the importance of indoor location analysis [5, 9–16]. Indoor location analysis aims to provide proximity marketing because it makes it possible for retail firms to collect data of customer behaviors and in-store movements in brick-and-mortar stores, which is similar to the online web-browsing analysis in e-commerce sites. Retail firms can also use this data to conduct store visits and path analysis in order to apply analysis results to implement store layout and store management strategies. This enables retail stores to implement various layout designs that can display companies’ strategic products of customers’ best interests. In addition, store managers can allocate store staff more efficiently and consider time and location in which the customers are interested. Moreover, through dwell time analysis, companies can conduct marketing active ties that can increase purchase probability. Dwell time and in-between time of the actual purchases in stores are correlated [17], and such tendency strengthens primarily in regard to dwell time in specific store zones and purchases of the products within relevant domains. Therefore, store managers can increase purchasing probability not by actively responding to the customers from the beginning, but by reacting to the acquired information according to the dwell time analysis. In addition, companies can link purchase analysis with indoor location analysis to develop personalized marketing applications [18].


## 2.2. Customer Experiences Management (CEM)

Many scholars that investigate retail industry emphasize the importance of customer experience management (CEM) [19–21]. CEM stands for internal and subjective response, which is obtained by customers via a firm’s direct or indirect contact [22]. In the retail environment, macro and firm controlled factors, such as promotion, price, merchandise, supply chain, and location, can lead to enhanced customer satisfaction, more frequent shopping visits, purchases, and profits, through superior customer experience [20]. In addition, through market place rituals of six key language categories, celebrating/commemorating, gift giving, greeting, inciting, parting, and edifying store’s employees, can enhance customer experience [21]. However, recent studies focus on enhancing customer satisfaction by providing new shopping trip experiences or bodily experiences to customers within stores, by using smart retail settings in addition to the methods that improve customer experience, and by improving interactions between customers and store employees [21, 23, 24]. The retail industry does not have a long history in considering technology on the full-scale as elements to enhance customer experience. Parasuraman [25] adds technology as the mediator to the two-dimensional “triangle model,” which explains interactions between company, customers, and employees. This leads to the suggestion of three-dimensional “pyramid model,” after which various researchers point out that self-service technologies play influential roles to the customers’ intentions [26–28] and customer satisfaction [29, 30]. In particular, Verhoef et al. [21] suggest separating the means that retailers use to provide customer experience into employee-based service and self-service technologies based service and point out the recent increase in the provision of blended services that include a mixture of the two categories. In addition, they point out that future research should proceed with investigations about how technology based service systems influence customers’ shopping experience. Moreover, their research aligns with our research agenda as they claim that customers should experience new shopping experience through a smart retail setting in brick-and-mortar stores. In this regard, Schmitt [31] categorizes human experiences into five keywords: sense, feeling, thinking, acting, and relating, and proposes that they should be organized and implemented by managers in order to achieve marketing strategy and objectives. In the same line, Naylor et al. [19] discover that transformational appeals influence consumers’ initial experiences by upgrading hedonic and symbolic benefits, after analyzing the influence of the consumer’s actual experiences to the transformational appeals by comparing information appeals in the retail sector. Therefore, the retailers can boost their sales by adopting smart retail settings and through improved customer experience and prior purchase exchanges. In addition, the study also claims that emphasis should be put on the experience because customers are satisfied simply with a better quality of previous products. For this reason, companies’ process of product consumption should emboss pleasure or sense of accomplishment as core elements.


## 3. Technologies for Smart Store

The transition from brick-and-mortar store to smart store is well identified in advanced retail stores, where they are based on the various technologies that support the smart store. Data collected through sensors installed inside and outside stores are used to make optimal suggestions to customers. In addition, customer behavior analysis provides customers with a virtual experience about the future of retail business. We identified the following four technologies as the fundamentals of the smart store in the fashion industry: indoor positioning, augmented reality, facial recognition, and interactive digital signage. In the following section, we examine concepts, market size, technical configurations, and practical cases.


### 3.1. Indoor Positioning

Indoor positioning system (IPS) refers to technology that grasps local situation of objects or people that are situated within the building through application of waves, magnetic fields, acoustic signals, or other sensory information by mobile devices [32]. Existing satellite based global positioning system (GPS) or cellular triangulation technology based location-based services (LBS) have limitations when applied in roofed buildings primarily because of the signal attenuation in accordance with construction materials [5]. However, Wi-Fi, Bluetooth, radio frequency identification (RFID), near field communication (NFC) and other similar NFC technologies, accelerometers, gyroscopes, and other recent location-sensor technologies make it possible to track user behaviors in the brick-and-mortar stores. In particular, proximity sensors, accelerometers, ambient light sensors, moisture sensors, gyroscopes, compasses, and other developments of sensor technologies play a significant role in providing detailed data of a user’s behavior in a roofed building. In addition, development of microelectromechanical systems (MEMS) through sensor miniaturization advances the era of Internet of things installed in all objects. Although there is a large body of previous studies about location-based services [33, 34], recent research on indoor location applications of smart CCTV and enabling technologies has made a lot of progress [35–37]. For example, Yaeli et al. [5] analyze CCTV customer movements in the retail sector using IBM’s Presence Zones and visualize optimized store layout. Meanwhile, many recent studies have studied technological elements for indoor positioning implementation. In particular, many focus on object detection and motion tracking implementation research.There are two in-store analytics methods for tactical object detection: mobile device based radio frequency (RF) positioning techniques and video analysis. While RF positioning techniques are implemented with Wi-Fi-enabled devices or sensors and interactive mobile apps, video analysis is implemented with highly specialized cameras that monitor object movements [38]. Although there are issues of reflection and isolation because of internal walls existing within RF technology, recent interlocking of beacons installed indoors and in Bluetooth features embedded in smart phones allows RF technology to be effectively utilized to perceive user movements within a 10-meter distance. Various methods that measure people or objects location through approximating signal propagation enable vast utilization of trilateration (distance form anchors), triangulation (angle to anchors), and other inverse trigonometry based mathematical modeling. In addition, Bayesian statistical analysis [39] and other probabilistic models are recently being used.Motion detection uses objects for dynamic territory detection on sight by comparing scene captures. The motion detection process can be divided into subprocesses, such as background subtraction, temporal differencing, and optical flow estimation [35]. Consideration and the mean shift algorithm are of vast use in object tracking. Researchers also analyze customer behavior through motion patterns classification, gained from object tracking modules, for customer path analysis [6]. Customer path analysis employs various methods, such as maximum entropy, Markov mixture models, and dynamic time warping (DTW).Existing video sensor network systems, used for indoor positioning in public areas to assure citizens safety, also have use in tacking criminals and assure antitheft in stores. However, recent CCTV systems go beyond safety assurance. They have been utilized to investigate customers’ shopping behavior [35]. In addition, recently specialized firms like American Retailnext and Euclid Analytics and Finish Walkbase have been appearing in the market. They provide retailers with visitors’ analysis results, including floating population, number of visitors, dwell time, rate of revisits, and conversion rate. Retailers can sell their products more efficiently by testing customers’ convenience through eye tracking, utilization of mobile Wi-Fi, CCTV or other better measures, and analysis of in-store movement path, types of collecting customers, and floating population.


### 3.2. Augmented Reality

Augmented reality (AR) refers to the computer graphic technology that visualizes things that exist in the natural environment by combining computer generated sensory inputs, such as sound, video, graphics, or GPS data from the physical, real world environment. AR feels more realistic in comparison to virtual reality (VR) primarily because it looks like the overlap of the virtual with the real world through utilization of cameras, glasses, and so forth. Myron Krueger, a VR research, first introduced the concept of AR in 1970s and Thomas Caudell, a Boeing researcher, applied and used AR full-scale in 1990 [40]. In augmented reality, the computer vision techniques like video tracking are extensively used, primarily because it is important to match the real world with information provided from devices [41]. At first, fiducial markers or the optical flow of the camera image are perceived by using feature detection methods, such as corner detection, blob detection, edge detection, or thresholding [42, 43]. Then the real environment and virtual information are matched and provided to the user. Mathematical methods, such as projective geometry, geometric algebra, and nonlinear optimization have been used for this purpose. Recently a vast number of studies have been conducted to enhance the application of augmented reality based smart mirror in the retail business area [44–47].In the beginning, the smart mirror was a mirrored digital signage, which has broadcasting, storage computing power, and network capabilities. This allows customers to take a picture of their clothes from various angles and view it in a 360-degree view, compare their appearance with other clothes, and send it to other people. Therefore, they can evaluate different products in real time. Recently, however, smart mirrors have started to incorporate augmented reality technology and add virtual fitting functions [48]. This allows stores to show off clothes in colors or sizes that are not available at the store, thereby preventing customers from leaving due to lack of interest and maximizing the customer experience.Augmented reality was first developed for military, industrial, and medical applications, but now its use has been expanded to entertainment areas and commercial areas such as games. Recently fashion stores have begun to adopt augmented reality by utilizing augmented reality and smart mirror to provide virtual fitting and fitting recommendation services. Virtual fitting is a service that allows the user to check images visually when wearing various sizes before purchasing clothes. Customers can find out what clothes and size fit their needs by entering information about their body shape, such as their height, and their desired fit (comfort, fit, etc.). On the other hand, fitting recommendation service is a service that displays recommended clothing types and size when the customer enters information about their body shape such as height and weight. A virtual fitting service of “Memory Mirror” of Neiman Marcus is a representative example of this kind of service. At the 2014 National Retail Federation expo, MomoMi introduced “Memory Mirror” which allows for 360-degree viewing through video shooting, contains a social network service and virtual fitting function utilizing augmented reality [49]. Since then, Neiman Marcus, a luxury department store’s Walnut Creek California branch, has adopted it to help customers with fitting in their stores. “Memory Mirror” is a 180 cm tall mirror that can be used to illuminate the customer’s whole body. In order to use it, the customer first shoots and stores various angles such as front, side, and back views in front of the mirror for 7 seconds after fitting. The customers can compare multiple fittings with videos and photos after they have tried different clothes on. The customers can also store videos and photos in the cloud, send them to their smart phones, or share them with family, friends, and so on using social media such as Facebook. MemoMi’s “Memory Mirror” also features augmented reality, showing different color clothes in real time without having to change clothes [49].


### 3.3. Facial Recognition

Facial recognition is a technique for identifying or verifying a person from a digital image or video frame from a video source. Facial recognition is a field of biometrics with fingerprint and eye iris recognition [50, 51]. Facial recognition is widely used in areas such as security and retail, because it has the advantage of mass identification in public places such as airports and multiplexes. The traditional facial recognition algorithms are used to identify facial features by analyzing the position, size, and shape of the face and to normalize and compress the stored face image to create face data. The most commonly used method is based on template matching techniques [52]. The recognition algorithm employs geometric approach, which recognizes faces based on the distance or shape between features, and photometric approach. Principal methods for facial recognition include principal component analysis, linear discriminant analysis, elastic bunch graph matching, hidden Markov model, and multilinear subspace learning [50, 53]. However, the recognition rate of the traditional algorithm decreases according to brightness, angle, and wearing accessories [50].In recent years, 3-dimensional (3D) facial recognition, skin texture analysis, and thermal cameras have been introduced to support the weaknesses of traditional algorithms. 3D facial recognition technique uses 3D sensors to collect information about the shape of a face and identifies a person using features that appear on the surface of the face, such as eye sockets, nose, and chin [54, 55]. This approach has the advantage of being able to recognize faces without being affected by changes in lighting or range of viewing angles. Skin texture analysis is a technique of recognizing a person by transforming characteristic lines, patterns, and dots in human skin into a mathematical space [53]. Thermal cameras are used to remove hats, glasses, make-up, and detect actual hair shape, complementing the existing facial recognition barriers [56]. Notable software used for facial recognition include Apple’s iPhoto, Google’s Picasa, Adobe’s Photoshop Elements, and OpenCV.Facial recognition has been widely used for security systems in Australia, New Zealand, or the US. Recently, facial recognition techniques have been utilized in retail stores for marketing purposes through customer identification and segmentation. At the National Retail Federation (NRF) conference in 2012, Microsoft demonstrated an advertisement that offers customized products to customers in real time when customers move past the show windows using Kinect, a motion detection technology applied to the Xbox game console. It scans people passing through Kinect and suggests products that match customer’s preference in Windows Embedded POSReady 7 by guessing gender, height, weight, race, age, and so on. Whole Foods has implemented the “Smarter Cart” project based on Microsoft’s Kinect tablet, UPC scanner, RFID reader, and speech recognition. Whole Foods utilizes Kinect’s motion capture function to allow carts to follow each other as they move through the store and apply voice commands using speech recognition technology. The reader can recognize the product name and price and create and share a shopping list. The goods’ information is able to be shared automatically without paying for them. In addition to Whole Foods, Microsoft is pursuing a commerce application project based on Kinect’s Windows platform for over 300 companies by tracking customer behavior in 3D and analyzing customer behavior patterns. Kinect has uses in a variety of commerce-based services, including showing advertisements that offer customized products to the customers in real time and enabling them to experience products through a virtual dressing room. San Diego based Emotient analyzes facial expressions of visitors to the store through facial recognition software and types them as joy, anger, sadness, surprise, fear, disgust, and contempt. Stores can create a salesperson response manual based on customer response and use customer feedback for marketing [57].


### 3.4. Interactive Digital Signage

Digital signage is digital media that provides information, entertainment, and advertisement by installing a digital display, which can be remotely controlled through a network in a public or commercial space. Media interactivity is becoming more important as consumers’ media usage patterns shift from a haphazard mode to the active mode of selecting and controlling media and consuming multimedia at the same time. As the user interaction technologies, for instance, touch screen, motion detection, and image capture, evolve, the importance of interactive digital signage becomes highlighted. Interactive digital signage is an intelligent and convergent media providing contents through network. Recently, the development of digital signage has taken many forms. For example, it streams high-resolution images using large displays, provides customized advertisements through interaction with users, and supports disaster prevention through linkage with surrounding situation recognition systems. In addition, in the fashion industry, interactive digital signage is gaining attention as a customer acquisition method. It attracts customers’ attention outside of the retail market, and maintains the incoming customers by drawing customers’ attention.Digital signage consists of (1) hardware for display and media player, (2) software for contents creation, distribution, management, and operation, (3) wired and wireless network for Internet and communication, (4) content authoring technology, and (5) UI & UX technology. As related technologies develop and the importance of user interaction grows, the technical elements of digital signage are rapidly changing. In terms of hardware, OLED displays, immersive displays, and large-format displays are common and the compatibility of the media player with different types of terminals is improving. Software is evolving from the limited functionality of traditional media operations to intelligent software, including hardware control, content retrieval, big data, and spatial and situational analysis. The network reflects interdevice communication and contents transmission technology. In content transmission technology, development and personalization of realistic contents based on bidirectionality are highlighted. In UI & UX, service and product development based on emotion are becoming important. As human computer interaction (HCI) technology develops, multiuser interaction via mobile devices are becoming possible.Interactive digital signage is attempting to combine various technologies such as artificial intelligence, facial recognition, near field communication, and augmented reality. In addition, multitouch displays are used to enable multiple users at the same time. In retail stores, the use of showcases combining transparent LCD is increasing. Many companies also use NFC chips in digital signage or use QR codes or barcodes to transmit personalized information or coupon to their smartphone. In recent years, there have been cases in which the digital signage of the situation is reflected in the actual situation of the place where the digital signage is installed. Interactive Digital Billboard, an outdoor billboard launched by British Airway in November 2014, is an interactive advertisement that shows the destinations of airplanes when young children in the billboard point to airplanes when they actually fly [58]. The digital signage installed in the subway history of the train senses the entry of the train into the station and shows the hair of the model in the digital signage to be scattered in the wind of the train effectively expressing the performance of the hair care product. Interactive digital signage, in this way, combines with the latest information technology to enhance consumer engagement [59].In the fashion industry, interactive display is implemented through interactive hangers, interactive fitting rooms, and interactive displays. Brazil’s fashion retailer C&A installed a digital hanger named “FashionLike” in its stores. In clothes hangers screen, C&A’s online shopping mall displays the number of customers’ thumbs-up. This is similar to Facebook’s Like, allowing visitors to recognize the preferences of other people’s products. This provides a selection recommendation for purchasing to customers who do not have a certain preference or hesitate to purchase between several products [60]. Ralph Lauren installed an interactive fitting room called Oak Fitting Room in Oak Lab in a flagship store on Fifth Avenue in Manhattan, New York, in 2015. Customers can request different sizes by pressing the buttons installed in the fitting room, and they can recommend clothing that matches the selected item. In the interactive fitting room, when a user brings in a product they want to wear, they know which product they have selected through the RFID information. The mirror is also touch screen, meaning the store clerk can be asked to adjust the lighting and find a related product or if you want different size, the store clerk can be notified with the press of one button. In addition, if the customer wants to buy a product later, they are able to buy it via mobile shopping through SMS [61]. In 2012, Adidas installed an interactive digital window on its customer’s smartphone without having to install a mobile application or scan a QR code in the Nürnberg NEO Label store in Germany. This storefront window is responsible for translating the virtual store. Customers can drag life-size images of products using the intuitive interface of the touch screen window to put their favorite products on their smartphones and pay for them on “Adidas NEO online.” Customers can also touch up hotspots on the window to view detailed information about the product or view a moving image wearing the pertinent product [62].


## 3.1. Indoor Positioning

Indoor positioning system (IPS) refers to technology that grasps local situation of objects or people that are situated within the building through application of waves, magnetic fields, acoustic signals, or other sensory information by mobile devices [32]. Existing satellite based global positioning system (GPS) or cellular triangulation technology based location-based services (LBS) have limitations when applied in roofed buildings primarily because of the signal attenuation in accordance with construction materials [5]. However, Wi-Fi, Bluetooth, radio frequency identification (RFID), near field communication (NFC) and other similar NFC technologies, accelerometers, gyroscopes, and other recent location-sensor technologies make it possible to track user behaviors in the brick-and-mortar stores. In particular, proximity sensors, accelerometers, ambient light sensors, moisture sensors, gyroscopes, compasses, and other developments of sensor technologies play a significant role in providing detailed data of a user’s behavior in a roofed building. In addition, development of microelectromechanical systems (MEMS) through sensor miniaturization advances the era of Internet of things installed in all objects. Although there is a large body of previous studies about location-based services [33, 34], recent research on indoor location applications of smart CCTV and enabling technologies has made a lot of progress [35–37]. For example, Yaeli et al. [5] analyze CCTV customer movements in the retail sector using IBM’s Presence Zones and visualize optimized store layout. Meanwhile, many recent studies have studied technological elements for indoor positioning implementation. In particular, many focus on object detection and motion tracking implementation research.There are two in-store analytics methods for tactical object detection: mobile device based radio frequency (RF) positioning techniques and video analysis. While RF positioning techniques are implemented with Wi-Fi-enabled devices or sensors and interactive mobile apps, video analysis is implemented with highly specialized cameras that monitor object movements [38]. Although there are issues of reflection and isolation because of internal walls existing within RF technology, recent interlocking of beacons installed indoors and in Bluetooth features embedded in smart phones allows RF technology to be effectively utilized to perceive user movements within a 10-meter distance. Various methods that measure people or objects location through approximating signal propagation enable vast utilization of trilateration (distance form anchors), triangulation (angle to anchors), and other inverse trigonometry based mathematical modeling. In addition, Bayesian statistical analysis [39] and other probabilistic models are recently being used.Motion detection uses objects for dynamic territory detection on sight by comparing scene captures. The motion detection process can be divided into subprocesses, such as background subtraction, temporal differencing, and optical flow estimation [35]. Consideration and the mean shift algorithm are of vast use in object tracking. Researchers also analyze customer behavior through motion patterns classification, gained from object tracking modules, for customer path analysis [6]. Customer path analysis employs various methods, such as maximum entropy, Markov mixture models, and dynamic time warping (DTW).Existing video sensor network systems, used for indoor positioning in public areas to assure citizens safety, also have use in tacking criminals and assure antitheft in stores. However, recent CCTV systems go beyond safety assurance. They have been utilized to investigate customers’ shopping behavior [35]. In addition, recently specialized firms like American Retailnext and Euclid Analytics and Finish Walkbase have been appearing in the market. They provide retailers with visitors’ analysis results, including floating population, number of visitors, dwell time, rate of revisits, and conversion rate. Retailers can sell their products more efficiently by testing customers’ convenience through eye tracking, utilization of mobile Wi-Fi, CCTV or other better measures, and analysis of in-store movement path, types of collecting customers, and floating population.


## 3.2. Augmented Reality

Augmented reality (AR) refers to the computer graphic technology that visualizes things that exist in the natural environment by combining computer generated sensory inputs, such as sound, video, graphics, or GPS data from the physical, real world environment. AR feels more realistic in comparison to virtual reality (VR) primarily because it looks like the overlap of the virtual with the real world through utilization of cameras, glasses, and so forth. Myron Krueger, a VR research, first introduced the concept of AR in 1970s and Thomas Caudell, a Boeing researcher, applied and used AR full-scale in 1990 [40]. In augmented reality, the computer vision techniques like video tracking are extensively used, primarily because it is important to match the real world with information provided from devices [41]. At first, fiducial markers or the optical flow of the camera image are perceived by using feature detection methods, such as corner detection, blob detection, edge detection, or thresholding [42, 43]. Then the real environment and virtual information are matched and provided to the user. Mathematical methods, such as projective geometry, geometric algebra, and nonlinear optimization have been used for this purpose. Recently a vast number of studies have been conducted to enhance the application of augmented reality based smart mirror in the retail business area [44–47].In the beginning, the smart mirror was a mirrored digital signage, which has broadcasting, storage computing power, and network capabilities. This allows customers to take a picture of their clothes from various angles and view it in a 360-degree view, compare their appearance with other clothes, and send it to other people. Therefore, they can evaluate different products in real time. Recently, however, smart mirrors have started to incorporate augmented reality technology and add virtual fitting functions [48]. This allows stores to show off clothes in colors or sizes that are not available at the store, thereby preventing customers from leaving due to lack of interest and maximizing the customer experience.Augmented reality was first developed for military, industrial, and medical applications, but now its use has been expanded to entertainment areas and commercial areas such as games. Recently fashion stores have begun to adopt augmented reality by utilizing augmented reality and smart mirror to provide virtual fitting and fitting recommendation services. Virtual fitting is a service that allows the user to check images visually when wearing various sizes before purchasing clothes. Customers can find out what clothes and size fit their needs by entering information about their body shape, such as their height, and their desired fit (comfort, fit, etc.). On the other hand, fitting recommendation service is a service that displays recommended clothing types and size when the customer enters information about their body shape such as height and weight. A virtual fitting service of “Memory Mirror” of Neiman Marcus is a representative example of this kind of service. At the 2014 National Retail Federation expo, MomoMi introduced “Memory Mirror” which allows for 360-degree viewing through video shooting, contains a social network service and virtual fitting function utilizing augmented reality [49]. Since then, Neiman Marcus, a luxury department store’s Walnut Creek California branch, has adopted it to help customers with fitting in their stores. “Memory Mirror” is a 180 cm tall mirror that can be used to illuminate the customer’s whole body. In order to use it, the customer first shoots and stores various angles such as front, side, and back views in front of the mirror for 7 seconds after fitting. The customers can compare multiple fittings with videos and photos after they have tried different clothes on. The customers can also store videos and photos in the cloud, send them to their smart phones, or share them with family, friends, and so on using social media such as Facebook. MemoMi’s “Memory Mirror” also features augmented reality, showing different color clothes in real time without having to change clothes [49].


## 3.3. Facial Recognition

Facial recognition is a technique for identifying or verifying a person from a digital image or video frame from a video source. Facial recognition is a field of biometrics with fingerprint and eye iris recognition [50, 51]. Facial recognition is widely used in areas such as security and retail, because it has the advantage of mass identification in public places such as airports and multiplexes. The traditional facial recognition algorithms are used to identify facial features by analyzing the position, size, and shape of the face and to normalize and compress the stored face image to create face data. The most commonly used method is based on template matching techniques [52]. The recognition algorithm employs geometric approach, which recognizes faces based on the distance or shape between features, and photometric approach. Principal methods for facial recognition include principal component analysis, linear discriminant analysis, elastic bunch graph matching, hidden Markov model, and multilinear subspace learning [50, 53]. However, the recognition rate of the traditional algorithm decreases according to brightness, angle, and wearing accessories [50].In recent years, 3-dimensional (3D) facial recognition, skin texture analysis, and thermal cameras have been introduced to support the weaknesses of traditional algorithms. 3D facial recognition technique uses 3D sensors to collect information about the shape of a face and identifies a person using features that appear on the surface of the face, such as eye sockets, nose, and chin [54, 55]. This approach has the advantage of being able to recognize faces without being affected by changes in lighting or range of viewing angles. Skin texture analysis is a technique of recognizing a person by transforming characteristic lines, patterns, and dots in human skin into a mathematical space [53]. Thermal cameras are used to remove hats, glasses, make-up, and detect actual hair shape, complementing the existing facial recognition barriers [56]. Notable software used for facial recognition include Apple’s iPhoto, Google’s Picasa, Adobe’s Photoshop Elements, and OpenCV.Facial recognition has been widely used for security systems in Australia, New Zealand, or the US. Recently, facial recognition techniques have been utilized in retail stores for marketing purposes through customer identification and segmentation. At the National Retail Federation (NRF) conference in 2012, Microsoft demonstrated an advertisement that offers customized products to customers in real time when customers move past the show windows using Kinect, a motion detection technology applied to the Xbox game console. It scans people passing through Kinect and suggests products that match customer’s preference in Windows Embedded POSReady 7 by guessing gender, height, weight, race, age, and so on. Whole Foods has implemented the “Smarter Cart” project based on Microsoft’s Kinect tablet, UPC scanner, RFID reader, and speech recognition. Whole Foods utilizes Kinect’s motion capture function to allow carts to follow each other as they move through the store and apply voice commands using speech recognition technology. The reader can recognize the product name and price and create and share a shopping list. The goods’ information is able to be shared automatically without paying for them. In addition to Whole Foods, Microsoft is pursuing a commerce application project based on Kinect’s Windows platform for over 300 companies by tracking customer behavior in 3D and analyzing customer behavior patterns. Kinect has uses in a variety of commerce-based services, including showing advertisements that offer customized products to the customers in real time and enabling them to experience products through a virtual dressing room. San Diego based Emotient analyzes facial expressions of visitors to the store through facial recognition software and types them as joy, anger, sadness, surprise, fear, disgust, and contempt. Stores can create a salesperson response manual based on customer response and use customer feedback for marketing [57].


## 3.4. Interactive Digital Signage

Digital signage is digital media that provides information, entertainment, and advertisement by installing a digital display, which can be remotely controlled through a network in a public or commercial space. Media interactivity is becoming more important as consumers’ media usage patterns shift from a haphazard mode to the active mode of selecting and controlling media and consuming multimedia at the same time. As the user interaction technologies, for instance, touch screen, motion detection, and image capture, evolve, the importance of interactive digital signage becomes highlighted. Interactive digital signage is an intelligent and convergent media providing contents through network. Recently, the development of digital signage has taken many forms. For example, it streams high-resolution images using large displays, provides customized advertisements through interaction with users, and supports disaster prevention through linkage with surrounding situation recognition systems. In addition, in the fashion industry, interactive digital signage is gaining attention as a customer acquisition method. It attracts customers’ attention outside of the retail market, and maintains the incoming customers by drawing customers’ attention.Digital signage consists of (1) hardware for display and media player, (2) software for contents creation, distribution, management, and operation, (3) wired and wireless network for Internet and communication, (4) content authoring technology, and (5) UI & UX technology. As related technologies develop and the importance of user interaction grows, the technical elements of digital signage are rapidly changing. In terms of hardware, OLED displays, immersive displays, and large-format displays are common and the compatibility of the media player with different types of terminals is improving. Software is evolving from the limited functionality of traditional media operations to intelligent software, including hardware control, content retrieval, big data, and spatial and situational analysis. The network reflects interdevice communication and contents transmission technology. In content transmission technology, development and personalization of realistic contents based on bidirectionality are highlighted. In UI & UX, service and product development based on emotion are becoming important. As human computer interaction (HCI) technology develops, multiuser interaction via mobile devices are becoming possible.Interactive digital signage is attempting to combine various technologies such as artificial intelligence, facial recognition, near field communication, and augmented reality. In addition, multitouch displays are used to enable multiple users at the same time. In retail stores, the use of showcases combining transparent LCD is increasing. Many companies also use NFC chips in digital signage or use QR codes or barcodes to transmit personalized information or coupon to their smartphone. In recent years, there have been cases in which the digital signage of the situation is reflected in the actual situation of the place where the digital signage is installed. Interactive Digital Billboard, an outdoor billboard launched by British Airway in November 2014, is an interactive advertisement that shows the destinations of airplanes when young children in the billboard point to airplanes when they actually fly [58]. The digital signage installed in the subway history of the train senses the entry of the train into the station and shows the hair of the model in the digital signage to be scattered in the wind of the train effectively expressing the performance of the hair care product. Interactive digital signage, in this way, combines with the latest information technology to enhance consumer engagement [59].In the fashion industry, interactive display is implemented through interactive hangers, interactive fitting rooms, and interactive displays. Brazil’s fashion retailer C&A installed a digital hanger named “FashionLike” in its stores. In clothes hangers screen, C&A’s online shopping mall displays the number of customers’ thumbs-up. This is similar to Facebook’s Like, allowing visitors to recognize the preferences of other people’s products. This provides a selection recommendation for purchasing to customers who do not have a certain preference or hesitate to purchase between several products [60]. Ralph Lauren installed an interactive fitting room called Oak Fitting Room in Oak Lab in a flagship store on Fifth Avenue in Manhattan, New York, in 2015. Customers can request different sizes by pressing the buttons installed in the fitting room, and they can recommend clothing that matches the selected item. In the interactive fitting room, when a user brings in a product they want to wear, they know which product they have selected through the RFID information. The mirror is also touch screen, meaning the store clerk can be asked to adjust the lighting and find a related product or if you want different size, the store clerk can be notified with the press of one button. In addition, if the customer wants to buy a product later, they are able to buy it via mobile shopping through SMS [61]. In 2012, Adidas installed an interactive digital window on its customer’s smartphone without having to install a mobile application or scan a QR code in the Nürnberg NEO Label store in Germany. This storefront window is responsible for translating the virtual store. Customers can drag life-size images of products using the intuitive interface of the touch screen window to put their favorite products on their smartphones and pay for them on “Adidas NEO online.” Customers can also touch up hotspots on the window to view detailed information about the product or view a moving image wearing the pertinent product [62].


## 4. Smart Store Framework

### 4.1. Solutions for Smart Stores

Company K, one of the leading companies in the Korean fashion industry, introduced three smart retail stores in the complex shopping mall in 2015. Company K has introduced six types of solutions for smart store implementation: smart CCTV, smart hanger, smart mirror, smart show windows, and smart shelf and smart showcase. Company K’s smart stores typify the underlying technologies of solutions: firstly, the smart CCTV has built-in people counting line analysis and area of interest analysis based on motion detection, object tracking, and facial recognition technology. The smart hanger has a built-in accelerometer to track user’s movements and to change digital display. The smart mirror provides a virtual fitting service based on augmented reality technology and provides a display in conjunction with the location of smart hanger, smart shelf, and so on. The smart show windows perform facial recognition using Kinect and beacon and provide product recommendation based on the user’s gender and age in interactive display. The smart shelf and showcase track the position of the product based on the beacon with built-in accelerometer and display the user’s interests on the digital signage via the infrared touch screen. Table1 summarizes the technology utilized by the smart retail solutions in the three smart stores of Company K, and Table 2 shows its detailed functions, application solutions, and specifications.Table 1
Solution-specific applied technology.
Technology
Solution
Smart CCTV
Smart hanger
Smart mirror
Smart show window
Smart shelf & showcase
Indoor positioning
V(motion detection, object racking)
V(beacon, accelerometer)
V(beacon)
V(beacon, motion detection)
V(beacon,accelerometer)
Augmented reality
V(camera)
V(camera)
Facial recognition
V(facial recognition)
V(facial recognition)
Interactive digital signage
V
V(barcode)
V(infrared touch)
V(infrared touch)Table 2
IoT technologies for K’s smart stores.
IoT technology
Function
Applied solution
Movement path analysis
It captures the in-stream population of the store and collects information such as staying in a specific place for a long time or coming into a main line
✓ Smart CCTV✓ Wearable device
Beacon
It provides unique location signals for each product to provide relevant information from nearby equipment and your smart phone
✓ Smart hanger✓ Smart mirror ✓ Smart show window✓ Smart shelf✓ Smartphone application
Accelerometer
By detecting the movement of objects, the customer recognizes the action of picking up or dropping the product
✓ Smart hanger✓ Smart shelf✓ Smart showcase
Motion detection
It provides interactive user experience with equipment through motion recognition in places where touch operation is impossible
✓ Smart show window
Facial recognition
Face recognition and age/gender analysis
✓ Smart CCTV✓ Smart show window
Camera
It provides the customer’s image to the screen and the fitting contents through the shooting/recording function
✓ Smart mirror
Near infrared sensor
It provides the ability to recognize the presence or absence of customers in front of objects
✓ Smart mirror
Barcode sensor
It recognizes the barcode in the tag of the product and provides the information of the product
✓ Smart mirror
IR touch sensor
Through a touch function of the display glass, it provides the operation function that allows the user to view the information of the product even if the customer does not take out the product
✓ Smart showcase
iWatch
Receiving a variety of notifications via watch
✓ Wearable device
Smart pad
In addition to simple alarms, it provides services that enable promoting products in connection with things
✓ Smart pad
Integrated control solution
Remotely managing various Internet of things equipment and distributing and operating each content
✓ Smart CCTV✓ Smart hanger✓ Smart mirror ✓ Smart show window✓ Smart shelf✓ Wearable device✓ Smart pad


### 4.2. Components of Smart Stores

Table3 shows the smart retail settings of Company K in the flagship stores of men’s clothing of S brand, women’s clothing of L brand, handbags, and wallets women’s accessory C brand.Table 3
Example of K’s smart stores.
Brand
Solution
Smart CCTV
Smart hanger
Smart mirror
Smart show window
Smart shelf & showcase
S brand(menswear)
V
V
V
V
L brand(petticoat)
V
V
V
V
C brand(accessories)
V
V
VL brand has installed 6 smart CCTVs for people counting, face recognition, and zone analysis. In addition, it analyzes customers’ behavior through smart hangers, smart mirrors, and smart show windows and provides a new shopping experience to customers. The layout of L brand emulated applications is shown in Figure1.Figure 1
Layout of L brand’s smart store.As shown in Figure2, smart show windows are responsible for customer acquisition from outside the store, while the other five solutions perform customer retention in the store based on its installation location. We use these solutions to enhance customer behavior analysis and customer experience in terms of marketing. Smart CCTV is used for target marketing, smart show window, smart shelf, and smart showcase for experiential marketing. Meanwhile, smart hanger and smart mirror will enhance brand accessibility by providing new experiences to customers who visit first. It plays a role of increasing the purchase conversion rate. The accumulated customer behavior is used to provide personalized product recommendation and store layout optimization through data analysis.Figure 2
Solutions for K’s smart stores.Company K’s smart stores are implemented as applications for customers and employees based on technologies such as indoor positioning, augmented reality, facial recognition, and interactive digital signage. Applications for customers include smart hangers, smart shelves and showcases, smart mirrors, smart show windows, and smartphone applications, and applications for employee include smart CCTVs, smart pads, and wearable devices. Customers who visit the store receives stimuli through shopping experiences such as viewing, listening, and touching smart retail settings. Afterwards, customers interact with applications to gain new experiences through cognition or applications that gain confidence in new information or purchases. The customer has an intention regarding shopping through the step of enjoying consciousness. This can be expressed as a customer’s inquiry about the product to purchase, a revisit to find the store again, and a purchase to buy the goods. A series of decision-related processes can be interpreted through the Stimulus-Organism-Response (SOR) framework [63].Customers who visit the store will be able to get detailed information about the product, such as material, price, and review, on the display installed in the vicinity as they touch, browse, and move items in smart hanger or smart shelf and showcase. Customers can use the smart mirror to compare various products at the same time through virtual fitting with 360-degree vogue and augmented reality, or to assist with purchase decisions by communicating with neighboring acquaintances. The show window’s gamification function stops the customer from stepping out of the store, and the display window of the show window is turned off. In addition, smartphone applications enable location-based marketing by sending coupons or event information when a customer is near a store, allowing customers can use them. It is possible to experience Omnichannel marketing by storing items of interest in the offline store in the shopping cart of the online shopping mall or by inquiring where the items of interest in the online shopping mall are located in the offline store. Retail applications provide cognition to the demanding shopper and consciousness to the entertainment shopper to guide the customer’s purchase and related decisions. The ultimate goal of a smart store in the retail industry is to attract its customers to purchase products. The purchase process in the marketing perspective consists of three activities: consulting the customer to have intention to purchase, inducing the customers to revisit the store, and persuading the customers to purchase the product. This can be measured by actual resource expenditure (e.g., product purchase), perceived resource expenditure (e.g., willingness to pay), and satisfaction (e.g., purchase intention). Smart retail settings give consciousness composed of fantasy, imagery, and creative play to the entertainment customers and give cognition to the demanding customers composed of direct interaction, secondary source information, and intentional belief through customer experience enhancement at the organization layer. These two enable the customer move from the organism layer to the response layer. Figure3 summarizes the customer shopping experience through applications applied to Company K’s smart store.Figure 3
Application of the SOR framework.


## 4.1. Solutions for Smart Stores

Company K, one of the leading companies in the Korean fashion industry, introduced three smart retail stores in the complex shopping mall in 2015. Company K has introduced six types of solutions for smart store implementation: smart CCTV, smart hanger, smart mirror, smart show windows, and smart shelf and smart showcase. Company K’s smart stores typify the underlying technologies of solutions: firstly, the smart CCTV has built-in people counting line analysis and area of interest analysis based on motion detection, object tracking, and facial recognition technology. The smart hanger has a built-in accelerometer to track user’s movements and to change digital display. The smart mirror provides a virtual fitting service based on augmented reality technology and provides a display in conjunction with the location of smart hanger, smart shelf, and so on. The smart show windows perform facial recognition using Kinect and beacon and provide product recommendation based on the user’s gender and age in interactive display. The smart shelf and showcase track the position of the product based on the beacon with built-in accelerometer and display the user’s interests on the digital signage via the infrared touch screen. Table1 summarizes the technology utilized by the smart retail solutions in the three smart stores of Company K, and Table 2 shows its detailed functions, application solutions, and specifications.Table 1
Solution-specific applied technology.
Technology
Solution
Smart CCTV
Smart hanger
Smart mirror
Smart show window
Smart shelf & showcase
Indoor positioning
V(motion detection, object racking)
V(beacon, accelerometer)
V(beacon)
V(beacon, motion detection)
V(beacon,accelerometer)
Augmented reality
V(camera)
V(camera)
Facial recognition
V(facial recognition)
V(facial recognition)
Interactive digital signage
V
V(barcode)
V(infrared touch)
V(infrared touch)Table 2
IoT technologies for K’s smart stores.
IoT technology
Function
Applied solution
Movement path analysis
It captures the in-stream population of the store and collects information such as staying in a specific place for a long time or coming into a main line
✓ Smart CCTV✓ Wearable device
Beacon
It provides unique location signals for each product to provide relevant information from nearby equipment and your smart phone
✓ Smart hanger✓ Smart mirror ✓ Smart show window✓ Smart shelf✓ Smartphone application
Accelerometer
By detecting the movement of objects, the customer recognizes the action of picking up or dropping the product
✓ Smart hanger✓ Smart shelf✓ Smart showcase
Motion detection
It provides interactive user experience with equipment through motion recognition in places where touch operation is impossible
✓ Smart show window
Facial recognition
Face recognition and age/gender analysis
✓ Smart CCTV✓ Smart show window
Camera
It provides the customer’s image to the screen and the fitting contents through the shooting/recording function
✓ Smart mirror
Near infrared sensor
It provides the ability to recognize the presence or absence of customers in front of objects
✓ Smart mirror
Barcode sensor
It recognizes the barcode in the tag of the product and provides the information of the product
✓ Smart mirror
IR touch sensor
Through a touch function of the display glass, it provides the operation function that allows the user to view the information of the product even if the customer does not take out the product
✓ Smart showcase
iWatch
Receiving a variety of notifications via watch
✓ Wearable device
Smart pad
In addition to simple alarms, it provides services that enable promoting products in connection with things
✓ Smart pad
Integrated control solution
Remotely managing various Internet of things equipment and distributing and operating each content
✓ Smart CCTV✓ Smart hanger✓ Smart mirror ✓ Smart show window✓ Smart shelf✓ Wearable device✓ Smart pad


## 4.2. Components of Smart Stores

Table3 shows the smart retail settings of Company K in the flagship stores of men’s clothing of S brand, women’s clothing of L brand, handbags, and wallets women’s accessory C brand.Table 3
Example of K’s smart stores.
Brand
Solution
Smart CCTV
Smart hanger
Smart mirror
Smart show window
Smart shelf & showcase
S brand(menswear)
V
V
V
V
L brand(petticoat)
V
V
V
V
C brand(accessories)
V
V
VL brand has installed 6 smart CCTVs for people counting, face recognition, and zone analysis. In addition, it analyzes customers’ behavior through smart hangers, smart mirrors, and smart show windows and provides a new shopping experience to customers. The layout of L brand emulated applications is shown in Figure1.Figure 1
Layout of L brand’s smart store.As shown in Figure2, smart show windows are responsible for customer acquisition from outside the store, while the other five solutions perform customer retention in the store based on its installation location. We use these solutions to enhance customer behavior analysis and customer experience in terms of marketing. Smart CCTV is used for target marketing, smart show window, smart shelf, and smart showcase for experiential marketing. Meanwhile, smart hanger and smart mirror will enhance brand accessibility by providing new experiences to customers who visit first. It plays a role of increasing the purchase conversion rate. The accumulated customer behavior is used to provide personalized product recommendation and store layout optimization through data analysis.Figure 2
Solutions for K’s smart stores.Company K’s smart stores are implemented as applications for customers and employees based on technologies such as indoor positioning, augmented reality, facial recognition, and interactive digital signage. Applications for customers include smart hangers, smart shelves and showcases, smart mirrors, smart show windows, and smartphone applications, and applications for employee include smart CCTVs, smart pads, and wearable devices. Customers who visit the store receives stimuli through shopping experiences such as viewing, listening, and touching smart retail settings. Afterwards, customers interact with applications to gain new experiences through cognition or applications that gain confidence in new information or purchases. The customer has an intention regarding shopping through the step of enjoying consciousness. This can be expressed as a customer’s inquiry about the product to purchase, a revisit to find the store again, and a purchase to buy the goods. A series of decision-related processes can be interpreted through the Stimulus-Organism-Response (SOR) framework [63].Customers who visit the store will be able to get detailed information about the product, such as material, price, and review, on the display installed in the vicinity as they touch, browse, and move items in smart hanger or smart shelf and showcase. Customers can use the smart mirror to compare various products at the same time through virtual fitting with 360-degree vogue and augmented reality, or to assist with purchase decisions by communicating with neighboring acquaintances. The show window’s gamification function stops the customer from stepping out of the store, and the display window of the show window is turned off. In addition, smartphone applications enable location-based marketing by sending coupons or event information when a customer is near a store, allowing customers can use them. It is possible to experience Omnichannel marketing by storing items of interest in the offline store in the shopping cart of the online shopping mall or by inquiring where the items of interest in the online shopping mall are located in the offline store. Retail applications provide cognition to the demanding shopper and consciousness to the entertainment shopper to guide the customer’s purchase and related decisions. The ultimate goal of a smart store in the retail industry is to attract its customers to purchase products. The purchase process in the marketing perspective consists of three activities: consulting the customer to have intention to purchase, inducing the customers to revisit the store, and persuading the customers to purchase the product. This can be measured by actual resource expenditure (e.g., product purchase), perceived resource expenditure (e.g., willingness to pay), and satisfaction (e.g., purchase intention). Smart retail settings give consciousness composed of fantasy, imagery, and creative play to the entertainment customers and give cognition to the demanding customers composed of direct interaction, secondary source information, and intentional belief through customer experience enhancement at the organization layer. These two enable the customer move from the organism layer to the response layer. Figure3 summarizes the customer shopping experience through applications applied to Company K’s smart store.Figure 3
Application of the SOR framework.


## 5. Components of Smart Stores

### 5.1. Smart CCTV

Company K has installed 4~6 smart CCTVs per store in 3 branded flagship stores to implement personalized marketing by analyzing user behavior in store through indoor positioning and facial recognition technology and implement store layout optimization. Smart CCTV performs the four following functions: people counting, facial recognition, customer movement analysis, and interest area analysis. First, Company K installed CCTV with sensors to measure the number of visitors at all outlets of the store. By measuring the number of customers who entered the store, it is possible to measure the purchase conversion rate, which indicates the number of visitors per day and per hour, average residence time of customers, and number of buyers compared to visitors. Therefore, it is possible to plan an efficient operation of employees. Secondly, a facial recognition sensor was attached to the CCTV installed at the main entrance of the store. As a result, it became possible to measure the number of unique visitors by solving duplicate visitors. Thirdly, the company installed CCTV for each major area of the store, and logged customers’ travel route and residence time in the store. By analyzing customer shopping lines, it is made possible to improve the store layout, to increase the efficiency of the use of the store area, and to relocate the products in the store, thereby inducing the strategic purchase of products. Fourthly, Company K measured the residence time at a certain point through CCTV and introduced a system to send a notification message to the shop staff when customers stay at a specific spot for more than 30 seconds. Customers are prevented from leaving due to lack of proper guidance even though they are interested in the product, and the store personnel can understand the customer’s interest area and respond to the customers rapidly.As shown in Figure4, it becomes possible to evaluate the movement of people, the value of the store area, and the attractiveness of the goods objectively, whereas previously this was only identified subjectively by store personnel. Furthermore, effects of promotions and events can be measured and utilized to encourage the purchases of the viewer.Figure 4
Operation of people counting sensor.The details of the technology applied for smart CCTV are as follows: firstly, sensor cameras are installed on the ceiling of all entrances in the facility to count customers. Then a counting area is set inside the entrance, so that only those who are out of the area are counted. Secondly, a facial recognition system is installed through five sensor cameras on the ceiling of the main entrance in order to get five face images of the customer through the processor. The supporting hardware consists of a camera module, a twist unit (TWU), and a processor. TWU converts video signals into differential signals and the process encrypts and transmits upper body photographs to a management PC. In order to protect the privacy of the customer, the management PC extracts the attributes for the face recognition from the pictures and deletes the photographs. The software of GikenTrastem provides the gender and the age resolution. The test results showed an accuracy of about 80%. However, the quality of recognition accuracy depends on the mask or face direction. Thirdly, the analysis of the customer movement and the analysis of the interest area are performed by measuring the travelling time of the customer through the sensor installed on the ceiling and the time spent at the specific location and time recording. In particular, Company K has introduced a system that notifies customers of the products they are interested in by sending signals to smart pad (iPad) and wearable device (iWatch).


### 5.2. Smart Hanger

Company K has installed a smart hanger in the flagship store of S and L brands to provide an intuitive user experience by using beacon and digital display as shown in Figure5. Smart hangers analyze products of interest, customer experience, and customer response. The implementation method and effect are as follows. Firstly, Company K can attach a beacon with an accelerometer on the smart hanger to measure how many customers were holding the hanger with the item and what distance they have moved with the hanger. The company can link the logs made by the smart hanger with the purchase data and analyze the link between the goods of interest and the purchase items. This is similar to linking e-commerce to click-through pages, shopping carts, and purchase items. Through this, Company K has been able to target marketing by tracking and analyzing the processes from the point of customer’s attention to the product to the process of purchasing. Secondly, Company K has smart hangers in conjunction with digital signage to provide customers with a new store experience. The display in standby mode shows the top three items and promotional images, such as model photos. However, if a customer holds a hanger with a beacon, the media pole digital display on the top of the hanger shows details of the product such as material and price and the number of customers who previously selected the product. In addition, when the customer takes the coat hanger to the mirror, details of the product are displayed in the mirror. This allows the customer to learn detailed information about the product without help of the store staff and make purchasing decisions based on the purchasing tendencies of other customers. Thirdly, smart hangers can also help retailers gather customer data by measuring and analyzing in-store behavior. The beacon embedded in smart hangers also enables smart pads and wearable devices used by the store staff to send messages related to products of customer interest, so that they can be prepared for invisible customer interaction.Figure 5
Smart hanger.Smart hangers consists of a hanger with an accelerometer-installed beacon, a 3-sided media pole, a video wall, a digital display, and an Android set-top box. Here, the accelerometer measures the time the customer picks up the hangers and movement of the hangers. The beacon attached to the smart hanger sends the signal to the digital display when the customer picks up the hanger to display detailed information about the product. The beacon of the smart hanger also sends a signal to the mirror to display the details of the product when the two are in close proximity. While this is happening, the store employee receives an alarm through a smart pad and wearable devices when the customer picks up the hanger. Customers can also receive detailed information on products related to the products they are interested in and a list of recommended products through the smart pad.


### 5.3. Smart Mirror

Company K has been providing virtual fitting and product recommendation services to customers by combining augmented reality and interactive display technology in mirrors installed in S, L, and C brands’ flagship stores (Figure6). Through this, Company K is providing new customer experiences to customers who visit stores and improving the purchase rate by recommending products optimized for customers.Figure 6
Smart mirror using augmented reality.The smart mirror provides beacon-related product information, virtual fitting, SNS sharing, on-offline shopping connection, and product recommendation. Firstly, the smart mirror displays details of selected items in conjunction with the smart hanger, enabling customers to shop more proactively without the help of clerks. When the customer holds the smart hanger and goes near the smart mirror, the sensor attached to the mirror recognizes it and displays the thumbnail image of the product in the mirror display. When the customer selects the thumbnail image, the smart mirror displays the related item in the mirror display along with the detailed information, such as the product name, price, size, product review, laundry guide, and material. Customers can also use the staff call feature in the mirror display to ask additional questions. Secondly, the smart mirror provides a virtual fitting function that shoots moving images at 360 degrees and changes the color of the video after shooting to estimate the wearing shot. This allows customers to see which color best suits their needs without having to try on individual items of clothing. Thirdly, the smart mirror stores the pictures and images it takes. From there, they can be transferred using MMS and QR codes to their mobile phones, transmit them to their friends, and provide the function of allowing their friends or SNS to evaluate their choices. This not only provides new shopping experiences to customers, but also increases the rate of conversion of purchases by receiving real time ratings from friends. Fourthly, besides the smart hanger, customers can tag product barcodes in the store with barcode sensors to access detailed information about products by tagging them on the smart mirrors. In addition, the customer can put the retrieved items in their shopping cart on the online shopping malls for stores. The smart mirror implements the Omnichannel marketing function, which is merely in its initial form but also able to extend to showrooming. For this purpose, Company K has installed a mirror display with a camera, a beacon sensor, and a bar code recognition sensor on the mirror display. We have also introduced software that includes an augmented reality function to provide virtual fitting. Meanwhile, Company K has developed a mobile application for online shopping interactions, so that customers can add items viewed offline to their shopping cart on the online shopping malls for stores.


### 5.4. Smart Show Window

As shown in Figures7 and 8, Company K has installed a smart interactive display in the show window outside stores for drawing customer attention through customer experience enhancement. Brand S has installed a show window with facial recognition and augmented reality technology, and brand L has installed a show window with a brochure function. Company K has developed an AR mannequin on the show window of brand S, which was built with a transparent display with the dual purpose of shop window. In order to maximize the effect of the transparent display device, the actual mannequin was also displayed inside. This mannequin is displayed in a virtual outfit chosen by the shop staff. Normally, the transparent display above the outfit on the mannequin displays information about the outfit, but the new technology allows the user to view detailed information about the outfit when they touch a button on the display. The user can also change the color of the garment, allowing them to experience an interactive display that uses augmented reality technology. In addition, shop staff can change the outfit worn on the mannequin using the iPad, meaning the display can be changed from time to time to suit seasonal fashion trends. By contrast, brand L’s show window brochure has added an interactive display function to the show window display of the store that shows the contents of the store as an interactive fashion pictorial. The store can use this to contain more displays on a limited show window screen and allows the user to choose a display based on their preferences. This type of display attracts potential customers’ attentions, encouraging them to linger outside the store for longer, as well as allowing customers to make decisions before entering the store and therefore increasing the ratio of purchase rate compared to visiting customers. In addition, this encourages customer engagement by enabling customers to capture photos and utilize them as pictorial models.Figure 7
AR mannequin on the smart show window.Figure 8
Show window brochure.Company K uses facial recognition technology to measure the age range of customers and provides customized product recommendation services. At the same time, Company K uses the gamification function to attract customers who pass by and utilizes them for customer acquisition. As shown in Figure9, Company K has installed a Kinect sensor with a facial recognition function in the mirror-type show window display of the store. This show window works like a mirror, but the Kinect sensor is activated when a potential user comes within 2 m. There are camera and proximity sensors installed behind the mirror glass which is invisible to the user. In reality, facial recognition is made possible by linking cameras in digital information display (DID) with NEC’s “Field Analyst for Signage” solution. Using this process, it is possible to measure up to 20 passengers passing at a common walking speed at the same time. Each customer’s age and gender are estimated through a process consisting of three combined modules: face detection, feature extraction, and face recognition. First, we used a geometry feature-based approach to analyze the features of eyes and nose and used the geometrical relationships between them, using graph matching algorithms, for face detection. Second, we extracted image (wrinkle) and shape (geometry) features of each face image for feature extraction. Finally, we used a support vector machine (SVM) classifier in the face recognition process to improve the accuracy of gender and age estimation systems using face images. We used a single linear SVM classifier trained using the feature vector representations for gender estimation. Because gender estimation is a binary classification that distinguishes between male and female and it is easy to obtain many learning data with accurate gender information, the SVM classifier achieves a high recognition rate of over 95%. On the other hand, it is difficult to improve accuracy in age estimation, because it is difficult to obtain learning data with accurate age information and there is no standard age estimation method. Therefore, we performed multilabel age classification and then performed one-versus-one linear SVM arrangement for each age group. Furthermore, it has also become possible to classify the floating population passing through the store and the customers visiting the store by gender (male/female) and age (1-year-old).Figure 9
Facial recognition using Kinect.This makes it possible to measure the effect of the show window display by measuring the residence time (display residence time) in front of the show window, the time of the gaze (screen gaze time), and the distance (distance between the screen and the person). When the upper illuminance is above 500 lux, the accuracy is more than 90% for flow population, 80% for sex, and 70~75% for age group. However, since privacy concerns exist in facial recognition [64], images and personal information are not stored in accordance with Article 25 (1) of the Personal Data Protection Act for the protection of personal information. Instead, data is stored as raw numeric and text data used for customer analysis. Stores use facial recognition technology to enhance the customer’s interest by providing appropriate games and provide product recommendation services based on information obtained. The customer can participate in a game (age estimation game) that calculates their age through the display installed in the store show window. The photographs taken here are used for customer segmentation by utilizing facial recognition technology and provides different product recommendation services appropriate to age and gender based on face recognition results through the smart window mirror display.


### 5.5. Smart Shelf and Showcase

As shown in Figures10 and 11, Company K has installed a smart shelf and smart showcase that combines indoor positioning and smart interactive display technology in brick-and-mortar stores for increased customer retention through customer experience enhancement. Tags on fashion accessories, such as handbags, have beacons attached to accelerometers attached to them. The operation of these tags are similar to smart hangers. When the user picks up a fashion accessory, the sensor attached to the wall display captures the movement of the beacon and displays product information. If several people pick up a product at the same time, this information is displayed on the mirror near to the product. It also provides an Omnichannel shopping function that links offline shopping items to online shopping carts in conjunction with online shopping malls. At the same time, brand C from Company K offers a new user experience by combining an interactive display with a showcase, which displays accessories such as wallets. Technically, it combines touch screen-based digital signage with existing showcases to attract visitors’ attention and allows customers to view detailed information about the products they are interested in without being directly involved. The smart showcase provides detailed information about the selected product such as material, release date, selling price, and related product information when the product shown on the display is touched or the attached accessory is placed on the RFID reader. This enables customers who do not feel comfortable asking store staff to inquire about the product without direct interaction. In addition, store staff are able to respond better to customers’ needs through receiving customers’ inquiry history on their iPad.Figure 10
Smart shelf using interactive display.Figure 11
Smart showcase using interactive display.


### 5.6. Smart Pad and Wearable Device

Company K utilizes information about customer preferences accumulated in the store and provides it to store staff who use smart pads (iPad) and wearable devices (iWatch) for efficient customer response. In addition, a dedicated application was developed to process data collected from various sensors and to provide customer information to shop staff. This enables shop staff to improve customers’ experiences in relation to customer identification, product recommendation, and store management. Firstly, store staff can focus on their customers and identify their needs without having to ask questions. Customer information collected from smart CCTV’s customer entry and point of interest notifications, smart hanger, and smart merchandise is transferred to shop staff through smart pads and wearable devices. This allows shop staff to understand customers’ interest areas and which products the customer are interested in. Secondly, analyzing integrated customer behavior data allows for efficient product recommendation. When a customer hesitates to purchase a product, the recommended product and the related product list on the smart pad may prevent the customer from leaving the store and provide functions that encourage additional purchases. In addition, the availability of information such as color, sales ranking, and inventory information on smart pads make it possible for store staff to provide quick response to the customers. Thirdly, the store staff can identify the location of the items in the store and the inventory quantity as well as a customer’s staying time and list of popular products through smart pad, thereby enabling efficient store management. In addition, shop staff can use the smart pad to change the display of the augmented reality mannequin displayed in the smart window, or to change the location of items on the smart showcase. Smart pads and wearable devices provided to employees in stores are used to provide real time trigger to customers by gathering information from smart CCTV, smart hangers, and smart displays and processing through event processing and data analysis. In other words, smart pads and wearable devices are important solutions to improve customer satisfaction and offer relevancy in smart stores.


### 5.7. Smartphone Application

Company K conducts targeted marketing through indoor positioning technology and smartphone application using beacons. The mobile application developed for this purpose provides push type information delivery, on-offline shopping link, SNS support, membership, and so forth. The following are detailed functions: firstly, it is possible to implement location-based marketing for retailers by linking indoor positioning technology and mobile application. When a customer who has installed a smart application on their smartphone passes a store, the customer can use information, such as purchase patterns, product details, events, and coupons through smartphone application. Secondly, showrooming and web rooming, which combine online mobile shopping and actual in-store shopping, becomes possible. Customers can find products of interest in offline stores and then add them into the shopping cart for mobile shopping using the smartphone application. In addition, when a customer registers in a smartphone application or comes online, shops can display location information through the mobile application when customers search for offline stores. Thirdly, the smart application provides a new shopping experience by linking the application with SNS services. Customers can transfer images shot through smart mirrors to their friends to receive their opinions and can promote products to their friends by putting products into the shopping cart. Finally, the membership function in the mobile application supports customized shopping. If a customer submits their membership card barcode through the smart window or the smart mirror, they can receive recommendations based on their interest and can receive convenient shopping service based on their purchase history.


## 5.1. Smart CCTV

Company K has installed 4~6 smart CCTVs per store in 3 branded flagship stores to implement personalized marketing by analyzing user behavior in store through indoor positioning and facial recognition technology and implement store layout optimization. Smart CCTV performs the four following functions: people counting, facial recognition, customer movement analysis, and interest area analysis. First, Company K installed CCTV with sensors to measure the number of visitors at all outlets of the store. By measuring the number of customers who entered the store, it is possible to measure the purchase conversion rate, which indicates the number of visitors per day and per hour, average residence time of customers, and number of buyers compared to visitors. Therefore, it is possible to plan an efficient operation of employees. Secondly, a facial recognition sensor was attached to the CCTV installed at the main entrance of the store. As a result, it became possible to measure the number of unique visitors by solving duplicate visitors. Thirdly, the company installed CCTV for each major area of the store, and logged customers’ travel route and residence time in the store. By analyzing customer shopping lines, it is made possible to improve the store layout, to increase the efficiency of the use of the store area, and to relocate the products in the store, thereby inducing the strategic purchase of products. Fourthly, Company K measured the residence time at a certain point through CCTV and introduced a system to send a notification message to the shop staff when customers stay at a specific spot for more than 30 seconds. Customers are prevented from leaving due to lack of proper guidance even though they are interested in the product, and the store personnel can understand the customer’s interest area and respond to the customers rapidly.As shown in Figure4, it becomes possible to evaluate the movement of people, the value of the store area, and the attractiveness of the goods objectively, whereas previously this was only identified subjectively by store personnel. Furthermore, effects of promotions and events can be measured and utilized to encourage the purchases of the viewer.Figure 4
Operation of people counting sensor.The details of the technology applied for smart CCTV are as follows: firstly, sensor cameras are installed on the ceiling of all entrances in the facility to count customers. Then a counting area is set inside the entrance, so that only those who are out of the area are counted. Secondly, a facial recognition system is installed through five sensor cameras on the ceiling of the main entrance in order to get five face images of the customer through the processor. The supporting hardware consists of a camera module, a twist unit (TWU), and a processor. TWU converts video signals into differential signals and the process encrypts and transmits upper body photographs to a management PC. In order to protect the privacy of the customer, the management PC extracts the attributes for the face recognition from the pictures and deletes the photographs. The software of GikenTrastem provides the gender and the age resolution. The test results showed an accuracy of about 80%. However, the quality of recognition accuracy depends on the mask or face direction. Thirdly, the analysis of the customer movement and the analysis of the interest area are performed by measuring the travelling time of the customer through the sensor installed on the ceiling and the time spent at the specific location and time recording. In particular, Company K has introduced a system that notifies customers of the products they are interested in by sending signals to smart pad (iPad) and wearable device (iWatch).


## 5.2. Smart Hanger

Company K has installed a smart hanger in the flagship store of S and L brands to provide an intuitive user experience by using beacon and digital display as shown in Figure5. Smart hangers analyze products of interest, customer experience, and customer response. The implementation method and effect are as follows. Firstly, Company K can attach a beacon with an accelerometer on the smart hanger to measure how many customers were holding the hanger with the item and what distance they have moved with the hanger. The company can link the logs made by the smart hanger with the purchase data and analyze the link between the goods of interest and the purchase items. This is similar to linking e-commerce to click-through pages, shopping carts, and purchase items. Through this, Company K has been able to target marketing by tracking and analyzing the processes from the point of customer’s attention to the product to the process of purchasing. Secondly, Company K has smart hangers in conjunction with digital signage to provide customers with a new store experience. The display in standby mode shows the top three items and promotional images, such as model photos. However, if a customer holds a hanger with a beacon, the media pole digital display on the top of the hanger shows details of the product such as material and price and the number of customers who previously selected the product. In addition, when the customer takes the coat hanger to the mirror, details of the product are displayed in the mirror. This allows the customer to learn detailed information about the product without help of the store staff and make purchasing decisions based on the purchasing tendencies of other customers. Thirdly, smart hangers can also help retailers gather customer data by measuring and analyzing in-store behavior. The beacon embedded in smart hangers also enables smart pads and wearable devices used by the store staff to send messages related to products of customer interest, so that they can be prepared for invisible customer interaction.Figure 5
Smart hanger.Smart hangers consists of a hanger with an accelerometer-installed beacon, a 3-sided media pole, a video wall, a digital display, and an Android set-top box. Here, the accelerometer measures the time the customer picks up the hangers and movement of the hangers. The beacon attached to the smart hanger sends the signal to the digital display when the customer picks up the hanger to display detailed information about the product. The beacon of the smart hanger also sends a signal to the mirror to display the details of the product when the two are in close proximity. While this is happening, the store employee receives an alarm through a smart pad and wearable devices when the customer picks up the hanger. Customers can also receive detailed information on products related to the products they are interested in and a list of recommended products through the smart pad.


## 5.3. Smart Mirror

Company K has been providing virtual fitting and product recommendation services to customers by combining augmented reality and interactive display technology in mirrors installed in S, L, and C brands’ flagship stores (Figure6). Through this, Company K is providing new customer experiences to customers who visit stores and improving the purchase rate by recommending products optimized for customers.Figure 6
Smart mirror using augmented reality.The smart mirror provides beacon-related product information, virtual fitting, SNS sharing, on-offline shopping connection, and product recommendation. Firstly, the smart mirror displays details of selected items in conjunction with the smart hanger, enabling customers to shop more proactively without the help of clerks. When the customer holds the smart hanger and goes near the smart mirror, the sensor attached to the mirror recognizes it and displays the thumbnail image of the product in the mirror display. When the customer selects the thumbnail image, the smart mirror displays the related item in the mirror display along with the detailed information, such as the product name, price, size, product review, laundry guide, and material. Customers can also use the staff call feature in the mirror display to ask additional questions. Secondly, the smart mirror provides a virtual fitting function that shoots moving images at 360 degrees and changes the color of the video after shooting to estimate the wearing shot. This allows customers to see which color best suits their needs without having to try on individual items of clothing. Thirdly, the smart mirror stores the pictures and images it takes. From there, they can be transferred using MMS and QR codes to their mobile phones, transmit them to their friends, and provide the function of allowing their friends or SNS to evaluate their choices. This not only provides new shopping experiences to customers, but also increases the rate of conversion of purchases by receiving real time ratings from friends. Fourthly, besides the smart hanger, customers can tag product barcodes in the store with barcode sensors to access detailed information about products by tagging them on the smart mirrors. In addition, the customer can put the retrieved items in their shopping cart on the online shopping malls for stores. The smart mirror implements the Omnichannel marketing function, which is merely in its initial form but also able to extend to showrooming. For this purpose, Company K has installed a mirror display with a camera, a beacon sensor, and a bar code recognition sensor on the mirror display. We have also introduced software that includes an augmented reality function to provide virtual fitting. Meanwhile, Company K has developed a mobile application for online shopping interactions, so that customers can add items viewed offline to their shopping cart on the online shopping malls for stores.


## 5.4. Smart Show Window

As shown in Figures7 and 8, Company K has installed a smart interactive display in the show window outside stores for drawing customer attention through customer experience enhancement. Brand S has installed a show window with facial recognition and augmented reality technology, and brand L has installed a show window with a brochure function. Company K has developed an AR mannequin on the show window of brand S, which was built with a transparent display with the dual purpose of shop window. In order to maximize the effect of the transparent display device, the actual mannequin was also displayed inside. This mannequin is displayed in a virtual outfit chosen by the shop staff. Normally, the transparent display above the outfit on the mannequin displays information about the outfit, but the new technology allows the user to view detailed information about the outfit when they touch a button on the display. The user can also change the color of the garment, allowing them to experience an interactive display that uses augmented reality technology. In addition, shop staff can change the outfit worn on the mannequin using the iPad, meaning the display can be changed from time to time to suit seasonal fashion trends. By contrast, brand L’s show window brochure has added an interactive display function to the show window display of the store that shows the contents of the store as an interactive fashion pictorial. The store can use this to contain more displays on a limited show window screen and allows the user to choose a display based on their preferences. This type of display attracts potential customers’ attentions, encouraging them to linger outside the store for longer, as well as allowing customers to make decisions before entering the store and therefore increasing the ratio of purchase rate compared to visiting customers. In addition, this encourages customer engagement by enabling customers to capture photos and utilize them as pictorial models.Figure 7
AR mannequin on the smart show window.Figure 8
Show window brochure.Company K uses facial recognition technology to measure the age range of customers and provides customized product recommendation services. At the same time, Company K uses the gamification function to attract customers who pass by and utilizes them for customer acquisition. As shown in Figure9, Company K has installed a Kinect sensor with a facial recognition function in the mirror-type show window display of the store. This show window works like a mirror, but the Kinect sensor is activated when a potential user comes within 2 m. There are camera and proximity sensors installed behind the mirror glass which is invisible to the user. In reality, facial recognition is made possible by linking cameras in digital information display (DID) with NEC’s “Field Analyst for Signage” solution. Using this process, it is possible to measure up to 20 passengers passing at a common walking speed at the same time. Each customer’s age and gender are estimated through a process consisting of three combined modules: face detection, feature extraction, and face recognition. First, we used a geometry feature-based approach to analyze the features of eyes and nose and used the geometrical relationships between them, using graph matching algorithms, for face detection. Second, we extracted image (wrinkle) and shape (geometry) features of each face image for feature extraction. Finally, we used a support vector machine (SVM) classifier in the face recognition process to improve the accuracy of gender and age estimation systems using face images. We used a single linear SVM classifier trained using the feature vector representations for gender estimation. Because gender estimation is a binary classification that distinguishes between male and female and it is easy to obtain many learning data with accurate gender information, the SVM classifier achieves a high recognition rate of over 95%. On the other hand, it is difficult to improve accuracy in age estimation, because it is difficult to obtain learning data with accurate age information and there is no standard age estimation method. Therefore, we performed multilabel age classification and then performed one-versus-one linear SVM arrangement for each age group. Furthermore, it has also become possible to classify the floating population passing through the store and the customers visiting the store by gender (male/female) and age (1-year-old).Figure 9
Facial recognition using Kinect.This makes it possible to measure the effect of the show window display by measuring the residence time (display residence time) in front of the show window, the time of the gaze (screen gaze time), and the distance (distance between the screen and the person). When the upper illuminance is above 500 lux, the accuracy is more than 90% for flow population, 80% for sex, and 70~75% for age group. However, since privacy concerns exist in facial recognition [64], images and personal information are not stored in accordance with Article 25 (1) of the Personal Data Protection Act for the protection of personal information. Instead, data is stored as raw numeric and text data used for customer analysis. Stores use facial recognition technology to enhance the customer’s interest by providing appropriate games and provide product recommendation services based on information obtained. The customer can participate in a game (age estimation game) that calculates their age through the display installed in the store show window. The photographs taken here are used for customer segmentation by utilizing facial recognition technology and provides different product recommendation services appropriate to age and gender based on face recognition results through the smart window mirror display.


## 5.5. Smart Shelf and Showcase

As shown in Figures10 and 11, Company K has installed a smart shelf and smart showcase that combines indoor positioning and smart interactive display technology in brick-and-mortar stores for increased customer retention through customer experience enhancement. Tags on fashion accessories, such as handbags, have beacons attached to accelerometers attached to them. The operation of these tags are similar to smart hangers. When the user picks up a fashion accessory, the sensor attached to the wall display captures the movement of the beacon and displays product information. If several people pick up a product at the same time, this information is displayed on the mirror near to the product. It also provides an Omnichannel shopping function that links offline shopping items to online shopping carts in conjunction with online shopping malls. At the same time, brand C from Company K offers a new user experience by combining an interactive display with a showcase, which displays accessories such as wallets. Technically, it combines touch screen-based digital signage with existing showcases to attract visitors’ attention and allows customers to view detailed information about the products they are interested in without being directly involved. The smart showcase provides detailed information about the selected product such as material, release date, selling price, and related product information when the product shown on the display is touched or the attached accessory is placed on the RFID reader. This enables customers who do not feel comfortable asking store staff to inquire about the product without direct interaction. In addition, store staff are able to respond better to customers’ needs through receiving customers’ inquiry history on their iPad.Figure 10
Smart shelf using interactive display.Figure 11
Smart showcase using interactive display.


## 5.6. Smart Pad and Wearable Device

Company K utilizes information about customer preferences accumulated in the store and provides it to store staff who use smart pads (iPad) and wearable devices (iWatch) for efficient customer response. In addition, a dedicated application was developed to process data collected from various sensors and to provide customer information to shop staff. This enables shop staff to improve customers’ experiences in relation to customer identification, product recommendation, and store management. Firstly, store staff can focus on their customers and identify their needs without having to ask questions. Customer information collected from smart CCTV’s customer entry and point of interest notifications, smart hanger, and smart merchandise is transferred to shop staff through smart pads and wearable devices. This allows shop staff to understand customers’ interest areas and which products the customer are interested in. Secondly, analyzing integrated customer behavior data allows for efficient product recommendation. When a customer hesitates to purchase a product, the recommended product and the related product list on the smart pad may prevent the customer from leaving the store and provide functions that encourage additional purchases. In addition, the availability of information such as color, sales ranking, and inventory information on smart pads make it possible for store staff to provide quick response to the customers. Thirdly, the store staff can identify the location of the items in the store and the inventory quantity as well as a customer’s staying time and list of popular products through smart pad, thereby enabling efficient store management. In addition, shop staff can use the smart pad to change the display of the augmented reality mannequin displayed in the smart window, or to change the location of items on the smart showcase. Smart pads and wearable devices provided to employees in stores are used to provide real time trigger to customers by gathering information from smart CCTV, smart hangers, and smart displays and processing through event processing and data analysis. In other words, smart pads and wearable devices are important solutions to improve customer satisfaction and offer relevancy in smart stores.


## 5.7. Smartphone Application

Company K conducts targeted marketing through indoor positioning technology and smartphone application using beacons. The mobile application developed for this purpose provides push type information delivery, on-offline shopping link, SNS support, membership, and so forth. The following are detailed functions: firstly, it is possible to implement location-based marketing for retailers by linking indoor positioning technology and mobile application. When a customer who has installed a smart application on their smartphone passes a store, the customer can use information, such as purchase patterns, product details, events, and coupons through smartphone application. Secondly, showrooming and web rooming, which combine online mobile shopping and actual in-store shopping, becomes possible. Customers can find products of interest in offline stores and then add them into the shopping cart for mobile shopping using the smartphone application. In addition, when a customer registers in a smartphone application or comes online, shops can display location information through the mobile application when customers search for offline stores. Thirdly, the smart application provides a new shopping experience by linking the application with SNS services. Customers can transfer images shot through smart mirrors to their friends to receive their opinions and can promote products to their friends by putting products into the shopping cart. Finally, the membership function in the mobile application supports customized shopping. If a customer submits their membership card barcode through the smart window or the smart mirror, they can receive recommendations based on their interest and can receive convenient shopping service based on their purchase history.


## 6. Discussions

The development of information and communication technologies is fundamentally changing the entire industry, including the retail industry [65–67]. The technologies related to Internet of Everything (IoE) connect people, processes, data, and things in the world, and add new value by adding information to things. Company K’s Smart store consists of various Internet of things technologies, such as smart hanger based on sensor technologies, smart CCTV based on indoor positioning and facial recognition, and smart mirror based on augmented reality and digital signage. The term “Internet of Everything” is an extension of the Internet of things. Dave Evans, who originated the concept of IoE, defined IoE as “the intelligent connection of people, process, data and things”. IoT focuses on machine-to-machine (M2M) communications, which are communications between machines, while the more expansive IoE concept includes M2M communications, machine-to-people (M2P), and people-to-people (P2P) interactions. We cited the term IoE because the technologies used in the smart store include M2P and P2P communications as well as M2M.In this environment, companies must utilize big data, including user behaviors, to optimize operations, adjust prices, predict demand, and provide desired product configurations. In this regard, Gartner forecasts that customer digital assistants will recognize individuals by face and voice across channels and partners by the end of 2018 [68]. They argued that customer experiences with conventional offline stores and online shopping malls will be integrated into a multichannel retail setting and this new service will support tech-savvy customers. Customers’ increased willingness to adopt facial and voice recognition technologies allows firms to acquire large amounts of information related to the consumer’s purchase process. The “smart store” is an example of the paradigm shift from “businesses use technology” to “technology defines businesses” in relation to business and technology. Recently, the characteristics of the retail industry have been transformed to “smart store” due to smart retail settings that utilize advanced technology.


### 6.1. Theoretical Contributions

Our research provides the following theoretical contributions. Firstly, we conducted an integrated research that combines marketing and computing perspectives in terms of the purpose and means of implementing a smart store. A variety of previous studies have addressed the technical elements and analysis methods for implementing customer recognition [50, 53, 69], indoor location analysis [5, 13], customer behavior logs analysis [18], and augmented realty [70–72].Secondly, our case includes case studies of indoor positioning, augmented reality, facial recognition, and interactive digital signage. Our case incorporates various technical elements and IoT into various objects in fashion shops, such as hanger, fitting mirror, show window, and shelf. While most previous studies have put emphasis on the technical content of one or two components of the smart store, our study is a comprehensive case study. In particular, our case has actually built in three branded flagship stores of the leading fashion company in Korea.Thirdly, we looked at a case of collecting data that can analyze customer behavior in the conventional offline store using various IoT technologies. This allows us to offer personalized marketing proposals to our users and optimize their circulation and layout. Tracking the user’s location, identifying products of interest, and tracking the user’s gender and age are similar to the analytic process of estimating the user’s personalities and preferences through online web log analysis. In the past, it was almost impossible to collect and analyze data on customer preferences in an offline environment. However, this study shows that it is becoming possible to analyze customer behavior using sensors and IoT technology installed in the retail stores.Fourthly, this study has demonstrated that retail smart settings in brick-and-mortar stores can increase sale within a short period. To measure the economic performance of Company K’s smart store, we compared the sales results between the “smart store” and other stores in the same brand before and after introducing “smart store.” This approach is reasonable because the fashion industry has highly fluctuating sales due to new product purchases, promotions, and seasonal changes. Table4 shows the ratio of the store sales to the average sales of all stores in the brand as a comparison index. In the case of brand S, sales of the store increased from 108.90% to 141.30% after introducing the smart store. Brand L’s sales increased from 45.98% to 61.56%, and brand C’s sales increased from 26.19% to 34.13%. Each brand showed a similar sales increase of around 30%. Average sales of brand L and C are lower than the average sales growth because there are many stores in department stores with high sales per store.Table 4
Comparison of sales before and after installation of smart store.
Before introduction(Average sales per store within brand)
After introduction(Average sales per store within brand)
Increase/decrease width
Fluctuation rate
Installation time
S brand
108.90%
141.30%
32.40%
29.76%
July, 2015
L brand
45.98%
61.56%
15.58%
33.87%
December, 2015
C brand
26.19%
34.13%
7.94%
30.33%
December, 2015


### 6.2. Limitations and Future Researches

Firstly, cost is the biggest problem with smart retail settings in brick-and-mortar stores. All retailers aim for larger profit. However, smart store technology is still too expensive to use in retail settings. A low return of investment caused by excessive investment costs and a long payback period are obstacles in spreading the smart store. For this reason, the fashion industry is building smart stores mainly in flagship stores. Therefore, it is important to overcome this problem in order to analyze user behavior at a lower cost.Secondly, from a long-term perspective, it is necessary to examine whether sales increase, increase of visitors, and purchase conversion rate were by introducing smart store. In addition, the technology and equipment required for the smart store are often expensive, so it is necessary to research whether the increase in profits from introducing the smart store exceeds the cost required. In the future, if a brand is recognized through the introduction of a flagship store, and if it has played a role in attracting customers’ attention, research on the efficiency of technology introduction is needed for universal utilization in future brick-and-mortar stores. In addition, seasonal factors, promotional effects, and longer-term changes in earnings should be considered to account for changes in sales across multiple shopping malls.Thirdly, the use of personal information, such as location, appearance, and preference of consumers collected for providing new experiences, immersions, and customized proposals comes with the risk of privacy infringement [73, 74]. Therefore, it is necessary to conduct detailed studies on the interest conflict issues between the efficiency improvement of using personal information and the protection of privacy, including legal and ethical reviews on the privacy concern.


## 6.1. Theoretical Contributions

Our research provides the following theoretical contributions. Firstly, we conducted an integrated research that combines marketing and computing perspectives in terms of the purpose and means of implementing a smart store. A variety of previous studies have addressed the technical elements and analysis methods for implementing customer recognition [50, 53, 69], indoor location analysis [5, 13], customer behavior logs analysis [18], and augmented realty [70–72].Secondly, our case includes case studies of indoor positioning, augmented reality, facial recognition, and interactive digital signage. Our case incorporates various technical elements and IoT into various objects in fashion shops, such as hanger, fitting mirror, show window, and shelf. While most previous studies have put emphasis on the technical content of one or two components of the smart store, our study is a comprehensive case study. In particular, our case has actually built in three branded flagship stores of the leading fashion company in Korea.Thirdly, we looked at a case of collecting data that can analyze customer behavior in the conventional offline store using various IoT technologies. This allows us to offer personalized marketing proposals to our users and optimize their circulation and layout. Tracking the user’s location, identifying products of interest, and tracking the user’s gender and age are similar to the analytic process of estimating the user’s personalities and preferences through online web log analysis. In the past, it was almost impossible to collect and analyze data on customer preferences in an offline environment. However, this study shows that it is becoming possible to analyze customer behavior using sensors and IoT technology installed in the retail stores.Fourthly, this study has demonstrated that retail smart settings in brick-and-mortar stores can increase sale within a short period. To measure the economic performance of Company K’s smart store, we compared the sales results between the “smart store” and other stores in the same brand before and after introducing “smart store.” This approach is reasonable because the fashion industry has highly fluctuating sales due to new product purchases, promotions, and seasonal changes. Table4 shows the ratio of the store sales to the average sales of all stores in the brand as a comparison index. In the case of brand S, sales of the store increased from 108.90% to 141.30% after introducing the smart store. Brand L’s sales increased from 45.98% to 61.56%, and brand C’s sales increased from 26.19% to 34.13%. Each brand showed a similar sales increase of around 30%. Average sales of brand L and C are lower than the average sales growth because there are many stores in department stores with high sales per store.Table 4
Comparison of sales before and after installation of smart store.
Before introduction(Average sales per store within brand)
After introduction(Average sales per store within brand)
Increase/decrease width
Fluctuation rate
Installation time
S brand
108.90%
141.30%
32.40%
29.76%
July, 2015
L brand
45.98%
61.56%
15.58%
33.87%
December, 2015
C brand
26.19%
34.13%
7.94%
30.33%
December, 2015


## 6.2. Limitations and Future Researches

Firstly, cost is the biggest problem with smart retail settings in brick-and-mortar stores. All retailers aim for larger profit. However, smart store technology is still too expensive to use in retail settings. A low return of investment caused by excessive investment costs and a long payback period are obstacles in spreading the smart store. For this reason, the fashion industry is building smart stores mainly in flagship stores. Therefore, it is important to overcome this problem in order to analyze user behavior at a lower cost.Secondly, from a long-term perspective, it is necessary to examine whether sales increase, increase of visitors, and purchase conversion rate were by introducing smart store. In addition, the technology and equipment required for the smart store are often expensive, so it is necessary to research whether the increase in profits from introducing the smart store exceeds the cost required. In the future, if a brand is recognized through the introduction of a flagship store, and if it has played a role in attracting customers’ attention, research on the efficiency of technology introduction is needed for universal utilization in future brick-and-mortar stores. In addition, seasonal factors, promotional effects, and longer-term changes in earnings should be considered to account for changes in sales across multiple shopping malls.Thirdly, the use of personal information, such as location, appearance, and preference of consumers collected for providing new experiences, immersions, and customized proposals comes with the risk of privacy infringement [73, 74]. Therefore, it is necessary to conduct detailed studies on the interest conflict issues between the efficiency improvement of using personal information and the protection of privacy, including legal and ethical reviews on the privacy concern.


## 7. Conclusion

In this paper, we investigated persuasive marketing and immersive customer experience based on various Internet of things technologies through a case study on smart store in a Korean fashion company. Smart retail settings will change customer behaviors through customer experience enhancement in the marketing perspective. In addition, various IoT technologies such as indoor positioning, augmented reality, facial recognition, and interactive display make it possible to create solutions and components for smart store implementations in the computing perspective. Although this paper thoroughly surveys the state-of-the-art marketing practices and technical applications in smart store, we believe that there should be further practical and theoretical research on smart store to implement it successfully in various situations.



---

*Source: 4738340-2017-03-05.xml*