# On the Use of Handheld Augmented Reality for Inventory Tasks: A Study with Magazine Retailers

Peter Mitts, Henrique Galvan Debarba

► To cite this version:

Peter Mitts, Henrique Galvan Debarba. On the Use of Handheld Augmented Reality for Inventory Tasks: A Study with Magazine Retailers. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.566-589, 10.1007/978-3-030-85623-6_33. hal-04331576

HAL Id: hal-04331576

https://inria.hal.science/hal-04331576v1

Submitted on 8 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# On the Use of Handheld Augmented Reality for Inventory Tasks: a Study with Magazine Retailers

Peter Mitts and Henrique Galvan Debarba

IT University of Copenhagen, Copenhagen, Denmark

**Abstract.** In this paper we investigate if handheld augmented reality, in the form of an application running on a mainstream smartphone, can serve as a practical and effective tool for inventory tasks. Taking magazine retail as an example, we have applied a user-centered design process to research, design, implement and evaluate a handheld AR application prototype. We conducted a qualitative user study at magazine retail stores, where staff responsible for magazines were interviewed (n = 8) and their primary magazine handling tasks observed. After an analysis of the study findings, we selected a key task as the basis for the design, implementation and test of an AR app prototype. The task consisted of collecting and registering a list of magazines for return to the distributor. We evaluated the AR app prototype in a user study (n = 22), where participants used it to perform the selected task. They also performed the task using the paper list currently in use, and a second, simplified app prototype, without AR features. Task performance was measured based on time and error rate. The participant's subjective experience was also captured in the form of a post-task survey and interview. Our findings suggest that handheld AR can prove effective when used for specific, focused tasks, rather than more open-ended ones.

**Keywords:** Augmented reality · Handheld devices · User-centered design · User studies.

## 1 Introduction

Augmented Reality (AR) has the capability to improve logistics workflows across a number of areas, from warehousing operations [17, 8], transportation optimization and last-mile delivery [27], to enhanced value-added services [9, 26]. In fact, AR technologies excel in situations where it is required to present information and functionality in a spatial, three-dimensional context, in relationship to physical objects and locations [13]. Notably, mobile augmented reality (MAR) applications are widely available today, on devices ranging from industrial grade headsets to smartphones [7]. Technological advances across the board, from mobile displays, cameras and sensors to broad availability of wireless network connectivity, and through it access to cloud computing resources, provide the building
blocks to enable widespread implementation of MAR for professionals as well as consumers [2].

Here we explore how MAR can be used in inventory tasks, and in particular, in the management of a magazine inventory. Physically handling a store's magazine inventory can require a significant investment of working hours spent on tasks such as receiving magazines, correctly placing them on stands, and locating outdated magazines to be returned to the distributor. Traditional tools, like paper pick lists and shipping waybills, leave a lot of work for the store staff. For example, the physical magazines still have to be checked against their identifying details from the list (name, volume, etc.), and the correct magazine stands located and eventually memorized.

An AR solution for magazine retail can leverage the built-in computer vision features of the mainstream AR platforms, coupled with magazine cover image databases, to more easily identify magazines without having to check a list or scan a barcode. While distributors can potentially provide supporting resources like these, they do not have the authority to dictate any specialized hardware purchases by the customer stores they serve. Therefore the smartphone is the most realistic hardware candidate, since it is likely to already be available, ready for installation of a new AR application. Being handheld, however, means that the benefits of AR have to be balanced against the drawback of requiring a hand to hold the device.

To better understand the specific challenges of AR for inventory management and analyze potential avenues for AR functionalities, we conducted a qualitative field research with seven magazine retail stores. We interviewed the staff responsible for magazines (n = 8), and carried observations as they performed typical inventory management tasks. This research informed the selection of a realistic task to be used as the basis for the design of a smartphone app prototype with AR features. The task consists of collecting and registering a list of magazines that have to be returned to the distributor. A user study was conducted to evaluate the AR app prototype (n = 22), compared to the paper list currently in use, and a second, simplified app prototype, without AR features. Our results indicate that different aspects of the task could be improved by adopting a combination of the AR app and the simplified list based app, over the commonly used paper list.

In summary, this study contributes to AR design research by:

- Investigating the potential role of AR in retail stores, with a focus on magazine retail, where current research on AR and logistics may not apply due to differences in scale, tasks and environments.

- Designing and evaluating a mobile AR application prototype developed to address some of the problems uncovered in the qualitative investigation.

- Providing reflections and design guidelines based on the results and observations of the user evaluation.
# 2 Background

## 2.1 Augmented reality in logistics

In [15], Porter et al. argue that AR can create business value in two broad ways; by becoming part of the product itself, or by improving performance across the whole value chain. This paper is particularly concerned with the latter, where AR can be used to enhance the flow of information from a database to the operator by presenting contextualized information. This can benefit many ordinary tasks in logistics and the industry, such as wayfinding, where AR is used to issue contextualized navigation instructions with the path to a given destination [4, 22], object search, where AR emphasizes the object of interest [17, 10], assembly instructions, where AR is used to detect the current stage of assembly and present step-by-step 3D information to an operator [24], and remote guidance, where a remote instructor can issue contextualized instructions to an AR supported operator [18, 23, 25].

Cirulis and Ginters [5] present the basics of logistics processes, and possibilities for improvement, such as reducing workload and easing decision making in routine operations. They claim that solutions built on AR technology can decrease object pickup time and error rates in human-operated warehouses by providing workers with the information they need to make better decisions. This information can be visualized in the worker's physical environment, to further simplify its interpretation. For example, guidance and wayfinding instructions can be displayed as 3D graphics that shows, rather than tells, where to go. Providing a more concrete example, Cirulis and Ginters [5] describe a process for designing an AR solution for order picking, which is a common warehouse task. First, a virtual model of the warehouse is constructed. It can be as simple as a 2D floor plan, or significantly more elaborate with areas, locations and paths represented in 3D space. An optimal pathfinding solution can then be generated for a given set of objects in the order they should be picked up. The worker is typically equipped with a mobile AR device such as head-mounted display (HMD) or a handheld screen device, such as a tablet or smartphone, that will present a sequence of steps to be followed as a visual overlay. In fact, assisted indoor navigation is a recurrent topic in AR research, and while warehouse research normally focus on head-mounted AR, the topic has also been investigated for a range of devices and from different perspectives. For instance, Chung et al. focused on the use of projection based AR [4], while Soulard [22] explored user interfaces for handheld devices.

According to Reif et al. [17, 21], the hardware in an AR system consists of three distinct parts; the visualization, interaction, and tracking system. They develop a concrete AR-based solution for warehouse workers, Pick-by-vision, that includes the first two components. A HMD is used for visualization, and the user interacts with the system using an adjusting knob and speech input. Evaluation of the system in a real warehouse environment shows that users are faster and make fewer errors, compared to using a paper list. While the results of the study were very promising, the paper also concludes with recognizing the
challenge of porting research systems built on similar hardware into practical industry use.

Moreover, a functional tracking and visualization system can be used for other inventory management tasks, besides wayfinding. Notably, Chen et al. [3] propose using the tracking and processing capabilities of mobile devices for registering the location of inventory items in a library, which can enable a system to effectively be used for both, navigation guidance and maintenance of the stock database.

Here we focus on a related but significantly different problem. The logistical challenges in magazine retail, and retail more generally, are quite different from the ones in warehouses. The distances to cover within the store are shorter, so wayfinding is less complicated. Conversely, a store presents a more dense, “target-rich” environment when it comes to actually spotting the magazine to find on a given stand. The environment is also less controllable and stable than a warehouse. As customers browse magazines, they are put back in different locations than where they initially were.

Moreover, literature that investigates the use of AR in retail settings seems to focus mostly on marketing strategies and consumer engagement, with the objective to enhance the experience of users with a specific product, brand or store. For instance, Bonetti et al. [1] investigate the augmentation of physical stores with a virtual model of the store and product information. Their exploration suggest that augmenting the store with AR can increase consumers’ desire to shop at a particular retailer. In contrast to that, here we are concerned with how handheld AR can improve the work of store employees.

## 2.2 Designing for handheld augmented reality

Augmented reality is a relatively recent technological development, and user interface conventions have not reached the same maturity level as traditional 2D user interfaces. According to de Sá and Churchill [19], challenges when designing for handheld, or any form of mobile, AR include discoverability, interpretability and usability.

AR applications need to remain usable while displaying their augmented world realistically. However, since they tend to present their content in a layered way, this can sometimes result in an inherently messy appearance [2]. To ensure an adequate user experience, Kourouthanassis et al. offer a set of general design principles [12]:

- Use the context (e.g. location) for providing content.

- Deliver relevant-to-the-task content (filter and personalize the AR content).

- Inform the user about content privacy.

- Support different application configurations, to ensure an adequate user experience based on the available processing power, system resources, network connectivity etc.

- Make it easy for the user to learn how to use the application.
Instead of trying to fit 2D User Interface (UI) conventions to a 3D AR UI, or
attempting to create a set of new conventions from scratch, we can be inspired
by other fields. Video games are such a field, which has already tackled the
challenges of viewing and interacting with a 3D world through a 2D display. In
[6] Dillman et al. reviewed the cues in a selection of video games, and classified
into a framework based on three dimensions; purpose, markedness and trigger,
i.e. the goal of the cue, how distinct the interaction cue is from the environment
and what makes the cue become visible, respectively.

The three dimensions are mapped to AR, followed by a series of examples
describing visual interaction cues from existing AR applications in terms of the
framework. The framework is also presented as a tool for generating new inter-
action cue design ideas.

3 Prototype

3.1 Field research

This research was developed in collaboration with one of the largest postal com-
panies in Denmark. Their magazine vendor customers are comprised of different
types of businesses, from corner shops and grocery stores to specialty magazine
stores. For many of them receiving, handling and returning of magazines requires
a significant investment of working hours. The purpose of the field research was
to achieve a more detailed understanding of current magazine handling and its
physical context, and specifically to identify routine, error-prone tasks where AR
methods may be a good fit.

The field research was conducted as a series of magazine vendor visits that
included the following activities:

- A semi-structured interview, where the participant was asked about their magazine handling routines, and where and how the tasks were performed. They were also asked to freely talk about any specific things they perceived as working well, or less so, in the current system.

- Task observation, where the participant was asked to think aloud while performing their routine magazine handling tasks.

- General inspection of the store layout, focused on the placement of magazine stands in relation to the other areas where magazine tasks were performed.

- Inspection of magazine placement in the stands.

Seven magazine vendors were recruited around the greater Copenhagen, Den-
mark, area in a collaboration with the distribution company. The smallest store
managed 68 different magazine titles whereas the largest store managed 689 dif-
ferent titles. At each vendor one or more employees responsible for magazine
handling were interviewed, for a total of 9 staff interviews. However, one vendor
later decided against participating in the interview, so the visit was limited to
the inspection of store layout and magazine placement. All participants read and
agreed to an informed consent form.
During the field observations two primary task candidates for prototyping and test were identified, replacing old magazines with new ones and registering magazines to return. The vendor needs to return the magazines that were re-placed by newer issues and only gets billed for the difference between the original delivery and the returned amount. Therefore, it is important for the vendor to carry these tasks without mistakes since they can affect the profitability of the operation.

**Replacing old magazines with new ones** When new magazines are received, they are first registered in the store's point of sale system and the price checked. New price labels may be attached if needed. Then the magazines are carried out to the magazine stands. One magazine at a time, the old issue needs to be found and replaced with the new issue. Note that the old magazine may of course be sold out, so the staff member needs to first determine if this is the case, and then decide where the new issue should be placed. Finally, the old magazines are taken out to the back room. When performing this task, the main challenges are determining what stand (and where on that stand) to look for the old magazine issue, and then identifying the physical magazine issue itself. An AR app prototype could help by indicating the expected location and then further positively identifying it (using image recognition) and indicating it more precisely. Also, simply showing the cover image for the old magazine is likely to simplify the search for it, even before the addition of AR. It would also be possible to display additional product information, but that is less likely to be important to complete this particular task.

**Registering magazines to return** Near the end of each week, the vendor receives a paper recall list, containing all the magazine issues to find, count, register and pack up in bundles for return at the end of that week. This workflow starts by going through the magazines in the back room that have already been taken down during the week (see the first task described above). Each magazine issue in the stack in the back room is checked against the list, and if it's on the list, the number of copies are registered by writing it in a designated field. The ID of the next unused bundle label (from the distributor) is also written in a separate field. Then the magazines are put in a new stack to the side. This is repeated for all the magazines in the stack. Any magazines that are not on the list are put to the side for later. Next, any remaining magazines on the list need to be found out on the magazine stands. Each remaining magazine issue on the list is looked for, and if found counted and registered on the list. Note that, as in the first workflow above, magazine issues may be sold out. Finally, the registered magazines are taken out to the back room.

This task has two distinct phases. During the first phase, the main challenge is matching a physical magazine issue with text on a row on the recall list, especially since the texts may not be identical in either content or format. The field for writing the count is also at the opposite side of the row from the title, so simply writing in the correct field can be an issue as well. An AR app prototype
could help by positively identifying the magazine (using image recognition), so
that the list lookup would be fully automated, and only entering the magazine
count done manually. Also, simply showing the cover image for each magazine
on the list is likely to simplify matching physical magazines to list items, even
before the addition of AR.

During the second phase, the main challenges are determining what magazine
stand (and where on that stand) to look for the magazine issue on the list, and
then identifying the physical magazine issue itself. Similar to the first workflow
above, an AR app prototype could help by indicating the expected location and
then further positively identifying it (using image recognition) and indicating it
more precisely. Again, simply showing the cover image for each magazine on the
list is likely to simplify the search for it, even before the addition of AR.

**Additional findings** We have also observed other user related aspects that may affect efficiency while carrying out the tasks.

Typically only one staff member was responsible for magazines at each ven-
dor. Other staff members may call them if they are not at work when there is
a magazine related issue, like customers asking for a particular magazine. The
"magazines responsible" is typically the only one on the staff who is familiar in
detail with the placement scheme for the magazine stands. When less experi-
enced staff place magazines on the stands, which can happens somewhat often,
they are likely to spend significantly more time on the task, and frequently place
magazines incorrectly. We believe that AR features can be particularly helpful
to the naive employee that may need to carry these tasks on occasion, and we
consider them as potential users later in the development and evaluation of the
prototype and during the discussion.

Many of the vendors did not find the current recall list helpful, and did not
register their returns on it. This meant that they did not have a record of their
returns to check against the invoice later. Therefore, they cannot anticipate the
cost of the invoice, having to rely solely on the verification made by the distrib-
utor. This could lead to disagreements between the vendor and the distributor
as the vendor cannot verify the correctness of the invoice. We believe that the
digitalization of the inventory management alone can improve the transparency
in the relation between both parts.

Finally, the vendors perceived it as problematic that, for many magazines, the
recall week could be weeks after the next issue has been received. This was mainly
due to the fact that they did not have any extra storage space for magazines
no longer for sale, but it also implies that they have to search the magazines
that need to be returned from larger piles, instead of simply registering these
magazines for return.

3.2 Design

The task “Registering magazines to return” was selected as a basis for the user
test and prototype concept. This was due to the fact that the two distinct phases
the task consists of allows for addressing two different types of user challenges; quickly identifying magazines in a stack, and finding a set of magazines on multiple magazine stands. Moreover, there is an opportunity to facilitate how vendors keep a record of the magazines that they return, which is critical to improve the transparency in the relation of the distributor with retailers.

The AR App prototype includes three main features: List, Scan and Find:

*List* includes a list of magazines to return, the ability to enter the return counts for each magazine and then register the counts on the list. It can also display a large version of each magazine's cover. This feature was included to address the complexity of matching a physical magazine issue with its information as presented in the recall lists, which can be time consuming. It provides users with visual information of the stock item in addition to name and issue number.

*Scan* allows for automatically selecting a magazine that is in front of the camera by recognition of its cover image. This simplifies the process of mapping physical items in the magazine stack with list entries in the recall list since it can immediately select a recalled magazine based on the cover of a physical copy, or indicate to the user that this magazine does not need to be returned just yet. It is an automated alternative to searching that aims to reduce the workload and number of errors of the operator.

Finally, *Find* indicates the expected locations of all the magazines on the list still without a return count entry. It combines wayfinding and image recognition to facilitate a picking task, similar to what was described earlier, but adapted to smaller and cluttered environments. This functionality should help staff with locating the recall magazines in the stands, even when they are not familiar with the layout of the shop and the different magazine categories. Therefore, we believe that this can be particularly helpful for naive users, who have little or no experience with the task.

The prototype UI includes the screens below:

**Magazine list screen:** This screen contains a list of all the magazines to be returned. For each magazine, the cover, title, issue and category are displayed (see Figure 1a). A magazine in the list can be selected by tapping it. The same scan functionality as described below for the Scan screen is also active here, so a magazine can also be selected by pointing the camera at the physical magazine's cover. When a magazine is selected, entry controls are revealed (see Figure 1b). Initially there is no value, but tapping the + button increases the count to 0, 1, 2 and so on. The - button decreases the value. Tapping the cover opens the Large magazine cover screen. At the bottom of the list is a registration button that becomes active when all the magazines in the list have their return count entered (see Figure 1c). Tapping the button submits the list and displays the Confirmation screen.

**Large magazine cover screen:** This screen contains a large version of the selected magazine's cover as well as the other information displayed for it in the magazine list (see Figure 1d). The next or previous magazine in the list can be selected by swiping left or right. The same scan functionality as described below
for the Scan screen is also active here, so a magazine can also be selected by
pointing the camera at the physical magazine's cover. The return count can be
increased and decreased with a pair of + and - buttons, just like on the Magazine
list screen.

*Scan screen:* This screen includes a large camera view, where any magazine cover recognized by the camera is selected and indicated with an outline frame. Below the camera view, the selected magazine is displayed with entry controls, so the return count can be entered directly (see Figure 2a).

*Find screen:* This screen provides indicators for finding all magazines that remain on the list without an entered return count. If the camera is not pointed towards an area where a magazine should be, the area is instead indicated by an off-screen indicator arrow, showing where to point the camera (see Figure 2b). Once the area of a magazine stand where a magazine should be is in view, it is indicated by a category area frame around it. This frame is based on the magazine's category, and the location where magazines of that category should be placed (see Figure 2c). Finally, if the camera comes close enough to the magazine itself to recognize it, the category area frame is replaced with a frame around the magazine (see Figure 2d).

*Confirmation screen:* This screen displays a message confirming that the entered return counts have been registered.

<figure><img src="image_3.png" /><figcaption>Fig. 1: Screenshots from the prototype: (a) Magazine list; (b) Item selected and entered value; (c) Active registration button; (d) Large magazine cover.</figcaption></figure>

3.3 AR Scan feature

The magazine cover Scan feature was intended to allow for selecting a maga-
zine in front of the camera automatically by recognizing its cover image. It was
<figure><img src="image_4.png" /><figcaption>Fig. 2: Screenshots from the prototype: (a) Scan; (b) Find, displaying off-screen indicator arrows; (c) Find, displaying category area frames; (d) Find, displaying a frame around an identified magazine.</figcaption></figure>

initially implemented only as a separate camera viewport screen, where the rec-
ognized magazine cover was indicated with an AR frame and title around it (see
Figure 2a). New ideas and experimentation during technical design iterations
implementation resulted in also enabling the Scan functionality, without the AR
graphics, on the List and Large cover screens (see Figure 1). Our intent was to
offer test participants multiple ways of scanning, without necessarily having to
have a dedicated camera viewport visible.

In the Dillman et al. [6] framework, the magazine frames map to the *Discover* purpose (inform the user that the magazine can now be acted upon), *Emphasized* markedness (the frame is visually distinct from the environment) and *Player* trigger (directly triggered by the user pointing the camera at a magazine to scan it when the Scan mode is active).

3.4 AR Find feature

The magazine *Find* feature was intended to provide a form of local wayfinding,
indicating the expected locations of the remaining unregistered magazines on the
magazine stands. Since a typical store and its few magazine stands represent a
concentrated environment, with short distances, and large numbers of potential
targets (magazines), it is more critical to be able to indicate a location on the
vertical axis than in a warehouse such as the one used by Reiff et al. [16, 17].
Thus, the *Find* feature was implemented with three different levels of precision.

At a very short distance, magazine frames are the same AR frame and title
(see Figure 2d) used to indicate an identified magazine as in the *Scan* feature.
The difference is that *Find* uses it to indicate magazines still on the magazine
stands. It should be noted that while the cropped tracking cover images increased
the probability of a positive match, they also appeared to reduce the distance at
which they could be identified. In the Dillman et al. [6] framework, the magazine
frames map to the *Look* purpose (make the user look at the identified magazine),
*Emphasized* markedness (the frame is visually distinct from the environment)
and *Context* trigger (indirectly triggered when the Find mode is active).

When magazines are not close enough to the camera to be identified, their approximate locations are indicated with category area frames. The test magazines had been placed realistically by category, so these categories could then be mapped to areas on the magazine stands (see Figure 2c). Then, the AR representations of the magazine stands were constructed, with a titled frame for each magazine category placed in front of the part of the magazine stand where magazines of that category should be placed (see Figure 3b). In the Dillman et al. [6] framework, the category area frames map to the *Go* purpose (make the user go towards the magazine stand where the frame is), *Emphasized* markedness (the frame is visually distinct from the environment) and *Context* trigger (indirectly triggered when the Find mode is active). When the user is near the magazine stand, it can be argued that the frame acts as a *Look*, rather than a *Go* purpose cue, since it is now directing the gaze of the user.

When the camera is facing away from the magazine stands, the locations
of the relevant category area frames (with remaining unregistered magazines)
are indicated by off-screen indicator arrows. These were inspired by a common
convention in video games for indicating objects of interest (e.g. enemies) outside
the screen area. They are implemented as 2D arrows lying "flat" on the screen,
rather than in 3D space (see Figure 2b). This ensures that they are always equally
visible (when shown), regardless of the angle of the smartphone. Each arrow
points towards a specific category area frame with unregistered magazines, and
once the frame is in view, the arrow is hidden. In the Dillman et al. [6] framework,
the off-screen indicator arrows map to the *Look* purpose (make the user look in
the direction of the arrow), *Overlaid* markedness (the arrows are placed on top
of the viewport, rather than 3D space) and *Context* trigger (indirectly triggered
when the Find mode is active).

## 3.5 Design variations

The initial idea was to construct the AR-based app prototype, and then test
it against the current paper-based method. However, since so many issues with
the current paper list were uncovered during the field research, it was considered
likely that simply going from a paper list with text to a digital list with more
relevant text and magazine cover images, would bring significant improvement,
before introducing AR features. Thus, the scope of the test was increased to
include testing three different methods to solve the same type of task. Three
prototypes were needed for testing.

- AR App - a smartphone app with AR features, in addition to a basic magazine recall list and cover images.

- Paper List - a paper list similar to the current magazine recall list layout.
- List App - a smartphone app with only the magazine recall list and cover images.

The List App prototype was implemented by stripping out all the AR features from the prototype described above, keeping only the list and large cover screens. Identical to Figure 1, but with the camera disabled and a different color scheme to help distancing the two prototypes.

The Paper List is simply a set of printed mockups of the paper list format currently in use, filled with the actual magazines that are used in the test and a field to write the number of magazines.

## 3.6 Implementation

The prototype was built on the *Unity*<fnref n="1" /> game engine, using the cross-platform AR framework, *AR Foundation*<fnref n="2" />, that works as a wrapper around the frameworks offered by Apple's iOS (ARKit<fnref n="3" />) and Google's Android (ARCore<fnref n="4" />). We run all the tests, including the user study, on an iPhone 8, but using AR Foundation meant we also had the option of installing on Android phone, if needed.

Database: the Magazine data needed by the AR App prototype consists of *display cover* images, *tracking cover* images and *magazine attributes*. The *display cover* is used in the magazine list and large cover magazine, as presented in Figure 1. They were created by taking pictures of the magazines. The *tracking cover* are images used by the *AR Foundation* subsystems to identify each magazine. Magazines are often layered when placed in the stand, and only a small portion of the cover is visible for identification. To increase the relative image area visible to the camera, these *tracking cover* images are cropped versions of the full cover, which only include the top-left corner of the cover. Finally, the *magazine attributes* consists of magazine title, issue and category.

# 4 User study

## 4.1 Task

Participants were asked to perform the same task for each test, with a different prototype and a different set of magazines. The task consisted of two subtasks.

*Subtask 1 - Register magazines from the stack:* They started at the “back room” table, facing away from the “store”. For each magazine in the stack on the table they should count the number of copies, register the count on the list, and then put the counted copies in a stack on the right side of the table.

*Subtask 2 - Find and register magazines from the stands:* Afterwards, for the remaining magazines on the list, they needed to go into the “store”. They should

<footnote n="1">¹ https://unity3d.com</footnote>

<footnote n="2">² https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@3.1/manual</footnote>

<footnote n="3">³ https://developer.apple.com/augmented-reality/</footnote>

<footnote n="4">⁴ https://developers.google.com/ar</footnote>
find the magazines on the stands, gather them on the table next to the stands, and register them on the list.

Participants were also told that magazines may be sold out, in fact exactly one title for each test was sold out, so they would not be able to find it, and should just register a count of zero once they had determined which one it was.

Finally, they should take the registered magazines back to the “back room” table and put them on the same stack as the others. The task was complete when all magazines were on the table, and the filled-out list had been submitted.

## 4.2 Experiment design

The purpose of the prototype evaluation tests was to acquire and compare objective task performance and subjective experience between three different prototypes, namely: the *AR App*; the *List App*; and the *Paper list*. The experiment followed a within subject design. Thus, participants completed the whole task three times, once with each of the prototypes. The presentation order of the prototypes was counterbalanced to control for learning effect due to task repetition.

## 4.3 Test environment

The test environment was set up with two magazine stands in one corner (the “store”), and a small table (the “back room”) in a room (Figure 3a). A large table in the center of the room served as a means to separate the two areas from each other and also recreate the relatively narrow space available in front of the magazine stands in a store.

<figure><img src="image_5.png" /><figcaption>Fig. 3: (a) Test room layout diagram. (b) Magazine category area frames objects, a placeholder plane for the anchor banner can be seen in the center. (c) Magazine stand with anchor banner.</figcaption></figure>
The stands were realistically filled with magazines based on typical magazine
categories and placement in stores. Three test magazine sets were prepared, so
that participants would not repeat the same set while testing the three different
prototypes. Each set consisted of:

– 10 titles, in a stack on the “back room” table;

– 5 titles, in the magazine stands. One of these always had a title that was
very similar to two other magazines nearby;

– 1 “sold out” title, that was neither in the stack nor on the stands.

The number of copies of each title varied, from one to four, but all stacks consisted of 20 copies, and the copies to find on the stands were always 15, adding up to 35 copies in each magazine set. The total of “sold out” copies were obviously 0. To control the systematic influence that the magazine visual attributes and placement could have on task performance, the use of the magazine sets was randomized.

A fourth smaller set was prepared for the introductory task training. It con-
sisted of two magazine titles in a stack, and a single title on one of the magazine
stands.

4.4 Dependent variables and procedure

Objective performance was measured by capturing the time to complete each of the two sub-tasks, total task time and number of errors. These metrics are relevant to the task and application context since shorter completion times could translate into productivity gains, whereas errors when documenting and returning stock can affect reimbursement and profit for store owners. Each participant's subjective experience was captured in a survey, as well as a brief post-test interview. We were particularly interested on perceived workload and ease to use of the prototypes, since a common argument for AR applications is the potential to reduce the mental workload of users, as Cirulis and Ginters discuss in [5].

To be able to test participants that had a basic pre-existing familiarity with
the environment, task and tools, participants were given basic training prior
to the three tests themselves. After an initial introduction to the agenda for
the test session they were given a couple of minutes to get acquainted with the
magazine stands and their contents, paying special attention to what categories
of magazines they could see.

Before each test the participant was introduced to features of the proto-
type they would use, and given a small hands-on training task to familiarize
themselves with how to perform the real test task. They were informed about
technical limitations and known issues of the prototype, and also encouraged to
ask questions about anything they were unsure of. It's worth noting that while
all the features of the List App prototype are included in the AR App prototype,
participants were explicitly instructed to make use of the AR features, and only
use other features if they felt necessary to be able to proceed.

After this the participant was to perform the actual test itself. They were
encouraged to think aloud during the test, but prioritize performing the task as
fast as made sense to them, without feeling rushed. They were also told that
they could ask for help if stuck, and that this would also be recorded as part of
the result.

After each test, the participant was asked to fill out a short survey, where they assessed their workload level with the NASA Task Load Index<fnref n="5" /> [11], adapted to a 7-point Likert scale. They were also asked to answer a Single Easy Question [20] to assess the overall difficulty of the task (“Overall, how difficult was this task?”) on a 7-point scale and write down any other free-form comments they had about their experience. This procedure was repeated for each of the three conditions during a testing session.

After completing all of the tests, participants were asked to fill out a survey to capture more general information about them, as well as their smartphone usage and previous experience with AR, VR (virtual reality) and video games. The test session concluded with a brief interview about their experience with the AR features in the AR App prototype. Each test session lasted about 60 minutes.

# 5 Results

A total of 22 participants took part in the experiment (age from 22 to 51, median of 28, 15 female), three of which have been removed due to malfunctioning of the prototype during the experiment, leaving a total of 19 participants. Recruitment was carried through the university mailing lists. Originally, we planned to test the AR App prototype with store employees, but this turned out not to be possible. However, our sample is representative of typical temporary employees hired in these stores, since it was composed mostly by students. In fact, one participant had previous experience with managing magazine inventories for a store. Participants did not know the experimenter before and were compensated with a gift card. All participants read and agreed to an informed consent form.

The statistical significance of the differences in task duration between prototypes was verified using the repeated measures ANOVA test, followed by pairwise comparisons with the t-test. The violation of the assumptions of normality of residuals and sphericity were verified using the Shapiro-Wilk test and Mauchly's sphericity test. The statistical significance of the differences in the Single Easy Question and Task Load ratings between prototypes was verified using the Friedman test, followed by pairwise comparisons with the Wilcoxon Signed Rank test. The significance threshold α was set to 0.05 for all tests. The collected data (time and questionnaire responses) and analysis code are available for download at [14].

## 5.1 Duration

During the first subtask, registering magazines from the stack in the “back room”, participants were generally faster using the AR App (mean $\mu$ = 90.84,

<footnote n="5">⁵ https://humansystems.arc.nasa.gov/groups/tlx/downloads/TLXScale.pdf</footnote>
standard deviation σ = 25.45) than List App (μ = 111.05, σ = 24.17) and Paper List (μ = 107.9, σ = 29.09), as shown in Figure 4a. A statistically significant difference was found (F(2,36) = 7.24, p < .003), with a decrease in task duration for the AR App compared to both the Paper List (t(18) = 2.66, p < .02) and List App (t(18) = 4.22, p < .001). We did not observe a statistically significant difference between the two list methods (t(18) = .54, p = .59).

During the second subtask, finding and registering magazines from the stands in the “store”, the List App (μ = 177, σ = 73.3) was generally faster than AR App (μ = 231.3, σ = 77.1) and Paper List (μ = 209.7, σ = 65.1), as shown in Figure 4b. However, the difference was not statistically significant (F(2,36), p = .06). We note, however, the presence of outliers in both App conditions, which could indicate that the App prototypes were more challenging to some users than others.

Looking at the full task, the App prototypes did not appear to decrease duration (Figure 4c). The AR App (μ = 322.2, σ = 84.9), List App (μ = 288.1, σ = 82.5) and Paper List (μ = 317.6, σ = 81.1) presented similar total time, with a small advantage for the List App prototype. However, we did not observe a statistically significant difference in duration between the three prototypes (F(2,36) = 1.15, p > .33).

<figure><img src="image_6.png" /><figcaption>Fig. 4: Box plots showing the duration of the two subtasks as well as full task duration: (a) registering magazines from the stack; (b) finding and registering magazines from the stands; (c) total time. ‘*’ indicates p < .05 and ‘***’ indicates p < .001.</figcaption></figure>

## 5.2 Errors

None of the recorded errors below were noticed by the participants themselves. There was a small number of basic entry errors for the List App and AR App prototypes, where participants appeared to enter a value that was 1 less than the correct value, due to how the return count entry controls work, i.e. the first button press would set the count to zero, in case there were not magazines to return, but users naturally expected that it would set it to one and would not visually inspect the input number. Even after being informed of this behavior
prior to the tests, some participants on occasion overlooked the mistake of only counting the number of button taps without checking the value on the screen. These errors (2 for the List App, 7 for the AR App, 9 in total) have been removed from further analysis.

The number of errors for all three prototypes was low, and no conclusions about differences in error rates could be drawn. However, different types of errors occurred for the different prototypes. These error types were split into two major categories; physical magazine picking errors, and list entry errors.

For the Paper List, the picking errors were picking a magazine with a similar title to the correct one (two errors in total). The entry error was writing in the wrong field on the paper list, most likely due to the title being on the left side of each row, and the entry field at the right side (two errors in total). When this occurred, it automatically triggered a second error, when the participant wrote the count for the next title in the remaining field.

For the List App, while there were no errors based on similar magazine titles, there were instead errors based on similar covers (two errors in total). In one case, while registering magazines from the stack, the count was incorrectly entered for a magazine that was on the stands. This in turn triggered another error; that the magazine on the stand was not searched for and picked. In another case, the same error occurred while searching for magazines on the stands (two errors in total).

For the AR App, a single entry error occurred when the camera was briefly turned so that it accidentally scanned another magazine, and the count then was registered for the wrong magazine (one error in total). In another case the wrong magazine, with a similar cover, was picked and registered by the participant, even if the AR features did not identify it as the right one (one error in total).

## 5.3 Questionnaires

When asked to rate the overall difficulty of performing the test task with each prototype, participants did not appear to consider the AR App ($Q1 = 2, Median = 3, Q3 = 3.5$) less difficult than the List App ($Q1 = 1.5, Median = 2, Q3 = 3$) or Paper List ($Q1 = 2, Median = 3, Q3 = 4.5$). No statistically significant difference was found for this question ($\chi^2(2) = 4.41, p = .11$).

Participants were also asked to rate their task load using the NASA Task Load Index questions (Figure 5). No statistically significant difference was found with regard to Mental ($\chi^2(2) = 4.72, p = .094$), Physical ($\chi^2(2) = 4.03, p = .133$) or Temporal ($\chi^2(2) = 5.41, p = .067$) Demand.

Participants rated their success in performing the task relatively highly, regardless of the prototype used (“Performance” score in Figure 5), and no statistically significant difference was found for this particular question ($\chi^2(2) = 4.1, p = .129$).

A statistically significant difference was found for the “Effort” question ($\chi^2(2) = 10.24, p = .006$). The effort required to use the AR App was rated lower than the Paper List, with a significant difference ($p < 0.05$). This was also true for
the List App compared to the Paper List (*p* < 0.004). The List App did not appear to require a significantly different level of effort, compared to the AR App (*p* = .35).

A statistically significant difference was also found for the “Frustration” question ($\chi^2_{(2)} = 6.12, p = .047$). The List App was less frustrating than the Paper List, with a significant difference (*p* < .04). However, the ratings did not suggest that participants were less frustrated while using the AR App, compared to the Paper List or the List App, there was no significant difference (*p* = .53 and *p* = .057 respectively).

<figure><img src="image_7.png" /><figcaption>Fig. 5: Box plots showing the ratings for the NASA Task Load Index questions: Mental Demand; Physical Demand; Temporal Demand; Performance; Effort; and Frustration. ‘*’ indicates *p* < .05 and ‘**’ indicates *p* < .01.</figcaption></figure>

## 5.4 Key observations

Here we highlight the key observations obtained in the semi-structured interview and through the observation of participants while performing the task using the AR App. Since we believe that the malfunction of the prototype did not affect these aspects of the user study we consider all 22 participants here.

The AR Scan feature was perceived by participants as being very fast and easy to use. In fact, some participants mentioned that it might almost be “too fast”, as it immediately selects any magazine that the camera is pointed at. During the tests, we observed situations where the participant, when about to register the return count for one magazine, briefly turned the camera so that it accidentally scanned and selected another magazine instead. This kind of error was observed once, as described above. Several participants stated that they felt
like they were on “auto-pilot” while using the AR Scan feature; just trusting the app, without having to think about what they were doing. This was not always perceived as positive, because afterwards they could not know for sure if the task had been completed without errors. Moreover, we recall that we have recorded a significant gain in performance with this feature, where participants were faster at registering magazines to return from a pile by scanning than by searching it on a list.

The AR Find feature's indication of multiple target magazines using arrows and frames was reported as overwhelming by several participants. They cited a feeling of lack of control over the app in this situation, and a preference for instead manually selecting and finding a single magazine at a time. During the test of the AR Find feature, many participants stayed very focused on the smartphone screen. Rather than e.g. trying to find the magazines by looking directly at the magazine stand, after having found the general area using the app, they appeared to continue to view their environment through this “lens”. This behavior, severely limiting their effective field of view and still remaining “locked” to the screen, was also noticed by several of the participants themselves. They mentioned it as something negative during the post-test interview. We recall that no gain of performance was observed for the AR App in this task, whereas the List App seem to offer some advantage, although the recorded difference was not statistically significant. Related to the issue presented above, the List App had the advantage that it did not requested too much attention from the user, as with the AR App.

While registering the magazines in the stack using the AR App, slightly more participants elected to scan magazines using the List screen (12 participants) over the dedicated Scan screen (8). Two participants started with the List screen and switched to the Scan screen mid-task. None used the Large cover screen for this subtask. When asked after the tests, most participants expressed a preference for the AR Scan (12) feature over the AR Find (6) feature in the AR App. The most common reasons to prefer the AR Scan feature were its speed and ease of use, and the expectation that in a real work situation, one would learn the physical locations of the magazines categories in the store within a week or so. Only five participants mentioned that having to hold the smartphone could be a challenge while performing the magazine tasks, while several have complained about the ergonomics of handling a paper list during the second sub-task.

# 6 Discussion

As expected, moving from registering magazines using a paper list, to using a smartphone app resulted in improved performance and, to some extent, user experience. However, the AR App itself did only show a performance improvement for the entire task. For the first subtask, registering magazines from a stack, the AR App performed the best of the prototypes. Test participants were offered a choice of three different screens where they could use the Scan functionality in the AR App. Interestingly, the majority chose the List screen, that used the cam-
era to identify and select magazines, but did not have a dedicated AR camera
viewport. This choice appeared to be primarily due to the List offering a better
overview of the remaining list of magazines, while the participant scanned them
one by one.

For the second subtask, finding and registering magazines from the magazine
stands, participants were not given the same freedom. Rather, they were to
use the single Find screen, which indicates the approximate location of all the
remaining magazines on their list. The intent was to explore if participants would
themselves choose a suitable order, as well as opportunistically pick up nearby
magazines. However, we found that this higher level of complexity in the AR
App did not perform well. Participants were often overwhelmed by the multiple
graphic elements on the screen (see the arrows on the right of Figure 2b for
an example), and would have preferred selecting a single magazine at a time to
locate. Moreover, the Find functionality can be appealing for new or temporary
employees, who are naive about the store layout and location of the magazine
categories, and possibly even for consumers, but not for the employees who are
used to manage the magazine inventory. They will quickly become familiar with
the layout of the store and typical location of the items.

Participants committed very few errors during the tests, and the number of
errors were also similar between prototypes. We found it interesting to see a few
cases where a magazine with similar cover image was picked instead of the target
magazine. This occurred for both the List App as well as the AR App. This type
of error appeared similar in nature to when users picked a magazine with similar
title, instead of the target magazine, when using the Paper List. Participants
were likely doing a quick pattern match (e.g. "Does the cover image contain a
row of band member faces?", "Does the title begin with 'Eisenbahn'?"), rather
than confirming that it was exactly the correct cover or title.

Overall the AR App performed the best when there was less information
to interpret for the participant, and a clear next action, like when registering
magazines in a stack. When presented with more complex graphics, and having
to make a decision on what to do next, performance decreased significantly, back
down to similar levels as the Paper List. Thus there seemed to occur a significant
mental mode shift for participants when going from using AR Scan to AR Find.
Some participants reported that they worked "on automatic" when using AR
Scan. AR Find then felt overwhelming, as the participant now was forced to
interpret the graphics and make decisions.

While using the AR App, most participants stayed focused on the smartphone
screen, as expected. But this also meant that they viewed their environment
through a relatively narrow viewport, instead of directly looking first at the
magazines they were scanning the magazine stands where they needed to find
more magazines. This behavior did not appear to have any adverse effect on how
fast they were able to register the magazines from a stack, but did afterwards
seem to impede how quickly they could find the magazines on the stands. There
the magazine covers were partially obscured by other magazines, and the distance
to the smartphone camera was typically greater, so the app could not help by
identifying magazines as fast and consistently as in the stack.

Generally speaking, the value of AR-based order picking appears greater when navigating a larger, complex space, with walls or tall shelves blocking the user's view. For example, a warehouse (as in [16, 17, 5]). Going to the wrong area costs time, both in terms of transport and then searching for the target product. In a magazine store environment, we found that AR is often more valuable as a means of directing the user's gaze towards the part of a magazine stand where the target product is likely to be, rather than providing wayfinding to the magazine stand itself. In the store, distances are short, and the product may already be visible from where the user is standing.

In lieu of having individually named locations for magazines, we used a
category-based method to show the approximate location where the target mag-
azine is likely to be. This method should also be applicable in other contexts
where object categories map to physical locations. One such context is a library.
The off-screen indicators and category area frames could be employed by library
visitor to find the shelf where a book is located. While the cover of the book
is unlikely to be visible, image recognition or even optical character recognition
(OCR) could be used to identify the book by the title on its spine. Both recog-
nition of the book's spine and front cover could be used when registering the
books to be borrowed, extending the ideas presented by [3].

A library visitor would be an occasional, relatively inexperienced user of such a handheld AR app. Similarly, the target user group for our magazine handling AR app concept is less experienced, temporary or part-time, store staff. They need a straightforward tool, that does not require lengthy, specialized training. Such an app could reduce the variance in how long it takes to perform inventory tasks between the experienced and less experienced staff. The AR Scan feature could benefit all, while a revised AR Find feature would be most valuable to new staff, who are not yet familiar with the store's magazine stands. As they become familiar over time, the AR Find feature would recede in importance.

6.1 Recommendations

Finally, based on the weaknesses that were identified in our prototype through
the user study, we compiled a set of recommendations, that we believe could im-
prove the prototype and may be useful for designers and researchers developing
mobile AR applications:

- AR features as a strong supporting cast - The basic list and cover image UI is a solid set of baseline functionality. The AR features should be available and quick to use when needed, but not dominate the user experience.

- Separation of registered magazines in list - As the return count is entered for magazines, move them to the top of the list, so that the remaining magazines are together at the bottom, near the Register button. This will continuously decrease the effective length of the list the user needs to pay attention to.
- List scan with toggle - Being able to scan magazines in the list (without a dedicated Scan screen) is sufficient to use the feature and was preferred by most users. Remove the Scan screen and add the ability to turn the scanning on/off on the List screen. Communicate a successful scan more explicitly to avoid accidental selections, by e.g. flashing the selected item, coupled with an audible beep.

- Numeric keyboard entry - Replace the +/- buttons with a numeric keyboard when a magazine is selected for entry. This will increase the physical size of the input buttons, and consistently place them in the same screen location. Also, if the scan feature is temporarily disabled until the keyboard is dismissed again, it effectively removes the risk of involuntarily scanning another magazine cover.

- Find selected magazine - Make Find accessible from a single selected magazine at a time, to reduce the number of magazines the user has to find at the same time. This addresses the problem that users often felt overwhelmed when information about all missing magazines were presented at the same time.

# 7 Conclusion

In this paper we addressed the question of whether handheld AR can serve as a practical and effective tool for inventory tasks. To answer the question, we applied a user-centered design process, starting with a round of field research. We interviewed magazine retail staff and observed them performing work tasks. This research informed the selection of a realistic task to be used as the basis for the design of a smartphone app prototype with AR features. This prototype was then evaluated in a user study, where it was also compared to the paper list currently in use, and a second, simplified app prototype, without AR features.

Based on the results of the user study, we conclude that the answer is yes. Handheld AR can indeed serve a practical and effective role in inventory tasks. However, our findings indicate that this role is best played as a supporting actor, rather than a dominant one, since it did not always improve performance and satisfaction of the users in our study. The limited handheld AR viewport afforded by a smartphone was most effective to use when the task itself was also focused, like when registering magazines in a stack.

# Acknowledgments

We wish to thank the store staff and prototype evaluation participants for their time, and the anonymous reviewers for their valuable comments. This research was supported by Bladkompagniet, which provided materials and compensation for the studies and participants. Their team participated in initial planning, field research recruitment, prototype evaluation environment construction and provided feedback. This research was also supported by Charlie Tango, which provided the infrastructure for the prototype evaluation. We wish to thank Ulla
Tønner, who provided valuable feedback on the design of the prototype evalua-
tion. Peter Mitts also wishes to thank Tessa, for her support during the devel-
opment of this project.

References

1. Bonetti, F., Pantano, E., Warnaby, G., Quinn, L., Perry, P.: Augmented reality in real stores: empirical evidence from consumers' interaction with ar in a retail format. In: Augmented Reality and Virtual Reality, pp. 3-16. Springer (2019)

2. Chatzopoulos, D., Bermejo, C., Huang, Z., Hui, P.: Mobile augmented reality survey: From where we are to where we go. IEEE Access 5, 6917-6950 (2017). https://doi.org/10.1109/ACCESS.2017.2698164

3. Chen, D.M., Tsai, S.S., Girod, B., Hsu, C.H., Kim, K.H., Singh, J.P.: Building book inventories using smartphones. In: Proceedings of the 18th ACM international conference on Multimedia. pp. 651-654 (2010)

4. Chung, J., Kim, I.J., Schmandt, C.: Guiding light: Navigation assistance system using projection based augmented reality. In: 2011 IEEE International Conference on Consumer Electronics (ICCE). pp. 881-882. IEEE (2011)

5. Cirulis, A., Ginters, E.: Augmented reality in logistics. Procedia Computer Science **26**, 14 – 20 (2013). https://doi.org/10.1016/j.procs.2013.12.003, iCTE in Regional Development, December 2013, Valmiera, Latvia

6. Dillman, K.R., Mok, T.T.H., Tang, A., Oehlberg, L., Mitchell, A.: A visual interaction cue framework from video game environments for augmented reality. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. CHI '18, ACM, New York, NY, USA (2018). https://doi.org/10.1145/3173574.3173714

7. Elia, V., Gnoni, M.G., Lanzilotto, A.: Evaluating the application of augmented reality devices in manufacturing from a process point of view: An ahp based model. Expert systems with applications **63**, 187-197 (2016)

8. Fang, W., Zheng, S., Liu, Z.: A scalable and long-term wearable augmented reality system for order picking. In: 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). pp. 4-7. IEEE (2019)

9. Glockner, H., Jannek, K., Mahn, J., Theis, B.: Augmented reality in logistics. changing the way we see logistics–a dhl perspective. DHL Customer Solutions & Innovation **28** (2014)

10. Guo, A., Raghu, S., Xie, X., Ismail, S., Luo, X., Simoneau, J., Gilliland, S., Baumann, H., Southern, C., Starner, T.: A comparison of order picking assisted by head-up display (hud), cart-mounted display (cmd), light, and paper pick list. In: Proceedings of the 2014 ACM International Symposium on Wearable Computers. p. 71-78. ISWC '14, ACM, New York, NY, USA (2014). https://doi.org/10.1145/2634317.2634321

11. Hart, S.G., Staveland, L.E.: Development of nasa-tlx (task load index): Results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, Advances in Psychology, vol. 52, pp. 139-183. North-Holland (1988). https://doi.org/10.1016/S0166-4115(08)62386-9

12. Kourouthanassis, P.E., Boletsis, C., Lekakos, G.: Demystifying the design of mobile augmented reality applications. Multimedia Tools and Applications **74**(3), 1045-1066 (Feb 2015). https://doi.org/10.1007/s11042-013-1710-7
13. Milgram, P., Takemura, H., Utsumi, A., Kishino, F.: Augmented reality: a class of displays on the reality-virtuality continuum. In: Das, H. (ed.) Telemanipulator and Telepresence Technologies. vol. 2351, pp. 282–292. International Society for Optics and Photonics, SPIE (1995). https://doi.org/10.1117/12.197321

14. Mitts, P., Debarba, H.G.: On the use of handheld augmented reality for inventory tasks: a study with magazine retailers - dataset (2021). https://doi.org/10.5281/zenodo.4926619

15. Porter, M.E., Heppelmann, J.E.: Why every organization needs an augmented reality strategy. HBR’S 10 MUST p. 85 (2017)

16. Reif, R., Günthner, W.A.: Pick-by-vision: augmented reality supported order picking. The Visual Computer 25(5), 461–467 (May 2009). https://doi.org/10.1007/s00371-009-0348-y

17. Reif, R., Günthner, W.A., Schwerdtfeger, B., Klinker, G.: Evaluation of an augmented reality supported picking system under practical conditions. Computer Graphics Forum 29(1), 2–12 (2010)

18. Robert, K., Zhu, D., Huang, W., Alem, L., Gedeon, T.: Mobilehelper: remote guiding using smart mobile devices, hand gestures and augmented reality. In: SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications. pp. 1–5 (2013)

19. de Sá, M., Churchill, E.F.: Mobile Augmented Reality: A Design Perspective, pp. 139–164. Springer New York, New York, NY (2013). https://doi.org/10.1007/978-1-4614-4205-9_6

20. Sauro, J., Dumas, J.S.: Comparison of three one-question, post-task usability questionnaires. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. p. 1599–1608. CHI ’09, ACM, New York, NY, USA (2009). https://doi.org/10.1145/1518701.1518946

21. Schwerdtfeger, B.: Pick-by-vision: Bringing hmd-based augmented reality into the warehouse. Ph.D. thesis, Technische Universität München (2010)

22. Soulard, O.: Development of an adaptive user interface for mobile indoor navigation. Master’s thesis, Lehrstuhl für Medientechnik, Fachgebiet Verteilte ... (2012)

23. Teo, T., Lawrence, L., Lee, G.A., Billinghurst, M., Adcock, M.: Mixed reality remote collaboration combining 360 video and 3d reconstruction. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. pp. 1–14. ACM, New York, NY, USA (2019). https://doi.org/10.1145/3290605.3300431

24. Vidal-Balea, A., Blanco-Novoa, O., Fraga-Lamas, P., Vilar-Montesinos, M., Fernández-Caramés, T.M.: Creating collaborative augmented reality experiences for industry 4.0 training and assistance applications: Performance evaluation in the shipyard of the future. Applied Sciences 10(24), 9073 (2020)

25. Wang, P., Bai, X., Billinghurst, M., Zhang, S., Zhang, X., Wang, S., He, W., Yan, Y., Ji, H.: Ar/mr remote collaboration on physical tasks: A review. Robotics and Computer-Integrated Manufacturing 72, 102071 (2021). https://doi.org/10.1016/j.rcim.2020.102071

26. Wang, W., Wang, F., Song, W., Su, S.: Application of augmented reality (ar) technologies in inhouse logistics. In: E3S Web of Conferences. vol. 145, p. 02018. EDP Sciences (2020)

27. Winkel, J.J., Datcu, D.D., Buijs, P.P.: Augmented reality could transform last-mile logistics. In: Symposium on Spatial User Interaction. pp. 1–2 (2020)