## 3 Model for Tracking User Interactions in AR-Based Serious Games
### Conceptual Model
#### 3.1.2 Augmented Reality  
Since the focus in this work is on serious games for handheld devices, the AR features to consider for the user interaction tracking model are related to the tasks and interaction techniques commonly appearing in such solutions. These tasks can be grouped into the following five categories: observing AR content, establishing physical/virtual correspondence, creating virtual content, transforming virtual content, and activating virtual content [11]. Some of the interaction techniques for AR tasks identified in an overview of AR solutions developed for people with disabilities [22] are as follows: pointing the device's camera to a marker/image/text, directing the device's camera to the point of interest/location, tapping to select or position the object, drag and drop to translate the object, swipe to rotate, pinch to zoom to resize, click to change the colour, double-tap on an object to activate it, etc. One of the first steps in most AR solutions is to establish an AR scene by scanning the environment or a marker/image on which the virtual object appears. This would be considered a crucial event to be tracked to identify if a user has issues. By tracking parameters related to this task, such as the number of scanning attempts, the duration of the task and the size of the recognized surface, one can get an insight into the usability of this task in general, or accessibility measures needed for an individual user. Similarly, it is possible to define interaction parameters related to other tasks in AR. Table 1 brings examples of parameters to track in the events related to a particular AR task. The parameters listed in the table are the most often tracked objective metrics in software solutions to detect usability problems, especially in research studies that compare different interaction techniques for performing AR tasks [40; 41; 42; 43; 60]. In addition, these parameters are most often tracked in tasks related to 3D manipulation of objects, which represent one of the most critical tasks of mobile AR. In AR for mobile devices, the most common input method for interaction with virtual objects is touch gestures, i.e., certain finger movements on the touch screen. The type of touch gesture used depends on the task. For example, a touch gesture is used to select or place a virtual object in the environment, a drag and drop gesture is used to translate a virtual object, a pinch gesture is used to scale a virtual object and rotate two fingers (counter) clockwise gesture is used to rotate a virtual object. Certain touch gestures can be implemented in many ways, e.g., to rotate an object, in addition to the abovementioned gesture, one can also use one finger in the direction in which you want to rotate the object or rotate the object using two thumbs (using two hands) [61]. Although most applications use touch gestures, in some cases it is suggested to use indirect controls, i.e., buttons on the screen, in such a way that the user does not need to change the way of holding the device, and it is recommended to use the transparency of the buttons to avoid obscuring the AR scene [62]. Although the use of transparent buttons is recommended, it should be considered that for some users they may not be sufficiently visible, so the display of the UI should be adapted accordingly. In case when there is more than one interaction technique implemented for the same task, those techniques must not be conflicting [60]. Considering the above, in the model for tracking user interaction it is necessary to distinguish the type and number of interactions that the user used in the given AR task. Depending on the type of serious game, the interaction parameters should be appropriately identified. For example, in addition to the mentioned AR tasks (select or position, translate, rotate and scale), some games will require tracking an event of _activating virtual content_. It can be triggered by, e.g., a swipe-up gesture for a jumping effect.