## 3 Early Prototyping  
The goal of the early prototyping phase was to provide us with insights and knowledge about the realization possibilities and limitations of marker-based TUIs. We adopted a research-through-design inspired process similar to the one by [PERSON] et al. (2018). In this section we briefly describe some of the prototypes that we developed and the reflections that resulted. Figure 1 shows some of these early prototypes. The initial prototypes (Figure 1a) were very simple and meant mostly for a basic understanding of how different physical properties could be simulated. Additional prototypes started exploring more complex forms and manipulations. Figure 1b shows three different prototypes for a tangible tower. The top version explored the detail in the physical object so as to mimic the architectural detail of the tower. The middle and bottom versions allow users to \"open up\" the tangible and inspect its interior. The bottom version further includes a free-moving element to represent the tower's bell: as the tangible is moved  
Figure 1: Selection of early marker-based TUI for VR prototypes. this element will naturally swing both in the physical and in the virtual environment. A book object (Figure 1c) was also prototyped. As users flip through the pages, they can see visual renderings of the page and simultaneously hear the audio version. We also explored the use of markers in and in relation to the body. Figure 1d shows three of the prototyped ideas: the use of markers on the feet and on \"staircase steps\", on the fingers, and on the wrists. Markers on the body serve the main purpose of allowing the system to provide visual feedback about the user's body, but they could also allow simple gesture detection for input. We also explored various kinds of input controllers such as \"buttons\", gestural and tangible sliders, portals, and handheld controllers (Figure 1e-i). The button controller works by detecting the absence/presence pattern over a specific marker (Figure 1e). The gestural slider works by detecting an absence/presence pattern over a succession of markers (Figure 1f). The tangible slider works by dynamically displaying different markers by sliding over a strip with several pre-coded marker patterns (Figure 1g). The portal works by detaching a markers from a a surface and bringing it closer than a pre-defined threshold to the user's eyes (Figure 1h). Finally, the handheld controller makes use of the earlier physical bottle prototype and detects proximity to other objects or rotation of the controller itself through the succession of visible markers (Figure 1i). ReflectionsSome physical properties are easy to convey such as form, size, weight, and texture even with cheap materials. Other properties such as smell or acoustics are harder to imitate. Markers are hard to include in the surface of some objects such as the plastic bottle because they need to be on flat surfaces. In these cases, the physical object may need to be adapted in order to allow or facilitate tracking such as extending it beyond it natural limits (Figure 1a). In our prototypes, we used a standard smartphone without any special camera lenses. This means that the field of vision is limited and particularly noticeable in tangibles made up from several parts as the openable tower. Without enough markers in the Field of View (FOV), some parts of the virtual representation suddenly disappear. Despite these issues, the effect of holding a physical architectural mockup and being able to bring it closely to our eyes, opening it up to look inside and inspect minute details is very compelling. The book prototype resulted in an easy to manipulate tangible where the natural affordances of a book are mostly directly transported to the VE. We can easily imagine taking advantage of the richness of digital information while keeping the manipulation simplicity of the book. In addition, the form factor results in an inconspicuous object that \"belongs\" most anywhere without attracting attention to its use.