## 3 Materials and Methods
### The Virtually Enhanced Senses System
#### 3.1.2 System Architecture  
The VES system is a software application developed with Unity game engine (v2018.2.18f1, Unity Technologies), Vuforia (v8.0.10, PTC), Resonance Audio (v1.2.1, Google) and ARCore (v1.5.0, Google). It runs on a hardware platform composed of two modules (Figure 1): The equipment carried by the user, i.e., earphones and two smartphones; and a smartphone or laptop used as a server.  
As stated before, the system generates stimuli according to the movements of the user. In this regard, the ARCore-compatible smartphone tracks the head pose of the user in the environment, while the remaining smartphone tracks hand movements. These data are provided to a server within a few tens of meters' radius from the position of the user, data which are updated each 20 ms. Finally, the server is used to control the multisensory feedback and the scenario to be loaded. Also, it allows for one-way communication with the user through a built-in call feature.  
Once the system starts running, the headset waits for an anchor image which fixes a virtual scenario to a specific position in the real world. The movement of the user--head and hand--is then captured and synchronized with an avatar's in a virtual scenario.  
Some of the developed virtual scenarios are replicas of real environments, which were previously recorded with a 3D camera (Figure 2b). To assist navigation in real spaces, virtual and real environments are aligned by placing the anchor image in a preset position.  
All virtual scenarios are constructed with primitive geometries, such as cubes, spheres or cylinders, both to lessen computing costs and avoid unnecessary data for BVI people's navigation. These primitives are mapped to an acoustic material used to model reverberation, impact sounds, etc. Particularly, Resonance Audio has been used to simulate early and late reflections in different locations according to the arrangement of primitives (Figure 2a).  
Figure 1: System architecture.  
The acoustic feedback is generated according to the user's movement, the scenario loaded and the chosen algorithms which trigger the stimuli, i.e., output modes. VES can also provide visual feedback, a feature which is currently limited to on/off options. However, it offers post-processing of the virtual camera's output, which can be applied to simulate several visual impairments for sighted users or to enhance visually impaired users' residual vision.  
As for the output modes, the system currently supports the following:  
* Geometry-based Virtual Acoustic Space (GbVAS): This includes three novel main modes of stimulation based on the Virtual Acoustic Space project, and a speech synthesis module. It provides information concerning points 1, 3, 5 and 6 of Section 3.1.1. Further details will be described in the next sub-section;
* Acoustic Compass: This provides the user with a fixed orientation cue through spatialized audio. Therefore, it directly addresses point 4;
* Wall-tracking assistance: This novel mode, which was specifically requested by the end users, is meant to help them move parallel to a wall (points 3, 4). Once the user distances from a reference wall, a haptic warning will be triggered in the hand-held device, and a virtual sound source will be spawned in the closest point of the closest wall. The position of this sound source will be periodically updated according to the user's movement. When the user reaches a threshold distance, the virtual sound source will be turned off.