## 3 Materials and Methods
### The TFTH Architecture  
For each basic function, in TFTH, several functional modules are available that differ in performance, memory footprint, computational load and other characteristics. The choice among the different modules available for each function is driven by design constraints. Figure 1 shows the structure of the general TFTH architecture. The basic functional modules are devoted to speech input and output, face detection and recognition, estimation of head orientation and gaze tracking, natural language understanding and a smart assistant. The latter is responsible for connecting to smart home applications and devices and offers some Internet-based services similar to those supported by the most common smart speakers. A wake-word detection module is therefore added  
Figure 1: The TFTH architecture with the basic functional modules and one of the supported graphical characters. to allow the user to initiate a conversation with the IVA whenever speech interaction is used. The graphical front-end is based on a 3D-animated character that shows facial expressions, makes head gestures and approximately moves its mouth according to the spoken sentences thanks to an ad hoc algorithm. The set of predefined facial expressions and head gestures is expandable, but those already available, derived from [12], are sufficient for most applications. The choice of character can arguably influence the effectiveness of the IVA [32], and therefore, several different characters were developed and are available for experimentation. Finally, a coordination module is in charge of making decisions based on the data made available by the other modules. Depending on the application, the coordination module uses a suitable combination of input and output modules. For example, in the specific case of the MICA application considered here, the coordination module analyzes the gaze of the eye and the position of the head to determine the actions to take. Currently, the decision is made according to very simple rules described in Section 3.2. However, more accurate approaches based on deep learning are being investigated. The whole TFTH architecture was developed under the assumption that only local computation is desirable when user-related data are processed; that is, all the processing needed to interact with the user is supposed to run locally on the handheld device, and no data should be transferred to external servers or cloud services. An exception is acceptable only if the user explicitly requests it, such as when a Google or Wikipedia search is commanded. This assumption is motivated by the privacy and reliability reasons that are explained in [33]. As a consequence, the low computational complexity was one of the main constraints in the development of the TFTH modules. The speech synthesis module is based on a software library called pyttsx3 [34], which in turn leverages the speech synthesis engine available on the device (Espeak for Linux or SAPIS for Windows) or, for Apple devices, on a small Python wrapper for the native TTS engine [35]. The speech recognition module is based on the VOSK library [36], which provides adequate recognition functionalities for our purposes. The same library is used by the wake-word detection module. The face detection and recognition module, derived from the model described in [12], is based on the OpenCV library [37] for face detection and the Dlib library [38] for face recognition. Both libraries are open source and run smoothly on the considered handheld devices. Gaze tracking is a fundamental function for TFTH-based mobile applications. In particular, the ability to track the eye's gaze allows the MICA application to determine where the user is looking and therefore estimate the level of attention that the user is paying to the reading activity. However, very few handheld devices include dedicated hardware for eye tracking, and moreover, most devices directly supporting eye tracking are specific aids for impaired people.