## 4. Actively Augmenting Mobile Devices
### Active Augmentation Framework
Beyond the aesthetic and philosophical implications of hybrid instruments (which are of great interest but are not the object of this paper), their practical goal is to leverage the benefits of physical and virtual elements to combine them. In practice, the digital world is more flexible and allows us to model/approximate many physical elements. However, even with advanced sensor technologies, it often fails to capture the intimacy (see Section 1.1) between a performer and an acoustic instrument allowing us to directly interact with its sound generation unit (e.g., plucked strings, hand drum, etc.) [13]. Thus, a key factor in the success of hybrid mobile instruments lies in the use of a physical/acoustical element as the direct interface for the performer, enabling passive haptic feedback and taking advantage of the randomness and unpredictability of acoustical elements (see Section 1.2). In other words, even though it is possible to combine any acoustical element with any digital one, we encourage instrument designers to use acoustical excitations to drive virtual elements (see Figure 12), implementing the concept of \"acoustically driven hybrid instruments\" presented in Section 1.2. While the single analog input available on most mobile devices allows for the connection of one acoustical element, having access to more independent analog inputs would significantly expend the scope of the type of instruments implementable with our framework. This remains one of its main limitation. Figure 11: Bidirectional connection between virtual and physical elements of a hybrid instrument. Figure 12: “Typical” acoustically driven mobile hybrid instrument model. ### Faust Physical Modeling Library

More than just a set of functions, the Faust Physical Modeling Library provides a comprehensive environment to implement physical models of musical instrument parts fully compatible with the hybrid instrument paradigm described in Section 5. This section summarizes its various features. #### 5.2.1 Bidirectional Block-Diagram Algebra

In the physical world, waves propagate in multiple dimensions and directions across the different parts of musical instruments. Thus, coupling between the constituting elements of an instrument sometimes plays an important role in its general acoustical behavior. In Section 5.1.3, we highlighted the importance of bidirectional connections to implement coupling between the performer, the physical, and the virtual elements of a hybrid instrument. While these types of connections happen naturally between physical elements, it is necessary to implement them when connecting virtual elements together. The block-diagram algebra of Faust allows us to connect blocks in a unidirectional way (from left to right) and feedback signals (from right to left) can be implemented using the tilde (-) diagram composition operation:

process = (A : B) ~ (C : D) ; where A, B, C, and D are hypothetical functions with a single argument and a single output. The resulting Faust-generated block diagram can be seen in Figure 13.

In this case, the D/A and the C/B couples can be seen as bidirectional blocks/functions that could implement some musical instrument part. However, the Faust semantics doesn't allow them to be specified as such from the code, preventing the implementation of \"bidirectional functions.\" Since this feature is required to create a library of physical modeling elements, we had to implement it. Bidirectional blocks in the Faust Physical Modeling Library all have three inputs and outputs. Thus, an empty block can be expressed as:

emptyBlock = _,_,_,_;

The first input and output correspond to left-going waves (e.g., C and D in Figure 13), the second input and output to right-going waves (e.g., A and B in Figure 13), and the third input and output can be used to carry any signal to the end of the algorithm. As we'll see in Section 5.2.2, this can be useful when picking up the sound at the middle of a virtual string, for example.