# Stability, Reliability, Compatibility: Reviewing 40 years of NIME design

John Sullivan, Marcelo Wanderley

<figure><img src="image_1.png" /></figure>

## To cite this version:

John Sullivan, Marcelo Wanderley. Stability, Reliability, Compatibility: Reviewing 40 years of NIME design. [Research Report] McGill University / Université McGill. 2018. hal-01745984

HAL Id: hal-01745984

https://hal.science/hal-01745984

Submitted on 28 Mar 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Stability, Reliability, Compatibility: Reviewing 40 years of NIME design

John Sullivan
IDMIL, CIRMMT
McGill University
Montreal, Canada
john.sullivan2@mail.mcgill.ca

Marcelo M. Wanderley
IDMIL, CIRMMT
McGill University
Montreal, Canada
marcelo.wanderley@mcgill.ca

## ABSTRACT

In the NIME community, many new digital musical instruments experience limited use in performance, or never progress beyond initial prototypes. For active, professional musicians, a DMI may not be viable unless basic standards for stability, reliability and compatibility have been met in the design process, ensuring trouble-free use during performance. Though literature has offered different frameworks and guidelines to inform the design of new instruments, there has been little to specifically address design choices that will adequately meet these standards. This paper presents a meta-review of proceedings from the ICMC and NIME conferences to see where and how these issues have been addressed previously. Collectively, the proceedings provide a historical account of the state of the art in DMI design, providing an ideal corpus for analysis. Our review, complemented by user survey data, highlights key factors for the design of new instruments to meet the demands of real-world use by professional musicians. We hope to add this perspective to an ongoing conversation about performing with DMIs, and narrow the gap between innovative new instrument design and prospective performers who would use them.

## Author Keywords

digital musical instruments, performance, stability, ICMC, NIME, review

## CCS Concepts

* Applied computing → Sound and music computing;

* General and reference → Surveys and overviews;

* Human-centered computing → Usability testing;

## 1. INTRODUCTION

While the NIME community is centered around research and development of new instruments and interfaces for musical and artistic production, many new digital musical instruments (DMIs) that are presented see limited use in real-world performances, while others never progress beyond initial prototypes [6]. One set of interrelated factors for the short lifecycle of a DMI has been identified as the lack of established instrumental techniques and repertoire, combined with the need for new forms of musical notation [3].

In addition to these, we consider a more fundamental set of factors and propose that basic issues of stability, reliability and compatibility may also preclude new instruments from successful and continued use.

For active and professional music performers, whose time may be stretched between concerts, rehearsals, travel, and more, these issues are critical. New technologies will be quickly discarded or passed over if they suffer from hardware or software instability, durability problems, or lack of compatibility with other instruments and performers. In the NIME community, a number of different frameworks and guidelines have been proposed to aid designers in the creation of new instruments [5]. Case studies in design evolution of certain instruments such as the Continuum [4] have highlighted aspects of design for stability and reliability in performance. However, there is a lack of systematic research that addresses these basic usability issues necessary for an instrument to meet the demands of real-world use.

To help bridge the gap between innovative new instrument design and the performers who would use them, we aim to identify areas that designers could target to adequately meet standards of instrument reliability in performance. Here we present an analysis of DMI design literature from 1977 to 2017 to understand where and how these issues have been addressed previously. We augment it with data from user surveys about new instrument use and abandonment. These sources highlight key focal points towards the development of new instruments that active musicians would be willing to work with, and are intended to contribute to a larger dialogue about DMI use in performance.

## 2. TERMINOLOGY IN DMI DESIGN

In order to construct a broad overview of how much attention has been paid to these topics and in what specific contexts, we conducted a linguistic analysis of proceedings from the International Computer Music Conference<fnref n="1" /> (ICMC) from 1977 to 2001, and International Conference on New Interfaces for Musical Expression<fnref n="2" /> (NIME) from 2001 to 2017. The ICMC proceedings included here served as a primary pre-NIME conference outlet for literature on digital instrument design, while NIME now represents the primary community and conference for the design and development of DMIs. Together, the two collections represent a focused and cohesive source of research for our topic, and a coherent corpus for review.

### 2.1 Methodology

The methodology for our review was based on techniques used by Jensenius in his analysis of “gesture” and associated

<figure><img src="image_3.png" /></figure>

Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Copyright remains with the author(s).

Research Report, Input Devices and Music Interaction Laboratory.
McGill University, 2018.

<footnote n="1">¹http://www.computermusic.org/page/23/</footnote>

<footnote n="2">²http://www.nime.org/</footnote>
<figure><img src="image_4.png" /><figcaption>Figure 1: Percentage of papers containing *stability* keywords. Note that a smaller selection of ICMC papers was included in our analysis. See Table 1 for distribution.</figcaption></figure>

terminology in [2]. First we performed a search through each year of proceedings to return the number of papers containing keywords relevant to our topic. This was followed by concordance and collocation analyses that yielded a ranked list of terms closely associated with the keywords. These were reduced through a coding process to reveal a set of key issues relevant to DMI stability.

## 2.2 Paper Selection

The first step in our analysis was to collect the proceedings for our entire corpus, which are freely available from the ICMC and NIME archives<fnref n="3" />,<fnref n="4" />. We analyzed data separately for each conference to accommodate differences in size of collections, overlap of years, and variation in how the papers were selected. To condense the large dataset of over 40 years, we present our results grouped in ten four-year blocks, with adjustments as noted.

From the ICMC, we used a collection of papers that had been selected as preparatory research for [4]. This included papers chosen by the authors on the topic of interface design and gestural control of music. This criteria is suitable for our purposes as it focuses on performance with new interfaces and instruments and leaves out unrelated ICMC topics such as computer music composition, aesthetics, musicology and more.

While the ICMC started in 1974, the first papers in our corpus appear in 1977. 1979 is skipped as there is no record of a conference being held that year. Therefore our first block of ICMC papers covers 1977, '78, '80, and '81. Blocks of 4 years continue successively to 1998 - 2001, which is the year NIME came into existence. In all, there were 2202 papers published in that time, from which we used 233 for analysis. Figure 1 shows the complete breakdown of total papers and those included in our review for each block.

In 2001, NIME got its start as part of the ACM Conference on Human Factors in Computing Systems (CHI)<fnref n="5" />. Citing the rapid evolution of technology and its applications in musical interface design, along with growing scholarship and interest in the area of musical expression [10], Poupyrev et al. [7] organized a CHI workshop dedicated to these topics, and NIME was born. Fourteen papers were presented in 2001, and the following year NIME expanded into a conference of its own.

Given NIME's central focus of new interface and instrument design for music performance and creation, we chose

<table><caption>Table 1: Number of papers included in our review</caption><thead><tr><th>Conference</th><th>Year</th><th>Total</th><th>Included</th></tr></thead><tbody><tr><td rowspan="6">ICMC</td><td>1977-1981</td><td>164</td><td>19</td></tr><tr><td>1982-1985</td><td>189</td><td>27</td></tr><tr><td>1986-1989</td><td>277</td><td>22</td></tr><tr><td>1990-1993</td><td>488</td><td>50</td></tr><tr><td>1994-1997</td><td>570</td><td>48</td></tr><tr><td>1998-2001</td><td>514</td><td>67</td></tr><tr><td>Total ICMC</td><td>1977-2001</td><td>2202</td><td>233</td></tr><tr><td rowspan="4">NIME</td><td>2001-2005</td><td>239</td><td rowspan="4">all</td></tr><tr><td>2006-2009</td><td>381</td></tr><tr><td>2010-2013</td><td>488</td></tr><tr><td>2014-2017</td><td>440</td></tr><tr><td>Total NIME</td><td>2001-2017</td><td>1548</td><td>1548</td></tr></tbody></table>

to perform our analysis on the entire corpus from 2001 to 2017, a total of 1548 papers. The first block of NIME papers covers 5 years including the 14 papers from the initial CHI workshop, with the rest grouped in four-year blocks up to 2017, as shown in Figure 1.

## 2.3 Keyword Occurrence

Our analysis began with a keyword search to identify papers containing terms relating to our topic of DMI stability. Six keywords were chosen that characterize the issues we are focused on: *stability*, *reliability*, *durability*, *compatibility*, *maintainability* and *robustness*. To automate the search, a utility script was written that returns the number and filenames of papers by year matching given search queries, which can be entered as command line arguments when it is run<fnref n="6" />. The results are shown in Figure 1 as the percentage of papers that contain the keywords. We removed *maintainability* from the table to preserve space, as it occurred in less than 0.5% of all papers, though it is included in the rest of the analysis.

We ran the same search a second time using a lemma list, which included syntactical variations of the same root word, (i.e., searching for *stability* would include all the variations of the lemma *stable*, including *stabilize*, *stabilization*, *instability*, *unstable*, etc.). However, we found this not to be useful, as the more generalized search returned many results that were not applicable to our topic. For example, the lemma of *reliability* is *rely*, which is commonly used in a wide variety of situations that fall well outside of our concern. Therefore the search was limited to only the specific

<footnote n="3">³https://quod.lib.umich.edu/i/icmc/bbp2372.*</footnote>

<footnote n="4">⁴www.nime.org/archive</footnote>

<footnote n="5">⁵https://chi2018.acm.org/</footnote>

<footnote n="6">⁶https://github.com/johnnyvenom/NIME_keyword</footnote>
<figure><img src="image_5.png" /><figcaption>Figure 2: Percentage of papers containing *performance* keywords. For comparison, the smaller rightmost bar indicates the percentage of papers containing *stability* keywords.</figcaption></figure>

terms.

The results revealed that the list of terms occurred infre-
quently in NIME literature. Across the entire corpus, the
most common of the terms, *stability*, occurred under 8% of
all papers, while the others appeared even less. However,
when viewed across all years, we can clearly see an upward
trend especially from 2001 on. The most recent block shows
that 27% of papers contained at least one of our keywords,
which is an encouraging sign.

2.3.1 Performance Keywords

For comparison, we ran the search again with a list of
performance-related keywords: performance, performing, con-
cert, stage, and professional, shown in Table 2. The results
show that, with the exception of the first two ICMC blocks,
around 90% of papers mentioned performance. This con-
trast suggests that use of new instruments is a fundamental
preoccupation within the research community, yet discus-
sion of the basic, practical qualities an instrument must
possess to reliably achieve that goal is less prevalent.

On the other hand, less than 20% of the literature men-
tions *professional*. This is also an important consideration,
as it may suggest that designing for professional use is not
currently a strong motivation in NIME. However, frequency
of the term has steadily increased each year, so a trend in
this direction can be inferred.

## 2.3.2 Comments

It deserves noting that the keyword search results are not
exact. The selection of terms is far from exhaustive, and
likely overlooks papers that deserve to be included but use
slightly different terminology. Conversely, the keywords are
general enough that results may include certain usage that
falls outside of our intended context (for instance, a discus-
sion about the *stability* of an oscillator may not have bearing
on the more general topic of DMI stability for performance).

Also, as previously mentioned, we ran our analysis on the
entire archive of NIME proceedings without filtering like
we had before with ICMC. NIME literature covers a broad
range of specialized topics, some of which may be unrelated
to our discussion. Thus, the search offers a general overview
of how frequently these issues are mentioned, and provides
a subset of the literature that we can analyze in greater
depth.

## 2.4 Concordance Analysis

To check the context of how these terms were used, and
to generate a list of associated terms, a concordance was

created for the keywords appearing in the literature. The
.pdf versions of the papers containing keywords were con-
verted to plain text using a free application PDF2Text Pi-
lot<fnref n="7" />. Then they were processed with CasualConc<fnref n="8" />, a linguis-
tic analysis tool, which showed each keyword occurrence in
its original context. A manual search and sorting of the
surrounding text produced a list of associated terms that
were common to the keywords. The premise was that these
terms would indicate some of the specific ways that the topic
of DMI stability has been addressed in DMI research, and
the frequency of terms could infer the historical importance
through the literature. The list was split into three cat-
egories of terms: quality attributes (including the original
search terms), functional and descriptive terms, and specific
areas of focus. The top results of each category can be seen
in Table 2.

<figure><img src="image_6.png" /><figcaption>Figure 3: Related terms from collocation analysis.</figcaption></figure>

Along with the concordance analysis, we also ran a col-
location analysis, which yielded a ranked list of words that
appeared directly before and after the keywords in the texts.
A stop list was used to filter out common words that that
are not relevant to our topic [8], and the results were com-
piled into a ranked list of terms for each keyword.<fnref n="9" /> The top
results are displayed in a word cloud in Figure 3.

<footnote n="7">⁷http://www.colorpilot.com/extract-pdf-text.html</footnote>

<footnote n="8">⁸https://sites.google.com/site/casualconc/Home</footnote>

<footnote n="9">⁹Results from collocation, concordance and keyword analy-
ses available at https://goo.gl/dzdmR9</footnote>
<table><caption>Table 2: Concordance analysis results</caption><thead><tr><td>Quality Attributes</td><td>Occurrences</td><td>Functional/Descriptive</td><td>Occurrences</td><td>Specific Areas</td><td>Occurrences</td></tr></thead><tbody><tr><td>stability</td><td>250</td><td>system</td><td>82</td><td>sensors</td><td>46</td></tr><tr><td>reliability</td><td>179</td><td>performance</td><td>49</td><td>instrument</td><td>29</td></tr><tr><td>robustness</td><td>161</td><td>design</td><td>36</td><td>data</td><td>27</td></tr><tr><td>compatibility</td><td>82</td><td>control</td><td>35</td><td>computer</td><td>26</td></tr><tr><td>instability</td><td>43</td><td>order</td><td>24</td><td>pitch</td><td>25</td></tr><tr><td>durability</td><td>36</td><td>parameter</td><td>22</td><td>hardware</td><td>23</td></tr><tr><td>accuracy</td><td>20</td><td>latency</td><td>19</td><td>sound</td><td>22</td></tr><tr><td>maintainability</td><td>17</td><td>problems</td><td>18</td><td>device</td><td>21</td></tr><tr><td>usability</td><td>12</td><td>noise</td><td>14</td><td>software</td><td>18</td></tr><tr><td>longevity</td><td>4</td><td>issues</td><td>13</td><td>MIDI</td><td>16</td></tr><tr><td>correctness</td><td>3</td><td>terms</td><td>13</td><td>motion</td><td>16</td></tr><tr><td>ease-of-use</td><td>3</td><td>level</td><td>12</td><td>time</td><td>15</td></tr></tbody></table>

<table><caption>Table 3: Key points in designing for stability</caption><tr><td>Physical design</td><td>Technology</td><td>Build quality</td></tr><tr><td>Feature set</td><td>Compatibility</td><td>Performance control</td></tr><tr><td>Functionality</td><td>Usability</td><td>Sustainability</td></tr><tr><td>Musicality</td><td>Timing & latency</td><td>Product support</td></tr><tr><td>Feedback</td><td>Communication</td><td>Versatility</td></tr></table>

## 2.5 Results

Both concordance and collocation analyses performed similar functions to identify related words from the corpus. However the analysis methods varied considerably. The terms that emerged from the concordance results were compiled manually and involved a certain amount of subjective judgment, while the collocation was a strict quantitative analysis.

To bring the results together, all terms produced from both processes were sorted and grouped though an informal coding process to extract key focal points from our search. The process yielded 15 terms, shown in Table 3. Collectively, we feel these terms characterize the most significant issues and aspects relating to DMI stability and reliability for performance.

## 3. SURVEYS AND USER FEEDBACK

To complement our review, we look at user feedback from two surveys of performers and instrument makers who work with DMIs.

In [9], Sullivan conducted a survey to gain a general understanding of factors that influence performers' adoption, continued use, or rejection of new technology. An on-line questionnaire collected information from musicians about their background, training, primary instrument and styles of music they perform, along with answers to specific questions about DMI use. The survey received over 100 responses, with 43% of participants identifying as professional musicians. Rock and popular music were the most prevalent musical styles, followed closely by experimental and computer music. Instrument choice was dominated by rock instruments: guitar, bass, drums and keyboards/piano, and 76% of respondents had been playing that instrument for 10 years or more.

### 3.1 Uptake

As shown in Figure 4, respondents indicated that they commonly learned about new instruments by experiencing it from someone else – a friend, bandmate, or seeing it used live in performance. This could indicate a challenge for DMI designers as their instruments are more likely to exist in re-

search and prototype phases than commercial instruments. However, this assumes that the designer intends for them to be used in performance, which is often not the case.

<figure><img src="image_7.png" /><figcaption>Figure 4: How do you learn about new digital and electronic instruments?</figcaption></figure>

In [6], Morreale and McPherson report on a survey of DMI makers with published works in NIME between 2010 and 2015 about new instruments they had built. The objective of the survey was to find out what their motivations were for building them, and to give information about the instruments' current state and how often they are used. The responses indicated that most instruments had not experienced much sustained use beyond their initial build and demonstration. The designers' original motivations were commonly not for performance at all, but for other reasons such as research, technology tests, in-progress prototypes, or academic exercises.

### 3.2 Abandonment

Another question asked on the first survey by [9] relates closely to our discussion about DMI stability for performance. Participants were asked if they had stopped using certain DMIs and if so, why. Several of the answers, chosen from a list and shown in Figure 5, are consistent with the factors like compatibility, functionality, product support maintenance and build quality. The answers that they wrote in under "Other" were relevant as well:

* "Instruments fell apart too easily, and I ran the risk of damaging them beyond repair or during a live performance."

* "Age and better tech option"

* "Difficult to maintain and keep current with collaborating technology"
* "New, better technology"

* "Do not run with current OS"

* "Cost too much to repair."

* "The company stopped firmware updates for it."

<figure><img src="image_8.png" /><figcaption>Figure 5: What factors influenced you to discontinue using an instrument or new technology?</figcaption></figure>

In their survey, Morreale and McPherson found that of the respondents' instruments that had been built for performance, just 1/2 of them were currently in playable condition. Reasons included lack of time, attention or interest, outstanding hardware and software maintenance issues, and dissatisfaction with the instrument.

User data from these surveys show that lack of use and abandonment of DMIs can be attributed to a variety of reasons. It comes as no surprise that, coming from a research-minded community like NIME, some instruments are simply never intended to go beyond prototype or developmental stages. This is understood not as a failure but part of the research and design process that may extend well beyond the design of a single instrument. However, both surveys also indicate that issues of stability, reliability and compatibility are prime factors that can easily contribute to the abandonment of an instrument intended to be used in performance.

## 4. CONCLUSION

In this paper, we have provided a meta-review of literature from over 40 years of DMI development and performance. Our analysis of nearly 1800 papers from the ICMC and NIME conferences showed that, despite the lack of dedicated attention, issues of stability, reliability and compatibility are common hurdles that instrument designers have to overcome if their instruments are to be put into use.

Data from two user surveys offer some context for our review. They provided evidence to support our observation that many DMIs are not used extensively beyond their design and initial demonstration. However, we can't assume that this is always an issue the instruments themselves, as performance is not always a strong motivation for new instrument design.

In [1], Cook wrote that “musical interface construction proceeds as more art than science, and possibly this is the only way it can be done”. DMIs are often idiosyncratic and highly experimental in their design, and are not widely used in mainstream professional performance contexts. While a professional musician might not be the prototypical target user for most DMI creations, designing for this type of user can be beneficial. An important part of NIME research is dedicated to performance with DMIs, and the community benefits when DMIs are put into use. By considering active, professional musicians who have to depend on the gear they use, we hope to highlight these design considerations that can lead to the creation of robust, reliable instruments ready for their moment in the spotlight.

## 5. REFERENCES

[1] P. R. Cook. Principles for Designing Computer Music Controllers. In *Proceedings of the 2001 International Conference on New Interfaces for Musical Expression*, pages 3–6, Seattle, 2001.

[2] A. R. Jensenius. To gesture or Not? An Analysis of Terminology in NIME Proceedings 2001–2013. In *Proceedings of the International Conference on New Interfaces for Musical Expression*, pages 217–220, London, 2014.

[3] C. Mamedes, M. Rodrigues, M. Wanderley, J. Manzolli, D. H. L. Garcia, and P. Ferreira-Lopes. Composing for DMIs - Entoa, a Dedicate Piece for Intonaspacio. In *Proceedings of the International Conference on New Interfaces for Musical Expression*, pages 509–512, London, 2014.

[4] E. R. Miranda and M. Wanderley. *New Digital Musical Instruments: Control and Interaction Beyond The Keyboard*. 2006.

[5] F. Morreale, A. D. Angeli, and S. O'Modhran. Musical Interface Design: An Experience-oriented Framework. pages 467–472, London, jun 2014.

[6] F. Morreale and A. McPherson. Design for Longevity: Ongoing Use of Instruments from NIME 2010-14. In *Proceedings of the International Conference on New Interfaces for Musical Expression*, pages 192–197, 2017.

[7] I. Poupyrev, M. J. Lyons, S. Fels, and T. Blaine Bean. New Interfaces for Musical Expression. *ACM Conference on Human Factors in Computing Systems (CHI)*, page 491, 2001.

[8] A. Rajaraman and J. D. Ullman. Data Mining. *Mining of Massive Datasets*, pages 114–142, 2011.

[9] J. Sullivan. *Interaction and the Art of User-Centered Digital Musical Instrument Design*. Masters thesis, University of Maine, 2015.

[10] M. M. Wanderley and M. Battier, editors. *Trends in Gestural Control of Music*. IRCAM - Centre Georges Pompidou, Paris, France, 2000.