<figure><img src="image_1.png" /></figure>

HAL
open science

# Detection of orchestral blends in musical scores

Francesco Maccarini, Olivier Anoufa, Mathieu Giraud, Florence Levé, Stephen Mcadams

► To cite this version:

Francesco Maccarini, Olivier Anoufa, Mathieu Giraud, Florence Levé, Stephen Mcadams. Detection of orchestral blends in musical scores. Sound and Music Computing (SMC 2025), 2025, Graz, Austria. hal-05049228

HAL Id: hal-05049228

https://hal.science/hal-05049228v1

Submitted on 13 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# DETECTION OF ORCHESTRAL BLENDS IN MUSICAL SCORES

Francesco MACCARINI<fnref n="1" />, Olivier ANOUFA<fnref n="1" />, Mathieu GIRAUD<fnref n="1" />, Florence LEVÉ<fnref n="2" />,<fnref n="1" />, and Stephen MCADAMS<fnref n="3" />

## ABSTRACT

In music orchestration, *orchestral blend* refers to the process in which multiple instruments, when played together under certain conditions, merge their individual sounds to form a cohesive and distinct timbre. Despite being primarily a perceptual phenomenon, previous studies have shown that some compositional aspects of music, such as the synchronization between different instrumental parts, can promote the generation of orchestral blends. Those characteristics can be exploited by composers who want to write blends. We hypothesize therefore that blends can be detected starting from the score only. In this paper, we propose a machine learning approach with perception-informed features to identify blending instruments in orchestral scores. We propose and use several evaluation metrics and show that our model improves the state-of-the-art algorithm. Moreover, we show that a simple algorithm based on comparing the number of notes played concurrently by two instruments is sufficient to obtain better results on these metrics than the state-of-the-art algorithm. This study confirms with a score-based approach the impact of music cues such as synchrony, harmonicity, parallelism, and other descriptors of the similarity between instrumental parts on the perception of orchestral blends.

## 1. INTRODUCTION AND GOALS

In the context of music orchestration, an *orchestral blend* refers to the process in which multiple instruments are playing at the same time, and their individual sounds fuse together to generate a new distinct combined timbre [?]. More precisely, this happens when the listener cannot identify or distinguish the different sources, that is to say, the different instruments that are playing at the same time loose part of their individuality. A blend can be intentional or not. For a composer or an orchestrator, achieving a good blend involves arranging the instruments in a way that their individual timbres complement each other, creating a unified and cohesive sound that often gives the impression of originating from a single source [?].

While orchestration has long been considered a purely artistic practice (“art and not a science” [?, p. 356]), perception scientists advocated for the integration of principles

from perception studies into music theory and composition practice, arguing against dismissing scientific approaches to studying orchestration as completely unattainable or undesirable [?]. Studies in music perception focused on how instrument combinations affect timbral response, on *blending* qualities of instruments (based on their own and combined timbres), and on how composers can combine timbres to create new sounds [?, ?].

The Taxonomy of Orchestral Grouping Effects (TOGE) [?] gives a systematic hierarchical description of the auditory phenomena stimulated by orchestral music. In this context the blending process (*concurrent grouping*) is situated at the lowest level of a hierarchical organization, being at the origin of two other classes of auditory grouping processes (*sequential* and *segmental grouping*). In the TOGE, blends are of two types: *timbral augmentation*, when the characteristic timbre of one instrument is enriched and amplified by the other blending instruments, and *timbral emergence*, when a completely new timbre is perceived. Finally, this phenomenon can be *sustained* or *punctuated*, depending on its duration, and if it is sustained, it can be *stable* or *transforming*.

Furthermore, studies have investigated which cues contribute to the perception of simultaneous musical elements, resulting in the formation of blends and streams. Those include onset synchrony, meaning that simultaneously starting sounds fuse better [?, ?]; harmonicity (or principle of tonal fusion), which means that sounds separated by unisons, octaves, and, to a lesser degree, other consonant intervals blend better; and parallel changes in dynamics and pitch (co-modulation) [?, ?, ?]. The spectral overlap between different sound sources has been identified as a contributing factor [?, ?], suggesting that instruments with similar timbral characteristics tend to blend better. The spatial disposition of the instruments, the characteristics of the performance room, and other factors associated with the musicians’ performance can also influence the degree of blending [?, ?, ?]. Although composers and orchestrators may have more control on the first four factors, they may indirectly control the three last ones, which are also more dependent on specific performances.

Additionally, Cambouropoulos [?] considers two other principles that pertain to the arrangement of sounds in time: temporal continuity, which refers to the use of continuous or recurring sounds rather than brief or intermittent ones, and pitch proximity, which indicates that successive tones with close pitches better maintain the coherence of the stream. In previous work, we also proposed to study

<footnote n="1">¹Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRISTAL, F-59000 Lille, France</footnote>

<footnote n="2">²MIS, Université de Picardie Jules Verne, F-80000 Amiens, France</footnote>

<footnote n="3">³Schulich School of Music, McGill University, Montreal, Canada</footnote>

<footnote>Copyright: © 2025. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 Unported License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.</footnote>
symphonies as a superposition of *orchestral layers*, that are constructed based on streaming and blending principles, but that can extend systematically to every measure of a score [?]. This allows their use in a measure-level analysis and in co-creative orchestration [?].

The only study that we are aware of to *detect* orchestral blends in scores was proposed by Antoine et al. [?], and consists of an algorithm that exploits onset synchrony, harmony, and parallelism in pitch and dynamics to identify orchestral blends from scores. It selects the blending parts with a system that includes three successive filters linked to three scoring algorithms, applied to every measure of the score. First, parts are filtered by onset synchrony. A synchrony score is calculated, and the synchronous groups are passed to the harmony filter. This second filter checks at every onset if the synchronous notes are in harmonic series or if they are part of a tonal chord. Then a harmony score is computed for all instruments that are either playing in a harmonic series or forming a tonal chord. Finally, the parallelism filter checks if the parts in the groups are parallel in pitch and if they follow the same dynamics curve. A parallelism score is calculated. The final score is computed as the average of the three scores of the three steps, and a group of instruments is retained as a potential blend if the score is higher than a given threshold. By considering each blend in each measure as one distinct event, they claim that the model can successfully retrieve on average 80.8% of all instruments constituting a blend. However, their (manual) evaluation allows multiple retrieved blends to be matched to the same target blend. In Section 4.2 we argue that this many-to-one matching is to be avoided, and we detail an automatic method to match each target blend to a maximum of one retrieved blend. Using this stricter evaluation criterion, their score falls to 69.7%.

In this study, as in Antoine et al. [?], we hypothesize that some of the musical features that give rise to perceptual phenomena are already present and discernible in the score, regardless of whether they were intentionally planned by the composer. Our objective is to examine the extent to which this assertion holds true for orchestral blends and to identify the insights that can be extracted about the characteristic attributes of blends. Differently from Antoine et al. [?], we employ machine learning instead of fixed rules to model the dependency of a blend from musical cues, including features to describe the spectral characteristics of instrumental sounds. We also carry out a more severe and extensive evaluation with different metrics to better capture the musical meaning of the different kinds of errors that the models can make.

In the following, we present the data, using the same ground truth as Antoine et al. [?] (Sec. 2) and our modeling approach (Sec. 3). We discuss metrics for the evaluation of the detection of blends (Sec. 4), present our results (Sec. 5), and finally conclude with some considerations on the impact of this work for music perception (Sec. 6).

<figure><img src="image_3.png" /><figcaption>15, 17, BL, DI, [Violin1]</figcaption></figure>

<table><caption>15, 17, BL, EI, [Flute1, Flute2, Oboe1, Oboe2, Bassoon1, Bassoon2, Viola, Cello, Bass]</caption><thead><tr><td>instrument 1</td><td>instrument 2</td><td>measure</td><td>target</td></tr></thead><tbody><tr><td>Flute 1</td><td>Oboe 1</td><td>15</td><td>1</td></tr><tr><td>Flute 1</td><td>Violin 1</td><td>15</td><td>1</td></tr><tr><td>Violin 1</td><td>Violin 2</td><td>15</td><td>0</td></tr></tbody></table>

Figure 1. Timbral augmentation blend in Mozart’s *Don Giovanni Ouverture* (meas. 15-17): score, OrchARD annotation, and ground truth encoding. A blend (BL) is annotated between the dominating instrument (DI, Violin 1) and the embellishing instruments (EI) in the measures from 15 to the start of 17. We encode each measure as a separate line in the ground truth table.

## 2. DATA

**Annotations.** The Orchestration Analysis and Research Database (OrchARD)<fnref n="1" /> contains expert annotations of orchestral effects, linked to scans of printed scores and recordings. The database contains annotations about many orchestral effects, including segregation, stratification, contrasts, and gestures, but here we only focus on detecting blends using OrchARD annotations as ground truth to train and evaluate our model. Blends of different kinds are represented differently in OrchARD, but in this project we avoid the distinction between timbral augmentation and timbral emergence, and we only focus on detecting the whole group of instruments of a blend. In the study by Antoine et al. [?], a handmade post-processing was used to determine the exact components of transforming blends at every measure of a piece. We include this information to build our target blend.

**Scores.** The company OrchPlayMusic<fnref n="1" /> maintains a catalog of high-quality MusicXML scores from a diverse range of composers and periods in music history, including Mozart, Berlioz, Debussy, and Vaughan Williams. A subset of their catalog overlaps with OrchARD and has also been used in the study by Antoine et al. [?]. We have been kindly provided with the same files, consisting of 32 excerpts of scores, totaling to 1650 measures of music. Our experiments have been performed on this set of excerpts, allowing us to make precise comparisons with the state-of-the-art.

<footnote n="1">¹https://orchard.actor-project.org/about/</footnote>

<footnote n="1">¹https://www.orchestraplayer.com/</footnote>
the-art algorithm.

**Spectral Descriptors.** We also included spectral descriptors (see Sec. 3.2.4) that are connected to instrumental timbre as features. These have been shown to affect blend perception [?]. To this end, we started from a dataset of spectral descriptors of different instruments that was constructed by Kazazis et al. [?] using the Timbre Toolbox [?, ?] and samples from the Vienna Symphonic Library (VSL),<fnref n="2" /> that we have expanded with additional instruments. Each timbre descriptor is characterized by its median value and inter-quartile range throughout the time evolution of the sound. The dataset contains all those descriptors computed on the whole playing range of every instrument, which means that each descriptor has a value for the pitch and dynamic level of each note.

## 3. MODELING BLENDS

The problem of modeling blends is framed as a binary classification task (supervised Machine Learning), consisting in assigning a blend or a non-blend label to every pair of instruments at every measure of the considered corpus. The choice of the measure as the time window for the analysis is motivated by its successful use in the algorithm by Antoine et al. [?] and by the fact that blends are annotated with measure granularity in OrchARD.

Our pipeline consists of encoding the ground truth for binary classification (Sec. 3.1), computing handcrafted features to be used in the classification model (Sec. 3.2), and training and evaluation (validation and testing) of the machine learning models (Sec. 3.3).

### 3.1 Ground Truth Encoding

We encode the blends annotations from OrchARD in a ground truth table where each row corresponds to a pair of instruments at a given measure, and which contains the target value for the binary classification task (blend vs non-blend), which has value 1 if the two instruments belong to the same blend and 0 otherwise (see Fig. 1). Of the 614,403 rows of the table, 6.8% are blends and 93.2% non-blends.

### 3.2 Feature Engineering

All features are computed at the measure level for a pair of instruments and are extracted from the musicXML scores using the Python library music21 [?]. They are: simple note distribution statistics, the onset synchrony score, the harmonicity score, the pitch parallelism score, the difference between the cosine contours, the instrument names, and spectral descriptors of the instruments. The rest of this section presents further explanations on their computation.

#### 3.2.1 Note distribution statistics

We first compute, for each instrument independently, the number of notes in a measure, the median pitch in the measure, and the pitch range in a measure, which consist in the

interval between the lowest and the highest pitch expressed in semitones. Then we compute the absolute difference between their values for every instrument pair and obtain the features note_countDifference, median pitch difference, and pitch range difference.

#### 3.2.2 Onset Synchrony, Harmonicity, Pitch Parallelism

Two parts are said to be synchronous if their notes share the same onset value and duration. We denote by s their number of synchronous notes.

They are said to be in harmony when the notes they play are in a consonant interval. We give a score $h_j$ to every couple of notes j that are overlapped for a duration $d_j$. The question of determining what the consonant intervals are and why they are consonant is still debated [?], but we adopt here a common convention [?, ?]: $h_j$ takes value 1 for perfect consonance (intervals of unison, octave, fifth and fourth), 0.5 for imperfect consonance (third and sixth), and 0 for dissonance (second, seventh and tritone).

Finally, two parts are said to be parallel in pitch when successive notes move in the same direction. We here call $O_i = \{o_{i,1}, o_{i,2}, \dots\}$ the set of the onsets of the notes of the instrumental part i in the given measure, and $Z_i = \{(o_{i,1}, z_{i,1}), (o_{i,2}, z_{i,2}), \dots\}$, i.e. the set of couples of onset and direction, the direction being in $\{\downarrow, \to, \nearrow\}$. We exclude the first note of the measure to avoid crossing to the previous measure.

The three features are then computed with a normalization ensuring that they take values in the interval [0, 1]:

$$ \text{synchrony} = \frac{s}{\max(\#\text{notes}(i_1), \#\text{notes}(i_2))}, $$

$$ \text{harmonicity} = \frac{\sum_j h_j d_j}{\sum_j d_j}, \quad \text{parallelism} = \frac{\#(Z_1 \cap Z_2)}{\#(O_1 \cup O_2)}. $$

#### 3.2.3 Cosine Contour Difference

The cosine contour is a compressed representation of melodic contours proposed in [?]. We consider it as an alternative method to evaluate pitch parallelism, or the similarity between the shapes of two instrumental parts. A melodic contour can be thought of as a step function that describes the height in pitch of the notes during time. The cosine contour is computed by taking the discrete cosine transform (DCT) of the melodic contour function. The obtained coefficients provide a condensed abstract description of the shape of an instrumental part without precise information on pitches and rhythm. The coefficient $c_0$ describes the average pitch of the part, low order coefficients describe the general shape of the contour (the authors of [?] call $c_1$ and $c_2$ "descendingness" and "archedness", respectively), while higher order coefficients describe high frequency variations of the contour.

We consider for our model the difference between the components $c_1$ and $c_2$ between two instrumental parts. If $c_k^i$ is the $k$th component of the cosine contour for instrument i, then the cosine contour difference features are

<footnote n="2">²https://www.vsl.co.at/</footnote>
$$cs_dist_1 = |c_1^{i1} - c_1^{i2}|, \\ cd_dist_2 = |c_2^{i1} - c_2^{i2}|.$$

We exclude the coefficient $c_0$, which corresponds to the average pitch, since we already included the median pitch in our features, and we discard also all coefficients $c_k$ with $k > 2$. The two features can take values in the interval $[0, +\infty]$. We expect that the more two parts are different in shape, the larger the differences in cosine contour coordinates, the less they are likely to blend.

### 3.2.4 Instrument Names and Spectral Descriptors

A way to consider instrumental timbres is to provide the model with the names of the instruments (with a one-hot encoding). Through this simple information, a model can learn from data which instruments tend to blend better together.

We also include the values of some spectral descriptors, that are known for having an effect on instrumental blending [?, ?]. We include the first four moments of the spectrum (centroid, spread, skewness, and kurtosis), two descriptors of the tonal quality of the sounds (flatness and crest), three descriptors of the spectral slope (slope, decrease, and roll-off), and two descriptors of the mutation of the spectrum in time (variation and flux), as defined and computed by the Timbre Toolbox [?, ?].

Here, these features are not computed on a recording, but rather on measure fingerprints computed from the score. For this, we use a dataset of precomputed spectral descriptors for different instruments (see Sec. 2). Each measure is represented by the precomputed spectral descriptor of the median pitch, regardless of the dynamic level (Fig. 2). We then compute the absolute difference between each spectral descriptor's mean and inter-quartile range (IQR) for all the pairs of instruments. The obtained features represent some sort of timbre distance between the two parts on the studied measure, and are negatively correlated with the target (Fig. 3).

<figure><img src="image_4.png" /><figcaption>Figure 2. Spectral centroid median of the violin samples from the Vienna Symphonic Library (small blue dots). We compute the median value per pitch (big red dots), selected among multiple points that record different dynamics and playing techniques for the same notes. The spectral centroid depends on the pitch, being higher than its overall median value (horizontal dotted red line) for high pitches, and lower for low pitches.</figcaption></figure>

## 3.3 Training and Evaluating Strategy

We train and compare several machine learning models on the classification task for instrument pairs at a given measure of a piece, using as input the features described in the previous section, for which we can observe a significant correlation with the target variable (Figure 3).

<figure><img src="image_5.png" /><figcaption>Figure 3. Synchrony and parallelism have strong positive correlation with the target variable. Most of the other continuous features have negative correlation with it.</figcaption></figure>

To train the models, we skip the samples that correspond to instruments that are silent in the given measure and that for this reason have generated missing values for some pitch-dependent features (107,427 out of the 614,403 rows are retained). This partially rebalanced the proportion of 1 and 0 in the target variable to 31.8% and 68.2%, respectively. At inference, we directly output a prediction of 0 for those samples, since the silent instruments should not blend. Some of the computed features may take infinite values, which we replace with the maximum value for the feature both at training and inference time, allowing all models to handle the data point with no distortion of the results. In our experiments, we compare between different models and different combinations of the features. We used simple statistical models: regularized logistic regression (log_regr) and decision tree (tree_D), with a maximum depth parameter $D \in \{1, 2, 3, 4, 5, 7, 10, 15, 20, 25\}$. Those models are lightweight (each of them has less than 1500 parameters) and can be quickly trained and validated on any laptop CPU, allowing us to implement a cross-validation scheme. We select 3 of the 32 pieces (totaling to 109 out of the 1650 measures) as test set and use the remaining pieces as training-and-validation set. This test set intentionally includes pieces for which Antoine et al.'s model both performed best and worst.<fnref n="3" /> We train and validate the models on the training-and-validation dataset using a leave-one-piece-out approach, i.e. we set aside one piece for validation while training on all remaining pieces, we iterate this process for every piece in the train-and-validation set, and we average the results obtained in each iteration.

The entire cross-validation scheme for all variations of the models took about five hours on a laptop equipped with an Intel Core i7-9750H CPU. All calculations were performed in Python using the scikit-learn library.

<footnote n="3">³ The excerpts held out as test set are Borodine's *In the Steppes of Central Asia* (measures 40-71), the fourth movement of Schubert's *Symphony 9* (meas. 543-564), and the second movement of Vaughan Williams's *Symphony 8* (meas. 71-107).</footnote>
<figure><img src="image_6.png" /><figcaption>Figure 4. In the blending graph, two instruments are connected by an edge when the couple is predicted to be blending in the given measure. Here we display one possible result for measure 15 of Mozart's *Don Giovanni Ouverture* (Fig. 1) that reconstructs three groups. The leftmost group contains both Flute 1 and Oboe 2, because there exist a path between them. Three instruments (Violins 1, Violins 2, and Timpani) are excluded from the retrieved blends.</figcaption></figure>

## 4. EVALUATION METRICS

We evaluate the results using a measure-as-a-unit approximation, which considers every measure of a given blend as an independent entity. We here present three evaluation metrics: *pairwise evaluation*, on the direct results of the binary classification, *groupwise evaluation*, on the ability of the model to correctly retrieve the groups of instruments, and *partition evaluation*, on the ability of the model to correctly partition the ensemble into blending groups.

### 4.1 Pairwise Evaluation

Considering the classification problem of pairs of instruments in each measure, we compute the usual classification metrics (accuracy, precision, recall, and f1 score).

### 4.2 Groupwise Evaluation

A second evaluation strategy is the one utilized by Antoine et al. [?], which considers the resulting blends as groups of instruments. For every blend in the target annotations and for every measure of its temporal extension, they evaluate the performance of the retrieval by computing the ratio between the number of correctly identified blending instruments, and the number of instruments in the target blend. They then average for all considered blends. We call this measure groupwise recall, and we extend the procedure by computing also the groupwise precision and f1 score.

The groupwise evaluation starts by building *groups of instruments* from the pairwise classification. Such groups are constructed as the maximal components of a graph, i.e. the maximal connected subgraphs, by a depth-first search algorithm [?] (Fig. 4).

Matching the predicted instrument groups to the target blends can be framed as an assignment problem [?]. The objective is to find the optimal matching between sets of target blends and sets of predicted blends such that the sum of the number of common elements in each pair is maximized (see Fig. 5). We solve this optimization problem with a modified Jonker-Volgenant algorithm with no initialization [?], as implemented by the python function `linear_sum_assignment` from `scipy.optimization`.

<figure><img src="image_7.png" /><figcaption>Figure 5. For measure 15 of Mozart's *Don Giovanni Ouverture*, the matching algorithm maps predicted blends (on the right, Fig. 4) to target blends (on the left, Fig. 1) by maximizing the number of common instruments (matrix in the middle). Here the best matching is (A, 1), which gives a groupwise precision of 1, a groupwise recall of 0.5, a groupwise f1 score of 0.667, an extra instrument number and score of 0, and an extra blend number of 2.</figcaption></figure>

**Matching one or several groups?** In the evaluation of their algorithm, Antoine et al. [?] match the predicted groups to the target blends by hand, and whenever their algorithm predicts two separate groups in correspondence of one big target group, they choose to match both predicted groups to the same target group and merge them for the evaluation. One may indeed argue that such a result is close to the target, as the only difference is that the target group has been split in two by the retrieval algorithm.

We decide not to merge the groups that have been output as separate, and to evaluate the algorithm on its capability to partition instruments into the same blending groups as in the ground truth. In the example shown in Fig. 5, allowing both A and B to be matched to the target blend 1 would overlook too lightly the separation the model made. If the number of target and predicted blends within a measure is the same, the optimization problem solver outputs a one-to-one matching, otherwise there can be an excess of predicted groups or an excess of target groups. We count the number of extra predicted blends and report it with the other groupwise metrics, and we consider excess target groups as not retrieved.

**Extra Instruments.** We also report an *extra instrument* score, which is the proportion of blends (with measure as unit) for which a predicted blend contains instruments that are not part of the true blend and the *extra instrument* number, which is the average number of extra instruments retrieved on all the blends.
## 4.3 Partition Evaluation (Purity)

A third point of view, which allows the evaluation for all blends within a measure to be condensed into one number, consists in looking at the results as a partition of the instruments in the given measure. Often used for the supervised evaluation of clustering algorithms, the purity measures the extent to which predicted clusters contain elements from a single target class:

$$pur = \frac{1}{N} \sum_{j \in \{1,...,\nu\}} \max_{i \in \{1,...,n\}} |P_j \cap T_i|,$$

where {$T_1, ..., T_n$} is the target partition, {$P_1, ..., P_\nu$} the partition predicted by the model, and N the number of instruments. The purity will tend to be higher if each predicted cluster contains mostly points belonging to the same target subset [?].

Some known problems of this index are its sensitivity to imbalanced data, and to a high number of clusters. Therefore we compare the partitions obtained by considering all blends as distinct clusters and by treating all the remaining out-of-blend instruments as one additional cluster.

# 5. RESULTS

Using the pairwise, groupwise, and partition evaluation metrics, Table 1 compares a selection of models, including three baselines that output respectively always 0, always 1, or a random prediction, and to the state-of-the-art model antoine. Our best performing model uses logistic regression with a selection of features (see Fig. 6) that excludes the spectral descriptors and the instrument names (log_regr_selection_nospectral_noins_names). It achieves a better performance than the state-of-the-art model antoine [?].

Something unexpected that we noticed is that we can train a very minimal decision tree model with depth 1 and with only the note_countDifference feature and obtain a better performance than the state-of-the-art model (based on synchrony, harmonicity, and parallelism) on most groupwise and partition metrics. This model (tree_1 note_countDifference) selects every pair of instruments that have the same (non-zero) number of notes as a blend, and represents an additional baseline.

**Cross-validation – Pairwise metrics.** Accuracy is very high, also when considering the baseline models. This happens because the proportion of blending and non-blending pairs of instruments 0, a given measure is very unbalanced (93.0% of the samples correspond to non-blends in the training and validation set). The baseline0 model that always outputs 0 is therefore correct for 93.0% of the samples. Moreover 82.3% of the samples are automatically classified as non blending for all models because of silent instruments. This fixes to 82.3% a lower bound for the accuracy. A high precision here corresponds to a low number of non-blending pairs misclassified as blending; conversely, a high recall means that most of the blending pairs are correctly classified. It is not

surprising therefore that baseline0 gives the highest recall. The f1 score is a balanced measure between the two, and log_regr_selection_no Spectral_no ins names and antoine perform similarly, the first having higher precision and the second one higher recall.

**Cross-validation – Groupwise and partition metrics.** Our model outperforms antoine in groupwise precision, recall and f1 score but tends to retrieve a higher number of extra blends (ebn). A high groupwise precision means that the predicted blend contains mostly instruments from the target group; conversely a high groupwise recall means that most of the instruments in the target blend are present in the predicted group, the groupwise f1 score is a balanced measure between the two. These groupwise metrics are closer to the musical meaning of a blend as a group of instruments than the pairwise metrics and therefore assume more importance in our analysis. One should not be surprised by the fact that the groupwise recall we report for the antoine model is lower than the performance score reported in the original model paper, as we have used a stricter criterion to match predicted groups to target groups (see Sec. 4.2).

The partition purity metric yields similar results with our model slightly exceeding antoine. A high purity score indicates that the target and predicted partitions closely match, differing only by a few elements.

**Test dataset.** Finally, Table 1 also reports the metrics obtained on the test dataset, which have been calculated by retraining the models on the entire training and validation dataset and by making inferences on the test dataset. We observe a similar drop in performance for both our logistic regression model and the state-of-the-art model. Since antoine is not trained, this suggests that the lower performance might be due to the difficulty of these examples rather than to overfitting. The tree_1 note_countDifference model generalises better to unseen data and maintains higher levels of performance on the test set than the other two.

**Feature contribution and qualitative analysis.** Figure 6 presents a SHAP analysis of feature impact on our best model's decisions, along with the values of the trained coefficients. A small set of core features – listed in Fig. 6 and related to the similarity in how parts are written – plays a more crucial role in blend detection than additional information from instrument names and spectral descriptors, even though we still observe a correlation between the spectral features and the target (Fig. 3). The spectral features probably come into play in special cases and in ways that are difficult for a linear model to capture. Including them in the log_regr_all model actually degrades the groupwise precision, probably also because of the collinearities they might introduce. Notice also that the parallelism score is more effective than the more complex cosine contour difference.

A preliminary qualitative analysis of the errors suggests that the model struggles more in detecting blends when
<table><caption>Table 1. Pairwise, groupwise, and partition metrics on models detecting orchestral blends. The abbreviations eis/ein/ebn stand for extra instrument score, extra instrument number, and extra blends number. The best model `log_regr_selection_no_spectral_no_ins_names` has the highest groupwise precision and f1 score, the second highest purity, a high groupwise recall, and low levels of eis, ein, and ebn. Notice that the `baseline0` model has very high accuracy and gives the best results in eis, ein and ebn (0.000) but does not predict any blend.</caption><thead><tr><th rowspan="2">model name</th><th colspan="4">Pairwise</th><th colspan="4">Groupwise</th><th rowspan="2">Partition<br>pur ↑</th></tr><tr><th>acc ↑</th><th>prec ↑</th><th>rec ↑</th><th>f1 ↑</th><th>prec ↑</th><th>rec ↑</th><th>f1 ↑</th><th>eis ↓</th><th>ein ↓</th><th>ebn ↓</th></tr></thead><tbody><tr><td colspan="12" style="text-align: center;">Cross-Validation - Leave One Piece Out (Training and Validation Dataset) - The best three values are in bold</td></tr><tr><td>baseline0</td><td>0.930</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td><strong>0.000</strong></td><td><strong>0.000</strong></td><td><strong>0.000</strong></td><td>0.871</td></tr><tr><td>baseline1</td><td>0.868</td><td>0.323</td><td><strong>0.822</strong></td><td>0.464</td><td>0.553</td><td><strong>0.808</strong></td><td>0.622</td><td>0.677</td><td>3.798</td><td><strong>0.000</strong></td><td>0.926</td></tr><tr><td>baseline_random</td><td>0.898</td><td>0.319</td><td>0.407</td><td>0.358</td><td>0.537</td><td><strong>0.781</strong></td><td>0.602</td><td>0.671</td><td>3.766</td><td><strong>0.005</strong></td><td>0.925</td></tr><tr><td><strong>tree_1 note_countDifference</strong></td><td>0.922</td><td>0.450</td><td>0.537</td><td>0.490</td><td>0.845</td><td>0.722</td><td>0.747</td><td>0.291</td><td><strong>0.782</strong></td><td>1.086</td><td><strong>0.962</strong></td></tr><tr><td>tree_1_all</td><td>0.927</td><td>0.482</td><td>0.584</td><td><strong>0.528</strong></td><td><strong>0.859</strong></td><td>0.748</td><td><strong>0.770</strong></td><td>0.270</td><td>0.819</td><td>0.980</td><td><strong>0.963</strong></td></tr><tr><td>tree_2_all</td><td>0.925</td><td>0.468</td><td><strong>0.595</strong></td><td>0.524</td><td>0.820</td><td>0.769</td><td>0.757</td><td>0.335</td><td>1.308</td><td>0.832</td><td>0.958</td></tr><tr><td>tree_1_selection_alternative</td><td>0.927</td><td>0.482</td><td>0.584</td><td><strong>0.528</strong></td><td><strong>0.859</strong></td><td>0.748</td><td><strong>0.770</strong></td><td><strong>0.270</strong></td><td><strong>0.819</strong></td><td>0.980</td><td><strong>0.963</strong></td></tr><tr><td>tree_2_selection_alternative</td><td>0.924</td><td>0.468</td><td><strong>0.611</strong></td><td><strong>0.530</strong></td><td>0.812</td><td><strong>0.776</strong></td><td>0.756</td><td>0.350</td><td>1.382</td><td>0.773</td><td>0.958</td></tr><tr><td>log_regr_all</td><td><strong>0.931</strong></td><td><strong>0.504</strong></td><td>0.542</td><td>0.522</td><td>0.739</td><td>0.757</td><td>0.709</td><td>0.416</td><td>1.886</td><td>0.464</td><td>0.948</td></tr><tr><td>log_regr_selection</td><td><strong>0.931</strong></td><td><strong>0.505</strong></td><td>0.534</td><td>0.519</td><td>0.754</td><td>0.767</td><td>0.722</td><td>0.402</td><td>1.819</td><td>0.490</td><td>0.948</td></tr><tr><td>log_regr_selection_no_spectral</td><td><strong>0.931</strong></td><td><strong>0.508</strong></td><td>0.534</td><td>0.521</td><td>0.765</td><td>0.764</td><td>0.724</td><td>0.388</td><td>1.758</td><td>0.564</td><td>0.950</td></tr><tr><td>log_regr_selection_no_spectral_no_ins_names</td><td>0.929</td><td>0.494</td><td>0.538</td><td>0.515</td><td><strong>0.865</strong></td><td>0.752</td><td><strong>0.776</strong></td><td><strong>0.257</strong></td><td>0.860</td><td>0.986</td><td><strong>0.962</strong></td></tr><tr><td>antoine [?]</td><td>0.926</td><td>0.476</td><td>0.556</td><td>0.513</td><td>0.795</td><td>0.697</td><td>0.713</td><td>0.303</td><td>0.930</td><td>0.741</td><td>0.953</td></tr><tr><td colspan="12" style="text-align: center;">Test Dataset - The best value is in bold</td></tr><tr><td>baseline0</td><td><strong>0.989</strong></td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td><strong>0.000</strong></td><td><strong>0.000</strong></td><td><strong>0.000</strong></td><td>0.907</td></tr><tr><td>baseline1</td><td>0.902</td><td>0.083</td><td><strong>0.788</strong></td><td>0.150</td><td>0.270</td><td><strong>0.752</strong></td><td>0.386</td><td>0.957</td><td>5.333</td><td>0.000</td><td>0.914</td></tr><tr><td>baseline_random</td><td>0.944</td><td>0.086</td><td>0.420</td><td>0.142</td><td>0.273</td><td><strong>0.752</strong></td><td>0.389</td><td>0.957</td><td>5.261</td><td>0.000</td><td>0.914</td></tr><tr><td>tree_1 note_countDifference</td><td>0.959</td><td>0.162</td><td>0.645</td><td>0.259</td><td><strong>0.501</strong></td><td>0.741</td><td><strong>0.561</strong></td><td>0.812</td><td>2.478</td><td>0.606</td><td><strong>0.952</strong></td></tr><tr><td>log_regr_selection_no_spectral_no_ins_names</td><td>0.957</td><td>0.168</td><td>0.732</td><td><strong>0.273</strong></td><td>0.284</td><td>0.570</td><td>0.355</td><td>0.870</td><td>4.304</td><td>0.197</td><td>0.929</td></tr><tr><td>antoine [?]</td><td>0.962</td><td><strong>0.172</strong></td><td>0.649</td><td>0.271</td><td>0.282</td><td>0.435</td><td>0.331</td><td>0.696</td><td>2.072</td><td>0.318</td><td>0.931</td></tr></tbody></table>

<figure><img src="image_8.png" /><figcaption>Figure 6. Median pitch difference, pitch range difference and synchrony have the most impact on the `log_regr_selection_no_spectral_no_ins_names` model decision according to the SHAP analysis (top). The first two decrease the probability of blending while the third one increases it (bottom).</figcaption></figure>

Long notes are present. The predictions appear to be synchrony-driven (in the example from Fig. 1, the model fails to connect Violin 1 to the other parts). The model performs best when the blends contain many short synchronous notes but it is prone to detecting false positive blends that contain synchronous instruments.

**Code and data availability.** The code to predict blends, the cross-validation strategy and the implementation of the evaluation measures are available under open-source license at https://algomus.fr/code. Annotation data and pre-computed features for each measure of the corpus are also available under open-source license from https://algomus.fr/data.

## 6. DISCUSSION AND CONCLUSIONS

We proposed a machine learning model with knowledge-driven features to identify orchestral blends in scores. It improves the state-of-the-art dedicated algorithm on several metrics. Evaluating blends with pairwise, groupwise, and partition metrics gives a clearer picture on the way the models make their decisions.

This work gives some insights on the musical elements that contribute to blends. It appears that six core features (see Fig. 6) that describe how similarly the parts are written are more important than the features describing the spectral characteristics of the instruments that are playing them, despite the correlation we observe between the spectral features and the target.

However, we should consider any speculation making parallels between the model and the functioning of human perception with special care. The statistical models we used perform inference based on correlations and this is an imprecise approximation of the highly non-linear functioning mechanisms of the human ear and brain. This is particularly evident when we look at the `tree_1 note_countDifference` model that obtains very good predictions by counting the notes, which is probably very far from what happens in human perception – but is here a decent heuristics in estimating how parts are similar.

In future work we could refine the computation of the features and further explore the role of timbre in the perception of blends. This includes investigating methods to capture non-linear relationships between the variables and the target, the use of tagging, graph-based models, and feature transformations. Spectral features could probably be improved by considering more realistic measure fingerprints averaging spectral descriptors for all present pitches rather than focusing on the average pitch, also differentiating the articulations. Models learning the features automatically
from the sequence of notes could also be investigated, potentially revealing features correlated to blending that have not yet been considered in music perception. More precise investigations on the type of errors that the model makes should also be carried out by comparing the results with the ground truth in the scores.

7. REFERENCES

[1] R. A. Kendall and E. C. Carterette, “Identification and blend of timbres as a basis for orchestration,” *Contemporary Music Review*, vol. 9, no. 1-2, pp. 51–67, 1993.

[2] A. Bernier-Robert and B. Duinker, “Blend,” Timbre Lingo - Timbre and Orchestration Writings, 2023.

[3] W. Piston, *Orchestration*. Norton, 1955.

[4] S. McAdams and A. Bregman, “Hearing musical streams,” *Computer Music J.*, vol. 3, pp. 26–60, 1979.

[5] G. J. Sandell, “Roles for spectral centroid and other factors in determining “blended” instrument pairings in orchestration,” *Music Perception*, vol. 13, no. 2, pp. 209–246, 1995.

[6] A. Schnittke, “Timbral relationships and their functional use,” *Orchestration: An anthology of writings*, pp. 162–175, 2006.

[7] S. McAdams, M. Goodchild, and K. Soden, “A taxonomy of orchestral grouping effects derived from principles of auditory perception,” *Music Theory Online*, vol. 28, no. 3, 2022.

[8] A. S. Bregman, *Auditory Scene Analysis: The Perceptual Organization of Sound*. The MIT Press, 1990.

[9] D. Deutsch, “Grouping mechanisms in music,” *The psychology of music*, vol. 28, pp. 299–348, 1999.

[10] A. Antoine, P. Depalle, P. Macnab-Séguin, and S. McAdams, “Modeling human experts’ identification of orchestral blends using symbolic information,” in *Perception, Representations, Image, Sound, Music*, 2021, pp. 363–378.

[11] E. Cambouropoulos, “Voice and stream: Perceptual and computational modeling of voice separation,” *Music Perception*, vol. 26, no. 1, pp. 75–94, 2008.

[12] S.-A. Lembke and S. McAdams, “The role of spectral-envelope characteristics in perceptual blending of wind-instrument sounds,” *Acta Acustica united with Acustica*, vol. 101, no. 5, pp. 1039–1051, 2015.

[13] S.-A. Lembke, S. Levine, and S. McAdams, “Blending Between Bassoon and Horn Players: An Analysis of Timbral Adjustments During Musical Performance,” *Music Perception*, vol. 35, no. 2, pp. 144–164, 2017.

[14] M. de Francisco, M. Kob, J. Kelly, D. Quiroz, and S. Ioannou, “Odessa: An interdisciplinary symphonic recording for the study of orchestral sound blending,” in *Audio Engineering Society Convention 149*, 2020.

[15] J. Thilakan, B. B T, O. Colella Gomes, J.-M. Chen, and M. Kob, “Exploring the role of room acoustic environments in the perception of musical blending,” *The Journal of the Acoustical Society of America*, vol. 157, no. 2, pp. 738–754, 02 2025.

[16] D.-V.-T. Le, M. Giraud, F. Levé, and F. Maccarini, “A corpus describing orchestral texture in first movements of classical and early-romantic symphonies,” in *Digital Lib. for Musicology (DLfM 2022)*, 2022, pp. 22–35.

[17] F. Maccarini, M. Oudin, M. Giraud, and F. Levé, “Co-creative orchestration of Angeles with layer scores and orchestration plans,” in *Artificial Intelligence in Music, Sound, Art and Design (EvoMUSART 2024)*, 2024, pp. 228–245.

[18] S. Kazazis, P. Depalle, and S. McAdams, “Ordinal scaling of timbre-related spectral audio descriptors,” *The J. of the Acoustical Society of America*, vol. 149, no. 6, pp. 3785–3796, 06 2021.

[19] G. Peeters, B. L. Giordano, P. Susini, N. Misdariis, and S. McAdams, “The Timbre Toolbox: Extracting audio descriptors from musical signals,” *The J. of the Acoustical Society of America*, vol. 130, no. 5, pp. 2902–2916, 11 2011.

[20] S. Kazazis, P. Depalle, and S. McAdams, “The Timbre Toolbox version R2021a, User’s manual,” 2021.

[21] M. S. Cuthbert and C. Ariza, “music21: A toolkit for computer-aided musicology and symbolic music data,” in *Int. Society for Music Information Retrieval Conf. (ISMIR 2010)*, 2010, pp. 637–642.

[22] N. McLachlan, D. Marco, M. Light, and S. Wilson, “Consonance and pitch,” *J. of Experimental Psychology: General*, 2013.

[23] B. Benward and M. Saker, *Music in Theory and Practice (Vol. 1)*. McGraw-Hill, 2008 (8th ed.).

[24] N. Cazden, “The definition of consonance and dissonance,” *Int. Review of the Aesthetics and Sociology of Music*, vol. 11, no. 2, pp. 123–168, 1980.

[25] B. Cornelissen, W. Zuidema, and J. A. Burgoyne, “Cosine contours: a multipurpose representation for melodies,” in *Int. Society for Music Information Retrieval Conf. (ISMIR 2021)*, 2021, pp. 135–142.

[26] S. Even, *Graph Algorithms*. Cambridge U. Press, 2011.

[27] H. W. Kuhn, “The Hungarian method for the assignment problem,” *Naval Research Logistics Quarterly*, vol. 2, no. 1-2, pp. 83–97, 1955.

[28] D. F. Crouse, “On implementing 2D rectangular assignment algorithms,” *IEEE Trans. on Aerospace and Electronic Systems*, vol. 52, pp. 1679–1696, 2016.

[29] C. D. Manning, P. Raghavan, and H. Schütze, *Introduction to information retrieval*. Cambridge U. Press, 2008.