Analyzing Byte-Pair Encoding on Monophonic and Polyphonic Symbolic Music: A Focus on Musical Phrase Segmentation (2024)

Abstract: Byte-Pair Encoding (BPE) is an algorithm commonly used in Natural Language Processing to build a vocabulary of subwords, which has been recently applied to symbolic music. Given that symbolic music can differ significantly from text, particularly with polyphony, we investigate how BPE behaves with different types of musical content. This study provides a qualitative analysis of BPE’s behavior across various instrumentations and evaluates its impact on a musical phrase segmentation task for both monophonic and polyphonic music. Our findings show that the BPE training process is highly dependent on the instrumentation and that BPE “supertokens” succeed in capturing abstract musical content. In a musical phrase segmentation task, BPE notably improves performance in a polyphonic setting, but enhances performance in monophonic tunes only within a specific range of BPE merges.

## introduction a major similarity between text and music lies in their nature as semiotic systems, as they can be represented as sequences of elements @xcite . this common characteristic has led to numerous adaptations of natural language processing (nlp) methods in the domain of symbolic music analysis and generation @xcite . mirroring the view of a text as a sequence of tokens representing words, subwords or characters, tokenization practices have also been adopted to process symbolic music. several choices of types of musical "characters" and various tokenization algorithms to segment the sequence of musical "characters" have been proposed @xcite . however, music profoundly differs from text, notably because of some structural characteristics such as rhythm or polyphony @xcite . we can, thus expect tokenization algorithms such as byte-pair encoding (bpe) @xcite to behave differently when applied to text or music. the aim of this study is to highlight some commonalities and differences in bpe behaviors with multiple types of music as compared to text. this work is twofold: we first propose a statistical description of the vocabulary of tokens obtained when bpe is applied to text compared to the vocabularies obtained with various types of music. this comparison highlights some musical properties captured by this tokenization algorithm (section 3). informed by these observations, we then focus on a downstream task, musical phrase segmentation, to quantitatively compare the impact of bpe on monophonic and polyphonic music (section 4). ## subword tokenization in symbolic music subword tokenization, where tokens are subwords instead of characters or words, is a common practice in nlp. it is used to deal with out-ofvocabulary words that are obtained by combining multiple subwords. multiple algorithms have been proposed to build from a corpus the most representative vocabulary of subwords, including byte-pair encoding (bpe) @xcite , wordpiece @xcite or unigram @xcite . bpe was initially developed as a compression algorithm @xcite before being applied to text as a tokenization method. the algorithm relies on creating new subword tokens by iteratively merging the most recurring pairs of successive tokens in a corpus until a chosen vocabulary size is reached. in the following, we call atomic elements the tokens from the initial vocabulary and supertokens the tokens added through bpe. some recent mir studies have applied these algorithms to symbolic music @xcite . bpe was first implemented to shorten token sequences @xcite . @xcite specifically analyzed bpe for midi generation purposes and showed that the learned embedding spaces are more structured. however, when applied to piano analysis tasks, a bpe with 4 times the initial vocabulary size does not seem to show any downstream improvement in model performance @xcite . in contrast, @xcite focus on specifically applying bpe on monophonic tunes using a pitch/duration-only representation and show that bpe enables the retrieval of style-specific motifs. to date, research on bpe for symbolic music has focused on its evaluation on generation or global sequence classification tasks. its behaviour has not been analyzed in depth, in particular when applied to various instrumentations. this work specifically focuses on these issues, with a descriptive analysis of bpe vocabularies followed by a quantitative evaluation of bpe on monophonic and polyphonic music on a musical phrase segmentation task. our experiments rely on the miditok package @xcite to handle the tokenization process and the huggingface library @xcite implementing transformer models. we publicly release the datasets and source code, which are available at http://algomus.fr/code/ . ## analyzing music bpe in this section, we present analyses of the vocabulary produced by byte-pair encoding when applied to text and music. we first analyse supertokens induced by various instrumentations as well as their relation to high-level or abstract musical features. ## evaluating bpe on musical phrase segmentation bpe applied to midi-derived tokenization has been mainly evaluated through classification tasks with composer classification, on a general multitrack dataset @xcite or specifically piano music @xcite . inspired by sentence segmentation tasks in nlp @xcite and given our preliminary results showing that supertokens can play a role in musical phrase boundaries, we aim to quantitatively evaluate bpe on a task of musical phrase segmentation for monophonic and polyphonic datasets. ## conclusion in this work, we show that byte-pair encoding behaves differently depending on the type of music it is trained on. with a descriptive approach, we highlight that the resulting vocabulary highly depends on the type of instrumentation, and supertokens can carry high-level musical content. on a downstream task, we confirm the impact of instrumentation on the model performance and show that the number of bpe merges should be chosen carefully. for future work, we think the initial tokenization impact over bpe performance should be investigated.