# A New Method for Detecting Onset and Offset for Singing in Real-Time and Offline Environments
## 1 Introduction
The training and test set come from a tiny dataset comprising 18 singing recordings from four singers with 1127 onsets. In the study conducted by [PERSON] and [PERSON] [17], a deep learning model was trained for musical onset detection in solo singing, and the authors discussed how their algorithm could lead to improve live onset detection models. They used two datasets, one of which contains more than 25,000 onsets, mostly complex mixtures or solo instrumental excerpts, and only three excerpts are of a solo singing voice, and the other dataset is a subset of a solo [PERSON] singing voice that contains 100 recordings. They employed seven deep learning-based architectures. In the [PERSON] and [PERSON] [17] study, it was preferred to use the score-informed method if the musical score information was available. Score-informed approaches evaluate the data with the assistance of musical scores. Based on the results, score-informed HMM outperformed peak picking for all of the architectures used in this experiment [17]. The reported F-measure for the combination of the peak picking method and a no-dense neural network architecture was 73.88%, with a \(p\)-value of 0.002. For the score-informed HMM method, a nine-layer CNN architecture worked best, giving an F-measure of 80.90% and a \(p\)-value of 0.001. Learning strategies for inter-dataset knowledge transfer were also studied, but due to different musical patterns, the authors claimed that when the musical patterns from the two datasets used to train their model were different, the onset prediction was not accurate. Despite these studies, onset detection of a musical note remains a challenge, primarily for the singing voice. [PERSON] and [PERSON] [18] explain several reasons for this, including inconsistency of articulation, singer-dependent tonal quality, and gradual variation in onset envelopes over time. In other words, the time-varying spectral envelope and the inconsistency of vocal tracks may produce fake maxima (i.e., peaks) in an onset detection function that can lower the precision rate for onset detection. Therefore, detecting onsets from the singing voice is still an active area of study because of waveform unpredictability and the occurrence of many noisy segments. Moreover, most methods are only suitable for recorded singing and are designed to work offline. According to the previously published results, most existing approaches do not work well for soft onsets, including singing music. A soft onset has a long attack duration or vague envelope shape that becomes a challenge to the peak-picking procedure. The underlying reason for these issues is that the singing voice is classified as a pitched non-percussive (PNP) instrument, and PNP instruments still present a challenge for onset detection [9]. The nature of the singing voice adds further complexity due to its natural inconsistency with respect to pitch and time dynamics. Unlike some instruments, where their timbre is usually consistent throughout a note, the singing voice inherently can produce more variations of formant structures (for articulation); sometimes, it may even variate within the duration of a single note [19]. While most onset detection algorithms are based on detecting spectral changes, they can fail to differentiate such variations in a singing voice because of singing features such as vibration and soft onset. Relevant challenges for onset detection in solo singing voices were identified in a report from the Music Information Retrieval Evaluation eXchange 2012 (MIREX 2012). According to this report, the best-performing detection method gives an F-measure of only 55.9% [4], which even becomes lower for solo sustained strings with an average F-measure of 52.8%. In addition, training datasets for dynamically changing patterns in a singing voice is still a challenge [17; 20]. One of the missing parts of most of the onset detection algorithms is considering the actual singing style features. In the [PERSON] et al.