dataesr/scientific-paragraphs-categorization huggingface dataset:
In this paper, we post-process our self-supervised BDF with the most recent DBN provided in  [30] , as provided by the Python package "madmom"  [42]. In  [43] , the authors (Davies and Böck) use the same DBN in conjunction with their supervised temporal convolutional neural network (CNN). We consider their method as the current state of the art in supervised beat tracking. Note that updated beat tracking results can be found in  [44]  and involve joint beat and downbeat estimation using a transformer architecture.
A. Automatic Curation As noted in Section I-C, we train ZeroNS by predicting the synchrony between percussive and non-percussive parts in the same song. To this end, we need to curate a large-scale training set in which every song contains both percussive instruments and non-percussive instruments. Yet, in a digital music archive, some songs may contain no percussive parts (e. g. , a cappella choir) while others may contain only percussive parts (e. g. , drum ensemble). Since sync-pred would become ill-posed on those songs, we exclude them from our training set beforehand. In practice, we use music data from the Free Music Archive. The FMA is one of the largest audio catalogs of royalty-free music, and was dumped to an immutable web repository in 2017  [56]. This dump, known as "FMA dataset", has frequently been used in MIR research for audio tagging; e.
g. ,  [57]. Yet, our publication is the first to use the dataset for beat tracking. We download the "large" version of the FMA dataset  2. This version contains 107k song excerpts, each of them lasting 30 seconds, and spans a 161, mostly Western, genre hierarchy. In order to verify the presence of percussive and nonpercussive sources, we process each of these song excerpts with Spleeter 3. Spleeter is a Python library for audio source separation, involving several pre-trained deep learning models. Spleeter is a supervised model because it was trained with separated sources as a ground truth. However, this ground truth is not equivalent to a human annotation of musical beats. Thus, we may still regard our pipeline as unsupervised with respect to the beat tracking task. Among the models that are present in Spleeter, we select 4stems: i.
25. In practice, we train the ZeroNS model f in PyTorch 4  with the Adam optimizer  [66]  and a learning rate of 10 -6. We split our unlabeled dataset (see Section III-A) into a training subset of size 28,800 and a validation subset of size 6,400. We monitor the validation loss and stop training once it reaches a minimum; i. e. , after 35 epochs.
E. Downstream Task: Beat Tracking We evaluate a total of seven systems for the beat tracking task. The first four of these are unsupervised, or self-supervised, and correspond to: i Random+DBN: a randomly initialized fully convolutional network of Section III-C with the unsupervised DBN of  [30]  (see Section II-B); ii Spectral Flux+DP: feature engineering (spectral flux) and dynamic programming  [29]  (see Section II-B); iii ZeroNS+DP: fully convolutional network of Section III-C trained with sync-pred on percussive and non-percussive parts of the FMA dataset (see Section III-D), followed by maximum pooling between the resulting embeddings (see below) and dynamic programming; iv ZeroNS+DBN: same as above, dynamic programming replaced by the unsupervised DBN; The results of the above can be found in Section IV-C.
The next three rely on human supervision, in part or in full. Their results can be found in Section IV-D; we outline the beat tracking training procedure below: v Random+DBN: same as (i), trained on each beat tracking dataset; vi ZeroNS+DBN: same as (iv), fine-tuned on each beat tracking dataset; vii TCN+DBN: temporal convolutional network with supervised training on multiple datasets, followed by the same DBN used in our experiments  [43]. In systems (iii), (iv), and (vi), we merge the two branches of ZeroNS via maximum pooling between unnormalized percussive (f p ) and non-percussive embeddings (f ¬p ): y(t) = max f p (Uϕ p x)(t), f ¬p (Uϕ ¬p x)(t). (9) This allows our system to perform beat tracking when either the percussive or non-percussive sources are silent; i. e. when x contains no drums (ϕ p x = 0) or nothing but drums (ϕ ¬p x = 0).
Conversely, in (i) and (v), the model comprises a single branch f which operates on the VQT of the mixture Ux; thus, no merging is necessary. In (iii), we pass y as observation to the dynamic program of  [29]  and retain its peaks as predicted beats. In (iv), we do the same with the DBN of  [30]. We set its BPM range to  [55, 215]. Both of these approaches (ZeroNS+DP and ZeroNS+DBN) combine a supervised source separation model (Spleeter), a self-supervised acoustic model (ZeroNS), and an unsupervised sequence model (DBN or DP): hence, they may be regarded as unsupervised with respect to the beat tracking task. In (v) and (vi), we convert the human annotation into a signal y * with the same hop length as y. Following  [43] , the value of y * (t) equals 1 if t is nearest to an annotated beat, 0.
A. Convergence on the Pretext Task The ZeroNS neural network is able to train appropriately on the curated dataset. During the first few iterations on the training set, the cosine similarities between all percussive and non-percussive embeddings are very close to one, suggesting that the networks are unable to recognize synchronized and non-synchronized outputs. After 25 epochs, the ZeroNS model reaches a loss value of 0. 634±0. 30. Synchronized embeddings in the validation set have a mean cosine similarity value of 0. 804 ± 0. 13 while non-synchronized embeddings have a mean cosine similarity value of 0. 053 ± 0. 01. These results show that both our data curation process and pretext task definition enable the model to converge appropriately. This probably would not have been the case if we had not filtered out songs with very few drums, for example.
A comprehensive overview of each measure can be found in  [68]. We also report the sample entropy of our embeddings. Originally proposed for physiological time-series  [69] , this measure estimates the amount of regularity present in a signal. Finally, we compute the maximum autocorrelation value obtained for shifts of 0. 25 to 4 seconds. Each embedding mean is subtracted before measuring the above. All values are also divided by the unlagged autocorrelation value. We compute each measure for five embeddings: a randomly initialized CNN; the percussive branch f p of ZeroNS; the non-percussive branch f ¬p of ZeroNS; the maximum of the two branches, f (see Equation  9 ); and a supervised CNN trained on the Ballroom dataset (see Section IV-D). All measures are computed on the GTZAN dataset. Figure  2  displays our results for each network.
C. Unsupervised Representations We evaluated our models on four annotated datasets: the Ballroom dataset 5  from  [70] ,  [71]  and annotated using  [41]   6  , the Hainsworth dataset  [72] , the GTZAN dataset from  [73]  and annotated in  [74] , and the SMC dataset  [75]. Unlike in  [43] ,  [12] ,  [76] ,  [39]   7  , we report results on each dataset separately. This was done in order to better study the effects of pre-training on small datasets. Each is annotated differently and spans a variety of musical genres. Table  I  further describes the datasets we used according to the number of files they contain and the duration of these files. We adopt the following evaluation metrics: F1-score, AMLc, AMLt, CMLc, and CMLt. The CMLc and CMLt metrics evaluate how continuously correct a beat tracking estimation is (use of the maximum length of correct predictions).
The AMLt and AMLc metrics are similar but allow offbeat variations of an annotated beat sequence to be matched with detected beats. One can read more about each metric in  [77]. Let us now explore the effects of pre-training on downstream task performance. In our first evaluation setting, we test our models in an unsupervised fashion. Our results can be found in Figure  3. When looking at the figure, one notices that the randomly initialized network is greatly outperformed by the Spectral Flux+DP, ZeroNS+DP, and ZeroNS+DBN methods. The randomly-initialized network does not surpass a mean CMLc score of 0. 01 or an F1-score of 0. 37 on each dataset. This was to be expected: it is neither trained or optimized to learn a form of musical meter, tempo, or synchrony to a pulse. In contrast, both ZeroNS methods perform just as well, if not better, than the method in  [29].
When used in conjunction with the latter's dynamic program, our ZeroNS method outperforms the Spectral Flux+DP method on 13/15 metrics for the SMC Fig. 3 : Benchmark of four different unsupervised models on the beat tracking task: our randomly initialized fully-convolutional network followed by the DBN in  [30] ; Librosa's spectral flux and dynamic programming (DP) algorithm  [29] ; ZeroNS followed by DP; and ZeroNS followed by the DBN. Each row corresponds to a different dataset (indicated on the right) while each column corresponds to a different evaluation metric (at the top). Reported figures result from the mean scores for each track. Mirex, Hainsworth, and Ballroom datasets. It is however consistently outperformed by the Spectral Flux+DP method on the GTZAN dataset. We leave as future work to determine where the ZeroNS method fails compared to it Spectral Flux counterpart.
These results do suggest that the onset functions output by the ZeroNS network are at the very least on par with the ones generated by Spectral Flux  [29]  for beat tracking. When used in conjunction with the SOTA DBN from  [30] , our ZeroNS model outperforms Spectral Flux+DP on all datasets for every single metric. When using an F1-score metric, the ZeroNS+DBN method beats the Spectral Flux+DP method by at least four percentage points. The method also achieves 70% (SMC Mirex), 80. 3%  (Hainsworth) , 82. 5% (Ballroom), and 92. 1% (GTZAN) of the performance of its fully-supervised counterpart (Random+DBN in next section). When using an AMLc metric, the ZeroNS+DBN method beats the Spectral Flux+DP method by at least eight percentage points on every datset. The method also achieves 56. 7%, 76.
D. Supervised Learning For the supervised models, we split our datasets into eight randomly selected folds. Each of these are then used as a test set during cross-validation. During each cross-validation iteration, the remaining folds' data is used for either our training or validation sets. These are selected randomly using a 6  7 and 1 7 split. For each 8-fold cross-validation iteration, training is stopped when the model's F1-score on the validation set has not increased for 20 epochs. The randomly initialized network and the fine-tuned ZeroNS network are all evaluated using the aforementioned methodology. Figure  4  displays our results on each dataset according to the metrics introduced previously. We also report the results from  [43] , obtained using a temporal convolutional network (TCN) and the the DBN from  [30].
Fine-tuning our ZeroNS model on each beat tracking dataset does not yield the same promising results as the unsupervised learning setting. On all datasets and metrics, the ZeroNS+DBN network's performance is roughly on par with the Random+DBN network's performance. On the SMC and GTZAN datasets, for example, the fine-tuned network outperforms the randomly initialized network by a slight margin (maximum ∼ 0. 03) on all metrics. We observe the opposite phenomenon on the Ballroom and Hainsworth datasets. This leads us to the following conclusion: in an 8-fold crossvalidation setting, there is enough annotated training data for the randomly initialized and ZeroNS model to perform similarly. When comparing our models' performances with the TCN+DBN network, we notice that the latter's performance on the SMC, Ballroom, and Hainsworth datasets is generally better for every metric.
This is particularly striking on the smaller Hainsworth dataset, where their mean F1-score is 0. 874 compared to our maximum of 0. 783. On the GTZAN dataset, however, our ZeroNS+DBN network outperforms their method for every single metric except AMLc. Our Random+DBN network outperforms their method for every single metric except AMLt and AMLc. This makes sense: 8-fold crossvalidation is performed on the combined SMC, Ballroom, and Hainsworth datasets in  [43]. The GTZAN dataset, on the other hand, is used solely for testing purposes. We surmise that the TCN+DBN performance would be much closer to ours had they computed their results on each dataset individually. Fig. 4 : Benchmark of three different supervised models on the beat tracking task: our randomly initialized fully-convolutional network followed by the DBN in  [30] ; ZeroNS followed by the DBN; and the SOTA temporal convolutional network of  [43].
E. Low-Data Beat Tracking We also test the effects of pre-training in a low-data learning setup. For each dataset, we extract two constant folds for testing and validation. These each contain an eighth of the total dataset. The remaining data is used to train a randomly initialized model and a ZeroNS model. The DBN from  [30]  is used in conjunction with all models. However, instead of using all the remaining data for training, we use subsets of size 1, 2,  3, 4, 6, 8, 12, 16, 24, 32, 48, 64 , and 96 tracks. For each subset, we train our networks on 10 randomly selected subsets of the training data, so as to maximize the diversity of our low-data learning reference examples. We also stop training once performance on the validation set has not increased for 20 epochs. The model that performs best on that set is selected for testing.
We observe similar behavior on the other datasets with a training set size of 1; the ZeroNS network has a mean F1-score of 0. 604 on the Hainsworth dataset, 0. 729 on the Ballroom dataset, and 0. 820 on the GTZAN dataset versus 0. 783, 0. 911, and 0. 847 for the randomly initialized network in the previous, fully-supervised experiment. As the number of training examples increases, the randomly initialized network's performance does catch up with the ZeroNS network's one. The exact same behavior is observed using other evaluation metrics. On the Hainsworth dataset, the AMLc has a mean value of less than 0. 04 when fewer than 12 examples are used to train a randomly initialized network. In comparison, the ZeroNS network reaches a value of 0. 559 with just one training example. Both networks only start performing similarly when 24 examples are used (0.
659 versus 0. 736); the ZeroNS network still performs more than a standard deviation better in this case. This leads us to conclude the following: our proposed pretext task significantly helps our network on the downstream task under a regime of limited labeled data. Indeed, ZeroNS embeddings already contain relevant information for beat tracking, and few annotated examples are needed for the system to perform well. A similar idea is broached in semi-supervised learning  [78] , where it is theorized that humans learn by being exposed to large amounts of unlabeled data, and a small amount of labeled examples that help guide their learning. Our network is learning in the same way; a large catalog of unlabeled data is used to understand the notion of musical beat, and only a few examples suffice to teach it how to track the beat.
F. Cross-Dataset Generalization Annotating a song's beats is a lengthy and time-consuming process. More importantly, most datasets are annotated using different techniques  [77] , which can lead to a form of bias. In our final evaluation setting, we explore the cross-dataset generalization capabilities of our pre-trained models. We train our randomly initialized and ZeroNS networks on each of the Ballroom, SMC, and Hainsworth datasets using 8-fold crossvalidation. For each fold, the highest performing model is tested on the GTZAN dataset, our largest and most diverse dataset. Figure  6   [43] , where the SMC, Ballroom, and Hainsworth datasets are combined for training and the GTZAN dataset is used for evaluation. In the future, we hope to use our pre-training to improve beat tracking results on musical genres or styles with little to no annotated data, so as to expand the scope of MIR beyond Western music and the U.
This is a simple model-agnostic task, involving a single audio file per batch and no artificial perturbations of the audio data. Thanks to an off-the-shelf source separation system (Spleeter), we have scaled up sync-pred to 98 hours of audio; that is, ten times more data than the largest annotated dataset for beat tracking (GTZAN, 9 hours). Besides, we have argued that sync-pred is interesting because it resembles two musicians grooving to a collective pulse when playing together. We have implemented sync-pred as part of a deep learning model, named "Zero-Note Samba" or ZeroNS. In doing so, we do not claim that the architectural design of ZeroNS is optimal; only that it showcases some interesting properties of sync-pred. First, we observe that sync-pred essentially acts as  Vertical purple lines represent the results of the SOTA temporal convolutional network of  [43] , which is trained on the SMC, Hainsworth, and Ballroom datasets and evaluated on GTZAN.
a "transientness prior" on the learned embeddings. Second, when used in conjunction with the dynamic programming algorithm from  [29]  or the dynamic Bayesian network from  [30] , the learned embeddings match or outperform previous SOTA unsupervised methods. Third, sync-pred offers a form of pre-training for supervised models that allows to drastically reduce the quantity of annotated data; typically from 100 songs down to ten songs or fewer. Fourth, this pre-training mitigates dataset bias and improves beat tracking metrics on a crossdataset evaluation setting by two standard deviations. These findings lead us to think that sync-pred has potential in beat tracking, and perhaps other problems in which multiple sources, agents, or modalities operate in synchrony. Our main negative finding is that, under a k-fold crossvalidation scenario, pre-training ZeroNS with sync-pred does not improve beat tracking performance, or marginally so.
It remains unclear how knowledge may be transferred from pretext to downstream task in a way that favors within-dataset generalization. In a recent report  [79] , we have proposed to do so with coincidence prediction instead of sync-pred, yet with limited success so far. Variations in the formulation of multitrack contrastive learning or in the architecture of ZeroNS might solve this shortcoming in the future. Beyond the task of beat tracking, it would be valuable to inquire how sync-pred could be employed at larger musical time scales: namely, downbeat estimation and structure analysis. To this end, sync-pred may not only be formulated between two audio sources, but also between audio and video, audio and motion, or audio and neurophysiology. Lastly, we note that sync-pred is not limited to the time domain: it could also be formulated in the frequency domain, with downstream tasks such as tonality estimation and chord transcription.
displays our results. We observe that ZeroNS pre-training drastically improves our model's generalization capabilities. The gap in performance is quite stark: when training on the SMC dataset, the Random model's mean F1-score is 0. 638 whereas the ZeroNS model's is 0. 748; when training on the Hainsworth dataset, the Random model's mean AMLt is 0. 834 whereas the ZeroNS model's is 0. 894; when training on the Ballroom dataset, the Random model's mean CMLc is 0. 584 whereas the ZeroNS model's is 0. 656. These results demonstrate that ZeroNS pre-training helps mitigate the bias towards certain genres or annotation strategies. ZeroNS pre-training is however not on par with the training method from.
In this work we present a phase-field based method to accurately predict the nucleation and evolution of damage. The damage pattern is then used as a criterion for automatic, remeshingbased, 3D crack insertion and propagation. The proposed framework has been implemented in Forge® finite element software and it offers a robust numerical tool for the modeling of damage to fracture transition in complex industrial processes. A bar shearing simulation will be used to show the robustness and efficiency of the approach. The flexibility of the approach is presented through the use of different damage criteria introduced into the phase-field formulation.
paho. org/journal | https://doi. org/10. 26633/RPSP. 2023. 63 antimicrobial consumption. At that time, it was agreed that there should be gradual and sustained progress by 2021 in the implementation of the prevention and control program for healthcare-associated infections, in controlling AMR, and in reporting antibiotic consumption in 90% of tertiary and secondary level hospitals 1 (high and medium complexity institutions)  (2, 3). Consequently, the Ministry, with support from the National Institute of Health (INS), implemented surveillance strategies for healthcare-associated infections, antibiotic consumption, and AMR through the National Public Health Surveillance System (Sistema de Vigilancia en Salud Pública-SIVIGILA). The INS has implemented a monitoring program for the consumption of seven antibiotics in secondary and tertiary hospitals at a national level since 2012  (2).
e. , maternity, general inpatient, surgery, and other wards, excluding emergency rooms) separately. These antibiotics were selected by the INS for consumption surveillance because they are the main antibiotics for clinical use in adult hospitalization services of high complexity health institutions  (11). Currently, reporting to SIVIGILA is voluntary for secondary level hospitals and mandatory for tertiary level hospitals. SIVIGILA is a software tool, developed by the INS, that has allowed the implementation of several epidemiological surveillance programs in the country. It is easy to use, and many hospital staff are familiar with it as a result of training efforts by the INS since the system came into operation in 2006. The reporting of antibiotic consumption through this system began in 2013  (2).
Source of data and variables SIVIGILA reports include details of seven antibiotics (cefepime, ceftriaxone, ciprofloxacin, ertapenem, meropenem, piperacillin, and vancomycin) and their consumption in ICU and non-ICU services for adult patient care in secondary and tertiary hospitals across Colombia. Data for this study were obtained from the SIVIGILA-INS database for the years 2018, 2019, and 2020. The study variables for compliance of hospital submissions were year and month of reporting, by type of patient service, and administrative region of the country. In addition, for the consumption trend analysis from 2018 to 2020, DDD per 100 occupied beds reported for each antibiotic were analyzed as a consumption indicator. Due to system drawbacks, it was not possible to retrieve and use the data for non-ICU services in 2018.
Statistical analysis Data were provided in a Microsoft Excel spreadsheet. To assess performance of the system, adherence to reporting and data quality were analyzed as parameters of interest. For the reporting parameter, each tertiary level facility was expected to submit one report per month for both ICU and non-ICU services, and each secondary level facility for non-ICU wards only (i. e. , 12 reports were expected per facility per service per year). The proportion of reports received was summarized as a percentage. If the same facility had submitted more than one report for a service in a month it was considered a duplicate report, and only the last one was included for the calculation of the number of submitted reports. For the data quality parameter, the distribution of reported values of monthly DDD/100 occupied beds was summarized   or excluding an antibiotic due to disuse (imipenem in 2020).