dataesr/scientific-paragraphs-categorization huggingface dataset:
Bayesian Neural Networks In most neural networks only the Θ MAP weights computed during training are kept for predictions. Conversely, in BNNs we aim to find the posterior distribution P (Θ | D) of the parameters given the training dataset, not only the values corresponding to the MAP. Here we can make a prediction y on a new sample x by computing the expectation of the predictions from an infinite ensemble corresponding to different configurations of the weights sampled from the posterior distribution: P (y | x, D) = P (y | x, Θ)P (Θ | D)dΘ, (3) which is also known as Bayes ensemble. The integral in Eq. (3), which is calculated over the domain of Θ, is intractable, and in practice it is approximated by averaging predictions from a limited set {Θ 1 ,. Θ J } of J weight configurations sampled from the posterior distribution: P (y | x, D) ≈ 1 J J j=1 P (y | x, Θ j ).
Discussion on LP-BNN We discuss here the quality of the uncertainty from LP-BNN. The predictive uncertainty of a DNN stems from two main types of uncertainty  [28] : aleatoric uncertainty and epistemic uncertainty. The former is related to randomness, typically due to the noise in the data. The latter concerns finite size training datasets. The epistemic uncertainty captures the uncertainty in the DNN parameters and their lack of knowledge on the model that generated the training data. In BNN approaches, through likelihood marginalization over weights, the prediction is computed by integrating the outputs from different DNNs weighted by the posterior distribution (Eq. 3), allowing us to conveniently capture both types of uncertainties  [47]. The quality of the uncertainty estimates depends on the diversity of predictions and views provided by the BNN.
Similarly to  [20]  we set M = 15. To evaluate the robustness to dataset shift via corrupted images, we first train the DNNs on CIFAR-10 [38] or CIFAR-100 [38] and then test on the corrupted versions of these datasets  [24]. The corruptions include different types of noise, blurring, and some other transformations that alter the quality of the images. For this scenario, similarly to  [69] , we use as evaluation measures the Corrupted Accuracy (cA) and Corrupted Expected Calibration Error (cE), that offer a better understanding of the behavior of our DNN when facing shift of data distribution and aleatoric uncertainty. In order to evaluate the epistemic uncertainty, we propose to assess the OOD detection performance. This scenario typically consists in training a DNN over a dataset following a given distribution, and testing it on data coming from this distribution and data from another distribution , not seen during training.
Image classification with CIFAR-10/100 [38] Protocol. Here we train on CIFAR-10 [38] composed of 10 classes. For CIFAR-10 we consider as OOD the SVHN dataset  [53]. Since SVHN is a color image dataset of digits, it guarantees that the OOD data comes from a distribution different from those of CIFAR-10. We use WRN-28-10  [75]  for all methods, a popular architecture for this dataset, and evaluate on CIFAR-10-C  [24]. For CIFAR-100  [38]  we use again WRN-28-10 and evaluate on the test sets of CIFAR-100 and CIFAR-100-C  [24]. Note that for all DNNs, even for DE, we average results over three random seeds for statistical relevance. We use cutout  [9]  as data augmentation, as commonly used for these datasets. Please find in the supplementary the hyperparameters for this experiment. Discussion.
Semantic segmentation Next, we evaluate semantic segmentation, a task of interest for autonomous driving, where high capacity DNNs are used for processing high resolution images with complex urban scenery with strong class imbalance. StreetHazards  [23]. StreetHazards is a large-scale dataset that consists of different sets of synthetic images of street scenes. More precisely, this dataset is composed of 5, 125 images for training and 1, 500 test images. The training dataset contains pixel-wise annotations for 13 classes. The test dataset comprises 13 training classes and 250 OOD classes, unseen in the training set, making it possible to test the robustness of the algorithm when facing a diversity of possible scenarios. For this experiment, we used DeepLabv3+ [8] with a ResNet-50 encoder  [22].
We denote the first version LP-BNN and the second one LP-BNN + GN. [23]. BDD-Anomaly is a subset of the BDD100K dataset  [74] , composed of 6, 688 street scenes for training and 361 for the test set. The training set contains pixel-level annotations for 17 classes, and the test dataset is composed of the 17 training classes and 2 OOD classes: motor-cycle and train. For this experiment, we use DeepLabv3+ [8] with the experimental protocol from  [23]. As previously we use ResNet50 encoder  [22]. For this experiment, we use the LP-BNN and LP-BNN + GN variants. Table  3 : Comparative results on the OOD task for semantic segmentation. We run all methods ourselves in similar settings using publicly available code for related methods. Results are averaged over three seeds.
A. 3. Computational complexity Recent works  [13, 72]  studied the weight modes computed by Deep Ensembles under a BNN lens, yet these approaches are computationally prohibitive at the scale required for practical computer vision tasks. Recently, Dusenberry et al. [10]  proposed a more scalable approach for BNNs, which can still be subject to high instabilities as the ELBO loss is applied over a high-dimensional parameter space, all BatchEnsemble parameters. Increased stability can be achieved by leveraging large mini-batches that bring more robust feature and gradient statistics, at significantly higher computational cost (large virtual mini-batches are obtained through distributed training over multiple TPUs). In comparison, our approach has a smaller memory overhead since we encode r j in a lower dimensional space (we found empirically that a latent space of size only 32 provides an appealing compromise between accuracy and compactness).
AUC ↑ AUPR ↑ FPR-95-TPR ↓ ECE ↓ Dataset mIoU ↑ StreetHazards OOD method Baseline (MCP) [25] 53. 90 TRADI [15] 52. 46 Deep Ensembles [39] 55. 59 DeepLabv3+ MIMO [21] 55. 44 ResNet50 BatchEnsemble [70] 56. 16 0. 8660 0. 8739 0. 8794 0. 8738 0. 8817 0. 0691 0. 0693 0. 0832 0. 0690 0. 0759 0. 3574 0. 3826 0. 3029 0. 3266 0. 3285 0. 0652 0. 0633 0. 0533 0. 0557 0. 0609 LP-BNN (ours) 54. 50 0. 8833 0. 0718 0. 3261 0. 0520 LP-BNN + GN (ours) 56. 12 0. 8908 0. 0742 0. 2999 0. 0593 Baseline (MCP) [25] 47. 63 0. 8515 0. 0450 0. 2878 0. 1768 BDD-Anomaly DeepLabv3+ ResNet50 TRADI [15] Deep Ensembles [39] MIMO [21] BatchEnsemble [70] 44. 26 51. 07 47. 20 48. 09 0. 8480 0. 8480 0. 8438 0. 8427 0. 0454 0. 0524 0. 0432 0. 0449 0. 3687 0. 2855 0. 3524 0. 3017 0. 1661 0. 1419 0. 1633 0. 1690 LP-BNN (ours) 49.
Experiment II Data overview. Data of 31 participants initially presented 7440 trials, wherein each of the 120 pairs was repeated twice, resulting in 240 trials per participant. The lexicality and AP conditions were evenly distributed in halves of the total trial count (3720 per condition). Following the same data filtering process as in Experiment I, the exclusion of full omissions led to a reduction in the number of trials to 7344 (1. 29% excluded). The removal of RT outliers (those falling outside the range of 100 ms < RT < 1000 ms) accounted for 144 trials (1. 44%). The resultant dataset comprised 7200 trials, distributed across conditions as follows : lexical (3579 trials), non-lexical (3621 trials), AP close (3588 trials), and AP distant (3621 trials). Given the balanced nature of all conditions, sum contrasts were employed for all factors using sum coding (contr.
ERP results. Three time windows in the response-locked epoch elicited significant differences for overt errors vs. correct condition in both datasets in the ROI analysis of 9 fronto-central electrodes. The first two time windows occurred before production onset. Firstly, the waveform of correct trials differed significantly from errors in the time window between [-483 to -383 ms] (Figure  6 ). This effect was followed by a significant difference between errors and correct trials during the pre-response positive drop [-115 ms, -15 ms] (Figure  4 ), ([-483 ms, -383 ms] and [-115 ms, -15 ms]). A third significant difference between correct trials and errors occurred after the onset of articulation ([62 ms, 162 ms]) (see Figure  6 , Figure  4 ). For the stimulus-locked epochs, no significant effects were observed consistently across both experiments.
Funding This work, carried out within the  Institute of Convergence ILCB  ( ANR-16-CONV-0002 ), has benefited from support from the  French government (France  2030), managed by the  French National Agency for Research (ANR)  and the  Excellence Initiative of Aix-Marseille University  (A*MIDEX). E. R. has benefited from support from the  French government , managed by the  French National Agency for Research (ANR)  through a research grant ( ANR-18-CE28-0013 ).
To perform the simulation, the values of the applied displacement that led to rupture were 2. 45, 5. 24, 6. 38, 9. 1, and 13. 10 mm, for the Notched, Sigma, D, ShapeOpt, and TopOpt, respectively. It is worth noting that for the post-processing step, the numerical data is extracted from the element's integration point in the case of Notched, Sigma, D, and ShapeOpt specimens since shell finite elements are used with reduced integration. However, for the TopOpt specimen, which is simulated using solid finite elements with full integration, the data was extracted from the centroid of the elements located in both front and back surfaces. This allows taking advantage of all the information that can be provided by the test. For all the specimens, independently of the location from which the numerical information was extracted, hereinafter, these locations will be referred as to material points for better readability.
Digital image correlation analysis The results obtained from the numerical analysis of each test were used to generate synthetic images from a real speckle pattern image. Using the MatchID FEDEF module  [60] , the FE mesh configuration as well as the nodal coordinates of each specimen at each load step were used to create a set of deformed images of the specimen during the test. A Digital Image Correlation analysis is then carried out numerically with the set of synthetic deformed images  [49]. A 2D-DIC procedure was carried out for the Notched, Sigma, D, and ShapeOpt specimens. For an accurate 2D-DIC analysis, the specimen is assumed to be planar, to remain planar throughout the test, and to be perpendicular to the camera axis. A large out-of-plane movement such as the one that is detected in the TopOpt specimen will cause errors in 2D-DIC  [45].
Therefore, a stereovision DIC procedure was used for this specimen. Both setup configurations are depicted in Tables B. 5  and B. 6 , respectively (see Appendix B). The DIC analysis was performed using the MatchID software  [60]. The goal of this approach consists in simulating a real experiment, in which the numerical results are passed through a DIC filter  [22, 49, 50]. This procedure adds to the numerical data the influence of real measurement errors. The performance of this procedure greatly depends on how suitable are the chosen DIC settings for the analysis at hand. The Performance Analysis module from MatchID  [60]  was used to make a more deliberate choice on the DIC's user settings. For each test configuration, several combinations of these settings were analyzed and the optimal set was chosen, taking into account a trade-off between resolution and noise.
The strain values of the DIC data points were interpolated to the FE mesh. the specimen design. In order to better understand the areas where the error is higher, the numerical strain field in the ⃗ y direction, represented by LE22 in Figure C. 21, is depicted alongside the map of the relative error for the surface of each test configuration. In the case of the TopOpt specimen, although numerical information has been retrieved from both front and back surfaces, a DIC procedure was carried out only considering the front one. In order to quantify the information given by these maps, the average error for each test is presented in Table  3. It is worth noting that the areas of the specimens where the error is more significant correspond to the areas where a higher strain magnitude is achieved.
Acknowledgments This project has received funding from the  Research Fund for Coal and Steel  under grant agreement No  888153. The authors also acknowledge the financial support under the projects  UIDB/00481/2020  and  UIDP/00481/2020 -FCT - Fundação  para a  Ciência e Tecnologia ; and  CENTRO-01-0145-FEDER-022083 -Centro   Portugal Regional Operational Programme  ( Centro2020 ), under the  PORTUGAL 2020 Partnership Agreement  through the  European Regional Development Fund. M. Gonçalves  is grateful to the  FCT  for the Ph. D. grant  Ref. UI/BD/151257/2021. The authors would like to acknowledge  R. A. F Valente  from the  University of Aveiro  for the invaluable insights and expertise on the Finite Element Method and element formulation that contributed to the improvement of the quality of this work.
By using probabilistic classifiers, Generalized Supervised Meta-blocking associates every pair of candidates with a score that can be used: (i) by any pruning algorithm for retaining the set of candidate comparisons; and (ii) by state-of-the-art progressive ER methods to identify the most promising candidates as early as possible (when time is a critical component for the downstream applications that consume the data). For higher effectiveness, new weighting schemes are examined as features. Through an extensive experimental analysis, we identify the best pruning algorithms, their optimal sets of features as well as the minimum possible size of the training set. The resulting approaches achieve excellent performance across several established benchmark datasets.
Introduction Entity Resolution (ER) is the task of identifying entities that describe the same real-world object among different datasets  [1]. ER constitutes a core data integration task with many applications that range from Data Cleaning in databases to Link Discovery in Semantic Web data  [2, 3]. Despite the bulk of works on ER, it remains a challenging task  [1]. One of the main reasons is its quadratic time complexity: in the worst case, every entity has to be compared with all others, thus scaling poorly to large volumes of data. To tame its high complexity, Blocking is typically used  [4, 5]. Instead of considering all possible pairs of entities, it restricts the computational cost to entities that are similar. This is efficiently carried out by associating every entity with one or more signatures and clustering together entities that have identical or similar signatures.
Our contributions Our goal is to facilitate real-world ER applications by minimizing the set of candidate pairs, while restricting human involvement for the generation of the labelled instances. At the same time, we want to maximize the throughput of ER applications with limited temporal and/or computational resources. To this end, we go beyond Supervised Meta-blocking and in the following ways: 1. We generalize Supervised Meta-blocking from a binary classification task to a binary probabilistic classification process. The resulting probabilities are used as comparison weights, on top of which we apply new pruning algorithms that are incompatible with the binary classification approach. 2. Using the original features, we demonstrate that the new pruning algorithms significantly outperform the existing ones through an extensive experimental study that involves 9 real-world datasets.
We also specify the top performers among the weight-and cardinalitybased algorithms. 3. To further improve their performance, we use four new weighting schemes as features for Generalized Supervised Meta-blocking. We examine the performance of all 255 feature combinations over all nine datasets for the top-performing algorithms. We identify the top-10 feature sets per algorithm in terms of effectiveness and then, we select the optimal one by considering their time efficiency, too, significantly reducing the overall run-time. 4. For each algorithm, we examine how the size of the training set affects its performance. We experimentally demonstrate that it suffices to train our approaches over just 50 labelled instances, 25 from each class. 5. We perform a scalability analysis over 5 synthetic datasets with up to 300,000 entities, proving that our approaches scale well both with respect to effectiveness and time-efficiency under versatile settings.
e. , Progressive Entity Resolution, which was not investigated in the original paper. To this end, we combined our unsupervised progressive algorithm  [16]  with the probabilities generated by the probabilistic classifier. Moreover, we further improve the description of the proposed method by adding Algorithms 1-5 in Section 3 and providing a deeper and more formal definition of the new and existing weighting scheme. Finally, in the experimental evaluation, we explain the behaviour of the algorithm when the training set size increases, we add considerations regarding the obtained recall in each employed dataset, and we compare our work with the state-of-the-art pretrained language model based blocking solutions  [17] [18] [19].
Preliminaries An entity profile 𝑒 𝑖 is defined as a set of name-value pairs, i. e. , 𝑒 𝑖 = {⟨𝑛 𝑗 , 𝑣 𝑗 ⟩}, where both the attribute names and the attribute values are textual. This simple model is flexible and generic enough to seamlessly accommodate a broad range of established data formats -from the structured records in relational databases to the semi-structured entity descriptions in RDF data  [6]. ) ⊆ 𝐸 × 𝐸 ∶ 𝑖 ≠ 𝑗 ∧ 𝑒 𝑖 ≡ 𝑒 𝑗 }. In both cases, the time complexity is quadratic with respect to the input, i. e. , 𝑂(|𝐸 1 | × |𝐸 2 |) and 𝑂(|𝐸| 2  ), respectively, as every entity profile has to be compared with all possible matches. To reduce this high computational cost, Blocking restricts the search space to similar entities  [5]. Meta-blocking operates on top of Blocking, refining an existing set of blocks, 𝐵, a.
2. The emission phase, which retrieves the next best comparison from a list of candidate pairs sorted in decreasing order of their matching likelihood. The initialization phase is computed only once, while the emission phase is repeated whenever a new comparison is requested. The performance of progressive methods is estimated through the progressive recall, which is the fraction of recall reached after emitting 𝑒𝑐 comparisons. Following  [16] , we measure the performance by using the normalized Area Under the Curve, 𝐴𝑈 𝐶 * @𝑒𝑐 * , which varies in the interval [0, 1], with 1 denoting the best progressiveness. 𝑒𝑐 * is the normalized number of emitted comparisons 𝑒𝑐 * = 𝑒𝑐∕|𝐷|, which allows for comparing different datasets on an equal basis. For example, 𝐴𝑈 𝐶 * @10 is the normalized AUC after emitting 𝑒𝑐 = 10 ⋅ |𝐷| comparisons.
Datasets-Table  1a  lists the 9 real-world datasets employed in our experiments. They have different characteristics and cover a variety of domains. Each dataset involves two different, but overlapping data sources, where the ground truth of the real matches is known. AbtBuy matches products extracted from Abt. com and Buy. com  [28]. DblpAcm matches scientific articles extracted from dblp. org and dl. acm. org  [28]. ScholarDblp matches scientific articles extracted from scholar. google. com and dblp. org  [28]. ImdbTmdb, ImdbTvdb and TmdbTvdb match movies and TV series extracted from IMDB, TheMovieDB and TheTVDB  [29] , as suggested by their names. Movies matches information about films that are extracted from imdb. com and dbpedia. org  [6]. WMAmazon matches products from Walmart. com and Amazon.
com  [30]. 3 https://github. com/Gaglia88/sparker. 2. 12 ⋅10 -5 Fig. 6. The average performance of all weight-based pruning algorithms over the block collections of Table  1a. Blocking-To each dataset, we apply Token Blocking, the only parameter-free redundancy-positive blocking method  [11]. The original blocks are then processed by Block Purging  [6] , which discards all the blocks that contain more than half of all entity profiles in a parameter-free way. These blocks correspond to highly frequent signatures (e. g. , stop-words) that provide no distinguishing information. Finally, we apply Block Filtering  [22] , removing each entity 𝑒 𝑖 from the largest 20% blocks in which it appears. The performance of the resulting block collections is reported in the rightmost part of Table  1a. We observe that in most cases, the block collections achieve an almost perfect recall that significantly exceeds 90%.
The only exception is AmazonGP, where some duplicate entities share no infrequent attribute value token -the recall, though, remains quite satisfactory, even in this case. Yet, the precision is consistently quite low, as its highest value is lower than 0. 003. As a result, F1 is also quite low, far below 0. 1 across all datasets. These settings undoubtedly call for Supervised Meta-blocking. To apply Generalized Supervised Meta-blocking to these block collections, we performed 10 runs and averaged the values of precision, recall, and F1. In each run, a different seed is used to sample the pairs that compose the training set. Using undersampling, we formed a balanced training set per dataset that comprises 500 labelled instances. We choose to use a small fixed training set instead of splitting the labelled instances 80/20, because in a real use case scenario manually labelling 80% of the pairs is not feasible in large datasets.
6  and 7 , respectively. Among the weight-based algorithms, we observe that the new pruning algorithms trade slightly lower recall for significantly higher precision and F1. Comparing BCl with WEP, recall drops by -5. 9%, while precision raises by 60. 8% and F1 by 42. 9%. This pattern is more intense in the case of RWNP, which reduces recall by -7. 2%, increasing precision by 68. 5% and F1 by 46. 3%. These two algorithms actually monopolize the highest F1 scores in every case: for ImdbTmdb, ImdbTvdb and TmdbTvdb, WEP ranks first with RWNP second and vice versa for the rest of the datasets. Their aggressive pruning, though, results in very low recall (≪0. 8) in four datasets. E. g. , in the case of AbtBuy, BCl's recall is 0. 852, but WEP and RWNP reduce it to 0. 755 and 0. 699, respectively.
For RCNP, we obtain recall = 0. 850, precision = 0. 248 and F1 = 0. 353 when combining 𝐶𝐹 -𝐼𝐵𝐹 , 𝑅𝐴𝐶𝐶𝐵 and 𝐿𝐶𝑃 with any pair of features from {𝐽 𝑆, 𝑅𝑆, 𝑁𝑅𝑆, 𝑊 𝐽 𝑆}. In this context, we select the best feature set for each algorithm based on time efficiency. In more detail, we compare the top-10 feature sets per algorithm in terms of their running times. This includes the time required for calculating the features per candidate pair and for retrieving the corresponding classification probability (we exclude the time required for producing the new block collections, because this is a fixed overhead common to all feature sets of the same algorithm). Due to space limitations, we consider only the two datasets with the most candidate pairs, as reported in Table  1a : Movies and WMAmazon. We repeated every experiment 10 times and took the mean time.
In Fig. 8 , we observe that the feature set 78 is consistently the fastest one for BLAST, exhibiting a clear lead. Compared to the second fastest feature sets over movies (75) and WMAmazon (96), it reduces the average run-time by 11. 9% and 16. 0%, respectively. For RCNP, the differences are much smaller in Fig. 9 , yet the same feature set (187) achieves the lowest run-time over both datasets. Compared to the second fastest feature sets over movies (184) and WMAmazon (239), it reduces the average run-time by 3. 3% and 4. 8%, respectively. Overall, BLAST models each candidate pair as the 4-dimensional feature vector (ID 78 in Table  2 ): {𝐶𝐹 -𝐼𝐵𝐹 , 𝑅𝐴𝐶𝐶𝐵, 𝑅𝑆, 𝑁𝑅𝑆}. Compared to the feature set of  [13] , recall raises by ∼0. 5% and F1 by ∼1. 5%, while the run-time is reduced to a significant extent (>50% as explained below), due to the absence of the time-consuming 𝐿𝐶𝑃 feature.