dataesr/scientific-paragraphs-categorization huggingface dataset:
{𝐶𝐹 -𝐼𝐵𝐹 , 𝑅𝐴𝐶𝐶𝐵, 𝐽 𝑆, 𝐿𝐶𝑃 , 𝑊 𝐽 𝑆}. This reduces recall by <0. 3%, but raises precision and F-Measure by 1. 2%, which is in-line with the desiderata of cardinality-based algorithms. Comparison with Supervised Meta-blocking-We now compare BLAST and RCNP in combination with the features selected above with BCl and CNP, which use the feature set proposed in  [13] , {𝐶𝐹 -  𝐼𝐵𝐹 , 𝑅𝐴𝐶𝐶𝐵, 𝐽 𝑆, 𝐿𝐶𝑃 }. All algorithms were trained over the same randomly selected set of 500 labelled instances, 250 from each class, and were applied to all datasets in Table  1a. Their average performance is presented in Fig. 10. We observe that BLAST outperforms BCl with respect to all effectiveness measures: its recall, precision and F1 are higher by 1. 6%, 13. 6% and 13%, respectively, on average. Thus, BLAST is much more accurate in the classification of the candidate pairs and more suitable than BCl for recall-intensive applications.
Among the cardinality-based algorithms, RCNP trades slightly lower recall than CNP for significantly higher precision and F1: on average, across all datasets, its recall is lower by -4. 1%, while its precision and F1 are higher by 34. 9% and by 33. 6%, respectively. As a result, RCNP is more suitable than CNP for precision-intensive applications. Regarding time efficiency, Fig. 11  reports the running times of these algorithms on the largest datasets, i. e. , Movies and Walmar-tAmazon. We observe that BCl, CNP and RCNP exhibit similar 𝑅𝑇 in both cases, since they all employ more complex feature sets that include the time-consuming feature 𝐿𝐶𝑃. BLAST is substantially faster than these algorithms, reducing 𝑅𝑇 by more than 50%. In particular, comparing it with its weight-based competitor, we observe that BLAST is faster than BCl by 2.
The effect of training set size We now explore how the performance of BLAST and RCNP changes when varying the training set size. We used the features sets selected  above (ID 78 and 187 in Tables  2  and 3 , respectively) and varied the number of labelled instances starting from 20, then from 50 to 500 with a step of 50. Figs. 12 and 13 report the results in terms of recall, precision and F1, on average across all datasets, for BLAST and RCNP, respectively. Notice that both algorithms exhibit the same behaviour: as the training set size increases, recall gets higher at the expense of lower precision and F1. However, the increase in recall is much lower than the decrease in the other two measures. More specifically, comparing the largest training set size with the smallest one, the average recall of BLAST raises by 2.
This is illustrated in Fig. 14 , which depicts the density of matching probabilities of candidate pairs over the AbtBuy dataset for various training set sizes, when using the Logistic Regression as the classification algorithm. The duplicate pairs are shown in red and the non-matching ones in blue. The upper line corresponds to the maximum pruning threshold and the lower line to the average one, across all nodes/entities. We observe that for the smallest training set, the matching probabilities of both types of candidate pairs fluctuate in  It is worth noting at this point that the same behaviour as BLAST is exhibited by BCl, the original binary classifier presented in  [13] , which simply retains all candidate pairs with a matching probability above 0. 5. As shown in Fig. 15 , both algorithms increase their recall and decrease their precision as more labelled instances as used during their training.
Given that 20 labelled instances yield very low recall, especially for RCNP, but with 50 instances, the recall becomes quite satisfactory for both algorithms (≫0. 85, on average, across all datasets), we can conclude that the optimal training set involves just 50 labelled instances, equally split among positive and negative ones. Comparison with Supervised Meta-blocking-Based on the above experimental results, we now compare the final algorithms of Generalized Supervised Meta-blocking with their baseline counterparts from  [13]. Table  4a -c reports a full comparison between the main weightbased algorithms, i. e. , BLAST and BCl (note that BCl 2 uses the training set specified in  [13] , i. e. , a random sample involving 5% of the positive instances in the ground-truth along with an equal number of randomly selected negative instances).
We observe that on average, BLAST outperforms BCl 2 with respect to all effectiveness measures, increasing the average recall, precision and F1 by 7. 1%, 5. 0% and 9. 9%, respectively. Compared to BCl 1 , BLAST increases the average recall by 3. 95%, but lowers the precision by 5. 9% and the F1 by 2. 2%. Recall drops below 0. 8 in four datasets for BCl 1 (and BCl 2 ), whereas BLAST violates this limit in just two datasets. This should be attributed to duplicate pairs that share just one block in the original block collection, due to missing or erroneous values, as explained in detail in Section 5. 4. 1. BCl 1 outperforms BCl 2 in all respects, demonstrating the effectiveness of the new feature set. In terms of run-time, BLAST is slower than BCl 1 by 8. 2%, on average, because it iterates once more over all candidate pairs.
Compared to BCl 2 , BLAST is 6. 7 times faster, on average across all datasets, because of 𝐿𝐶𝑃 and of the large training sets, which learn complex binary classifiers with a time-consuming processing. Similar patterns are observed in the case of cardinality-based algorithms in Table  4d -f, where RCNP with its best features and 50 labelled instances is compared with CNP, which uses the best feature set from  [13]  and two different training sets: the same 50 labelled instances as RCNP (CNP 1 ) and the training set specified in  [13]  (CNP 2 ). In more detail, RCNP typically outperforms both baseline methods with respect to all effectiveness measures. Compared to CNP 1 (CNP 2 ), RCNP raises the average recall by 5. 3% (9. 2%), while achieving the highest precision and F1 across all datasets, except for AbtBuy and ImdbTmdb (and ScholarDblp in the case of CNP 1 ).
The relative increase in precision in comparison to CNP 1 ranges from 7. 5% over TmdbTvdb to 10 and 24 times over Movies and WalmartAmazon, respectively. Compared to CNP 2 , precision raises from 5. 0% over DblpAcm to 49 times over WalmartAmazon. In all cases, F1 increases to a similar extent. These patterns suggest that RCNP is typically more accurate in classifying the positive candidate pairs. In terms of run-time, RCNP is slower than CNP 1 by 6. 2%, on average, as it retains the candidate pairs that are among the top-k weighted ones for both constituent entities (i. e. , it searches for pairs in two lists), whereas CNP 1 simply merges the lists of all entities. CNP 2 employs a much larger training set, yielding more complicated and time-consuming classifiers than RCNP, which is 3 times faster, on average, across all datasets.
Considerations regarding the obtained recall Blocking recall typically sets the upper bound on matching recall, i. e. , on the overall recall of ER. The problem of low recall appears in cases where 𝑃 𝐶 < 0. 9 in Table 4a-c, which reports analytically the performance over all datasets for the weight-based algorithms that emphasize recall. The rest of the tables report the average recall across all datasets, which is very close to 0. 9 for BLAST in most cases, while Table  4d -f examines cardinality-based algorithms, which emphasize precision (e. g. , progressive ER applications that by default operate with lower recall). Studying Table  4a -c, we observe that the recall of BLAST is lower than 0. 9 in five datasets: AbtBuy, AmazonGP, ImdbTmdb, ImdbTvdb and TmdbTvdb. Comparing these datasets with the remaining four, where recall is well above 0.
This should be attributed to the new features we have proposed in this work, which normalize the co-occurrence patterns by considering all blocks containing every entity. In block collections with high levels of redundancy, where the vast majority of duplicates co-occurs in multiple blocks, the recall remains high even after Supervised Meta-blocking. This is verified in Figs. 17 and 18, which plot the portion of matching pairs (on the vertical axis) with respect to the number of common blocks (on the horizontal axis). In every dataset, the bar that  corresponds to 𝑥 = 0 indicates the portion of duplicates that are missed by the original block collection. The bar that corresponds to 𝑥 = 1 indicates the portion of duplicates that are missed by (Generalized) Supervised Meta-blocking. We observe that for all datasets in Fig.
Scalability analysis We assess the scalability of our approaches as the number of candidate pairs |𝐶| increases, verifying their robustness under versatile settings: instead of real-world Clean-Clean ER datasets, we now consider the synthetic Dirty ER datasets, and instead of SVC, we train our models using Weka's default implementation of Logistic Regression  [31]. The characteristics of the datasets, which are widely used in the literature  [4, 11] , appear in Table  1b. To extract a large block collection from every dataset, we apply Token Blocking. In all cases, the recall is    We consider four methods: BCl and CNP with the features and the training set size specified in  [13]  as well as BLAST and RCNP with the features in Table  4a  and d, respectively, trained over 50 labelled instances (25 per class).
In each dataset, we performed three repetitions per algorithm and considered the average performance. The effectiveness of the weight-and cardinality-based algorithms over all datasets appear in Fig. 16a  and b , respectively. BLAST significantly outperforms BCl in all cases: on average, it reduces recall by 3. 5%, but consistently maintains it above 0. 93, while increasing precision and F1 by a whole order of magnitude. Note that BLAST's precision is much higher than expected over 𝐷 100𝐾 , due to the effect of random sampling: a different training set is used in every one of the three iterations, with two of them performing a very deep pruning that decrease recall to a minor extent. RCNP outperforms CNP to a significant extent: on average, it reduces recall by 7. 9%, but maintains it to very high levels -except for 𝐷 200𝐾 , where it drops to 0.
77, due to the effect of random sampling; yet, precision raises by 2. 8 times and F1 by 2. 3 times. These results verify the strength of our approaches, even though they require orders of magnitude less labelled instances than  [13]. Most importantly, our approaches scale better to large datasets, as demonstrated by speedup in Fig. 19. Given two sets of candidate pairs, |𝐶 1 | and |𝐶 2 |, such that |𝐶 1 | < |𝐶 2 |, this measure is defined as follows: 𝑠𝑝𝑒𝑒𝑑𝑢𝑝 = |𝐶 2 | |𝐶 1 | × 𝑅𝑇 1 𝑅𝑇 2 , where 𝑅𝑇 1 (𝑅𝑇 2 ) denotes the running time over |𝐶 1 | (|𝐶 2 |) -in our case, 𝐶 1 corresponds to 𝐷 10𝐾 and 𝐶 2 to all other datasets. In essence, speedup extrapolates the running time of the smallest dataset to the largest one, with values close to 1 indicating linear scalability, which is the ideal case.
We observe that all methods start from very high values, but BCl and CNP deteriorate to a significantly larger extent than BLAST and RCNP, respectively, achieving the lowest values for 𝐷 300𝐾. This should be attributed to their lower accuracy in pruning the non-matching comparisons, which deteriorates as the number of candidate pairs increases. As a result, they end up retaining and processing a much larger number of comparisons, which slows down their functionality. Overall, Generalized Supervised Meta-blocking scales much better to large datasets than Supervised Meta-blocking  [13]  for both weight-and cardinality-based algorithms. For a bit lower recall, it raises precision and F1 by ≥2 times and maintains a much higher speedup.
This applies to all Clean-Clean and Dirty ER datasets in Table  1. The results appear in Figs. 20  and 21 , respectively. Starting with the Clean-Clean ER datasets, we observe that Supervised PPS consistently outperforms Unsupervised PPS to a significant extent across all datasets and settings. The lowest difference between the two approaches with respect to 𝐴𝑈 𝐶 * 𝑚 @1, 𝐴𝑈 𝐶 * 𝑚 @5 and 𝐴𝑈 𝐶 * 𝑚 @10 corresponds to ScholarDBLP, where the former outperforms the latter by 14. 3%, 19. 6% and 26%, respectively. For 𝐴𝑈 𝐶 * 𝑚 @20, the lowest difference (25. 5%) appears in Amazon-GP. Yet, there is a large variation in the difference between the two methods among the various settings. For example, the progressive recall of the supervised approach is at least twice higher than the unsupervised one in many cases: in Movies for 𝑒𝑐 * ∈ {1, 5, 10}, in WalmartAmazon for 𝑒𝑐 * ∈ {5, 10, 20} and in Imdb-Tvdb and Tmdb-Tvdb for 𝑒𝑐 * = 1.
7 ⋅ 10 -17 ) under all settings. We actually observe that the deviation between the two methods is low for 𝐷 10𝐾 , but gets higher and almost stable for the rest of the datasets. This applies to all settings of Progressive Recall, i. e. , for 𝑒𝑐 * ∈ {1, 5, 10, 20}. In more detail, 𝐴𝑈 𝐶 * 𝑚 @1 is higher for Supervised PPS than for Unsupervised PPS by 66. 3% over 𝐷 10𝐾 and by 79. 1 ± 2. 2% over 𝐷 50𝐾 -𝐷 300𝐾. For 𝐴𝑈 𝐶 * 𝑚 @5 (𝐴𝑈 𝐶 * 𝑚 @10), the superiority of the former approach raises from 34. 8% (27. 2%) over 𝐷 10𝐾 to 49. 8 ± 2. 2% (44. 9 ± 1. 5%) over the rest of the datasets. This is caused by the more challenging tasks that are posed by the larger datasets: the number of duplicates increases linearly with the size of the input data, while the possible comparisons increase quadratically.
As a result, the portion of non-matching pairs over all comparisons is lower in 𝐷 10𝐾 than the rest of the datasets, thus enabling Unsupervised PPS to provide more accurate weights. In other words, the individual weights of Unsupervised PPS become less capable of distinguishing between matching and nonmatching comparisons than the probabilities of Supervised PPS when the candidate pairs increase. The only exception is 𝐴𝑈 𝐶 * 𝑚 @20, because the higher computational budget yields higher absolute recall for both methods. As a result, the difference between Supervised and Unsupervised PPS increases gradually, as the task becomes more challenging, due to a higher portion on non-matching comparison. Indeed, the former approach is better than the latter by 19. 5% over 𝐷 10𝐾 , by 37. 6% over 𝐷 50𝐾 and by 46.
6 DeepBlocker  [17]  proposes a self-supervised technique to learn the embeddings per entity profile, without requiring any labelled training data. As stated in  [17] , it is crafted for datasets with perfectly aligned attributes. However, among the datasets in Table  1 (a), only Dblp-Acm, AbtBuy, and Scholar-Dblp satisfy this requirement. We solved this problem by concatenating all the values of the different attributes in a single string on which the blocking was performed, i. e. , using schemaagnostic settings just like Generalized Supervised Meta-blocking. For the generation of tuple embeddings, we used the AutoEncoder, because it constitutes the most effective module under the schema-aware settings, while ranking second in the schema-agnostic ones  [17]. The Hybrid module exhibits slightly higher effectiveness under these settings, but suffers from significantly higher space and time complexity, raising out-of-memory exceptions in most cases, while being orders of magnitude slower than the AutoEncoder.
ContextualBlocker  [19]  is an unsupervised and schema-agnostic method: first, it generates the embedding vectors by combining SimCSE   [34]  with the BERT language model  [35]  and then, it computes the 𝑘 nearest neighbours of each profile. The retrieved candidates construct a graph, where every node corresponds to an entity profile and edges connect the candidate pairs. Finally, it applies the Louvain community detection algorithm  [36]  on the resulting graph to generate the blocks. The experimental evaluation was conducted by setting 𝑘 = 50 for the kNN selection so as to maximize recall. All kNN experiments were performed on an NVIDIA GeForce RTX 4090 GPU with 24 GB of RAM. The obtained results across all the datasets are reported in Table  5. DeepBlocker exhibits a slightly better recall than BLAST on four datasets (namely AbtBuy, Dblp-Acm, Amazon-GP, and Tmdb-Tvdb) and on three (i.
e. , AbtBuy, Dblp-Acm, and Tmdb-Tvdb) when compared to RCNP. On all of the other datasets, their recall is basically the same. The only exception is Movies, where DeepBlocker obtains a very low recall. That happens because the duplicate films are mostly identified by the names of the cast rather than the film's title; yet the embedding vectors of personal names have low accuracy (even for the fine-grained character-level LM of fastText), given that the training corpora are generic, not domain-specific. Sudowoodo with 50 labelled pairs performs worse that our approach basically on all the datasets but Dblp-ACM and Scholar-Dblpon these two datasets, almost all blocking methods achieve very high recall, due to the relatively clean, unambiguous bibliographic records they contain. Overall, Sudowoodo obtains a very low recall, which is improved when increasing the number of labelled training data from 50 to 500-even though it does not reach the performance of Generalized Supervised Meta-blocking.
This indicates that if more labels are provided, Sudowoodo raises its effectiveness to a significant extent, at the expense of a much higher labelling effort by human experts. ContextualBlocker outperforms our solutions only on AbtBuy, obtaining a much lower recall on all the other datasets. Regarding precision, BLAST and RCNP outperform all the other methods on all the datasets by at least two orders of magnitude. Finally, on average, the execution time is almost the same for BLAST, RCNP, and DeepBlocker, while ContextualBlocker and Sudowoodo are 2. 9 and 8. 8 times, respectively, slower than BLAST, on average. The differences are significantly higher (in favour of BLAST and CNP), when running the semantic-based blocking methods on a CPU core. [20]. Sparkly is a recent approach based on TF/IDF  [37]  that has been shown to outperform pre-trained language model-based blocking techniques  [20]   1 ).
Comparison with Sparkly Finally, we measured the execution time of Sparkly after configuring it so that it uses a single CPU core -this way, we can perform a fair comparison with Supervised BLAST. With the exception of the two smallests datasets, AbtBuy and DblpAcm, where the two methods are equally fast, Sparkly is consistently slower than Supervised BLAST to a significant extent. Tts average run-time, across all nine datasets, is 3. 5 times higher than that of Supervised BLAST. It is worth stressing at this point that one major limitation of Sparkly is that the parameter 𝑘 has to be manually set. This is rarely easy, but has a significant impact on the final performance. For example, if we set 𝑘 = 1 on AmazonGP, Movies and WalmartAmazon, its recall drops to 0. 4908, 0. 6559 and 0. 8536, respectively, which is much lower than that of Supervised BLAST.
The opposite is true if 𝑘 = 10, the lowest value that is used in  [20] , whose performance is reported in Table  5 (h). We observe that on average, across all nine datasets, its recall raises by ∼4%, but its precision drops by 6. 7 times, being 7 times lower than Supervised BLAST. This means that expert knowledge or perhaps a training set is required for fine-tuning 𝑘 on the data at hand. Overall, we conclude that our techniques are still competitive with LM-based blocking solutions, yielding the same level of recall for a significantly better level of precision, while exhibiting a lower execution time without requiring GPUs. Compared to Sparkly  [20] , a recent TF/IDF-based approach that outperforms LM-based ones, Supervised BLAST offers a much better trade-off between recall and precision in most cases, while being consistently faster.
Progressive ER  [14]  is employed when resources are limited (e. g. , cloud budget or human time to refine the ER pipeline), or the data is periodically changing and it has to be consumed within a certain time to be valuable for downstream applications. Indeed, existing progressive ER methods  [14, 16, 41, 44]  try to quickly sort candidate matches by their matching likelihood (typically estimated through a proxy measure derived from a blocking strategy), so as to discover as many matches as fast as possible. In particular, PPS  [16]  is the state-of-the-art schemaagnostic progressive ER method; in this paper we adapted it to our proposed weighting strategy and showed the non-negligible benefit of doing so on benchmark datasets.
Conclusions We have presented Generalized Supervised Meta-blocking, which casts Meta-blocking as a probabilistic binary classification task and weights all candidate pairs in a block collection with the probabilities produced by the trained classifier. These weights are processed by a pruning algorithm that can be: (i) weight-based, determining the minimum weight of retained pairs in a way that promotes recall, or (ii) cardinality-based, determining the maximum number of retained pairs in a way that promotes precision. Through a thorough experimental study over 9 established, realworld datasets, we verified that BLAST and RCNP constitute the best weight-and cardinality-based pruning algorithms, respectively. We also demonstrated that four new weighting schemes give rise to feature sets that outperform the one determined in  [13]  as optimal.
Acknowledgments This work was partially supported by the  Horizon Europe  project  STELAR  (Grant No. 101070122 ) and the  Department of Engineering  '' Enzo Ferrari '' within the  FARD-2022  ( FAR2022DIP_DIEF ) and  FARD-2023  ( FAR_DIP_2023_DIEF-SIMONINI   - Grant No. E93C23000280005 ) projects.
[0. 5, 0. 95], with the density of the matching and the non-matching pairs being higher in the upper and the lower part, respectively. As we move to larger training sets, the matching pairs are pushed up, fluctuating in [0. 75, 0. 95]. This indicates the higher discriminatory power of the probabilistic classifier, which increases the number of true positives, raising recall. However, the probabilities of the non-matching pair continue to fluctuate in [0. 5, 0. 85], but are now concentrated around 0. 7, while the most pruning thresholds are confined in [0. 63, 0. 65]. As a result, the number of false positive increases, too, dropping precision to lower levels. Similar patterns appear in the rest of the datasets in combination with other classification algorithms, such as SVC.
To achieve this purpose, it needs a new way of handling the computation of matrix elements, since it relies on a self-generated FormCalc  [5]  code, therefore not meant to be particularly flexible. An effort in the direction of generalising and automating the computations has already been performed through the development of the code MARTY  [6] , which allows for the automatic calculation of amplitudes, crosssections and Wilson coefficients in a large variety of NP models up to the one-loop level. However, a step further is needed to compute dark matter observables, such as dark matter relic density, which can involve thousands of scattering amplitudes. Other codes publicly available to calculate dark matter observables are DarkSUSY  [7, 8] , MadDM  [9, 10]  and MicrOMEGAs  [11]. Despite its name DarkSUSY is model-independent.
In fact, provided some inputs such as the DM mass and its self-annihilation cross-section, it is possible to compute the relic density, alongside other observables related to direct or indirect detection of dark matter. Moreover, it has a modular structure that makes it easy to link with other tools. MadDM and MicrOMEGAs are also model-independent: the former relies on Madgraph  [12] , and the latter on CalcHEP  [13]. It is therefore possible to provide model inputs compatible with those softwares and then all the relevant quantities for relic density calculation, as for instance the annihilation cross sections, are computed from their respective backends, in order to compute DM relic density and other DM related observables. In order to test a new model in any of these frameworks, the generation of the necessary input files would have to be performed with Sarah  [14].