dataesr/scientific-paragraphs-categorization huggingface dataset:
37, instead of 98. 37 of PARIS and F1-score is 99. 37, instead of 99. 10 of PARIS), while the precision is lower (99. 37) but still close to  PARIS (99. 85). This happens because of the low literal similarity that these datasets exhibit (affecting the Precision) and the high textual descriptions and entity names similarity (improving Recall and F1-score). Finally, in the new datasets, even if the literal similarity is high, the lack of functional relations affect negatively the performance of PARIS, and as a result, BERT_INT is the clear winner in these datasets (e. g. , in imdb-tmdb F1-score is 15. 53 for PARIS, while BERT_INT achieves 99. 70).
Effectiveness vs efficiency trade-offs In this section, we investigate whether it is worth paying the time overhead of each method, with respect to its effectiveness (Q3). In this respect, in Sect. 4. 2. 1, we first rank the methods according to their effectiveness across all datasets of our testbed. For that, we rely on a statistically sound methodology for ranking the different EA methods across all datasets of our testbed, that exploits the non-parametric Friedman test  (Demsar 2006 ) and the post-hoc Nemenyi test  (Nemenyi 1963). Then, in Sect. 4. 2. 2, we report the execution time of each method (Table  14 ) and analyze their trade-offs in terms of effectiveness and efficiency (Fig. 10 ), demonstrating also 1 3 Knowledge graph embedding methods for entity alignment: an… the training curves of the methods and the time that a method needs to reach 90% of its highest MRR (Fig.
Effectiveness-based ranking of EA methods In this section, for each of the metrics used in Table  8 , we first calculate the ranks (rank 1 corresponds to the best performance) of all methods per dataset and average their ranks for each metric. The resulting ranks are reported in  Tables 10, 11, 12, and 13. Then, to infer a statistically significant ranking of the methods, we rely on the non-parametric Friedman test  (Demsar 2006). The null hypothesis H 0 of the Fried- man test is that "The mean performance for each method is equal", while the alternative hypothesis (H a ) states exactly the opposite. With p-values 0. 004, 0. 003, 0. 001 and 0. 003 of Friedman test for Hits@1, Hits@10, MR and MRR, respectively, we can reject the null hypothesis H 0 at a 10% confidence-level ( ). In the sequel, we conduct the Nemenyi post-hoc test to compare the methods pairwise.
This test reports as significance the average ranks of two methods if they differ by a critical distance (CD) given by q √ k(k+1) 6N , where N is the number of the datasets, q is a constant based on , and k is the number of methods in total. For 9 EA methods, 8 datasets and = 0. 1 , the value of CD is 3. 90. From Fig. 9 , we can see that BERT_INT is a clear winner, since it outperforms the other methods with statistically significant difference (bigger difference than CD). Figure  9a , which captures accuracy with 0 error tolerance (Hits@1), shows that BERT_INT has statistically significant performance difference to MTransE, MTransE+RotatE, AttrE and RDGCN, while RREA(semi) only to MTransE and MTransE+RotatE. In case we accept a 10% error tolerance (Hits@10 in Fig. 9b ), none of the examined methods has significant performance difference to the other methods, so the choice should depend on different factors, e.
g. , the training time (see Sect. 4. 2. 2) or dataset characteristics (see Sect. 4. 3). In addition, BERT_INT that exploits multiple sources of similarity evidence of entities (e. g. , textual descriptions, entity names) and not only structural information, as relation-based methods such as RREA(basic) do, improves the general performance, improving the ranks even of lower ranked entities in the similarities lists (see  Sect. 3. 3. 1). The result is that BERT_INT has significant performance difference with MTransE, RDGCN, MTransE+RotatE, KDCoE and AttrE, while RREA(basic) has significant performance difference with only RDGCN, MTransE+RotatE, KDCoE and AttrE, for MR. In contrast, MRR (Fig. 9d ) which is affected by the first ranks (see  Sect. 3. 3. 2)  and is less sensitive to outliers, shows that BERT_INT is the only method that has statistically significant performance difference with other methods (MTransE and MTransE+RotatE).
3. 3. 2), and the y-axis represents the relative difference in execution time, with respect to the baseline. In particular, the red quadrant (bottom right) indicates methods that both run faster and have a better effectiveness than the baseline (i. e. , dominate the baseline) in a specific dataset, and the green quadrant (top left) exactly the opposite (i. e. , methods dominated by the baseline). The bottom-left grey quadrant indicates methods that run faster with lower performance than the baseline and the top-right grey quadrant indicates methods that have better performance, but are slower than the baseline method. The choice of BERT_INT as baseline is because it is the method that dominates most methods (highest number of methods on the top-left quadrant), and it is not dominated by any other method (0 methods in the bottom-right quadrant).
As shown in Fig. 9d , BERT_INT is the method with the best MRR in both OpenEA and new datasets. Indicatively, the corresponding values for the fastest method, MTransE,  when chosen as a baseline, are: 16 (top left), 46 (top right), 0 (bottom left), and 0 (bottom right). As BERT_INT outperforms all methods in all datasets (except of the two versions of D_W_15K) in terms of MRR, with low a execution time, the top-left quadrant consists of 36 out of 62 (58. 06%) points. The bottom-left grey quadrant contains 16 out of 62 (25. 80%) points, as BERT_INT is slower than the fastest method MTransE in all datasets, as well as slower than the competing method MultiKE in some datasets. More precisely, MTransE is on average 394. 14% faster than BERT_ INT, achieving 138. 73% lower MRR, while MultiKE is 43.
79% faster than BERT_ INT, at a cost of 10. 61% lower MRR in the two versions of D_Y_15K. Finally, since BERT_INT is negatively affected by the low attribute (e. g. , textual descriptions and entity names) similarity in dense KGs, in terms of effectiveness, the top-right Fig. 10  Trade-off between efficiency and effectiveness (MRR) of methods across all datasets, using BERT_INT as baseline. The x-axis is log-scaled using symmetric log function 1 3 Knowledge graph embedding methods for entity alignment: an… quadrant contains 10 out of 62 (16. 12%) points. In KGs with such characteristics, RREA(basic) and RDGCN outperform BERT_INT, achieving 88. 63% and 45. 45% better effectiveness with an overhead of 226. 62% and 42. 53% longer execution time, respectively. In Fig. 11 , we demonstrate the training curves of the methods for each of the OpenEA datasets, while we also point out the training time that a method needs, in order to achieve 90% of its highest MRR (represented as a dot in each curve), similar to the time-to-accuracy (TTA) metric in the literature  (Coleman et al.
2017 (Coleman et al. , 2019)) , measured here in seconds (x-axis labels) and epochs (first element of the points coordinates). As suggested in  Coleman et al. (2017) ,  Coleman et al. (2019) , we run the methods in only one fold of the datasets. We exclude the AttrE method, since it cannot run with D_W_15K_V1 and D_W_15K_V2, while for D_Y_15K_V1 and D_Y_15K_V2 it yields MRR close to zero. In addition, it is worth mentioning that the training curve of RREA(basic) (Fig. 11 ) overlaps with part of the training curve of RREA(semi), since the former is the first iteration (out of 5) of the latter. Last but not least, in Table  16  we demonstrate the coefficient of variation (CV) of methods training epochs per dataset. More precisely, CV measures the relative variability of the methods epochs and is given by ∕ , where is the standard deviation and is the mean.
Meta-level analysis of EA methods In this subsection, we report the statistically significant correlations between the various meta-features (Table  5 ) extracted by the KGs of our datasets and the performance achieved by EA methods (Q4). In this experiment, we exclude the Restaurants dataset, since only AttrE was able to run on this dataset. The null hypothesis (H 0 ) of our correlation analysis is that "there is no significant correlation between the performance metric X and the meta-feature Y", while the alternative hypothesis (H a ) states exactly the opposite. We reject the null hypothesis when the p-values of correlations are less than a significance level. We use the Spearman's correlation, which measures the degree of association between two ranked variables, with a coefficient r s = 1 - 6 ∑ d 2 i n(n 2 -1) , where d i is the difference between the ranks of the two variables and n is the maximum rank.
Seed alignment size As we can see in Table  17 , the size of the seed alignment (#Entity_Pairs) is positively correlated with the supervised and semi-supervised methods MTransE, MTransE+RotatE, RDGCN, RREA(basic), RREA(semi), KDCoE, MultiKE and BERT_INT and negatively correlated with the unsupervised method AttrE. Increasing the seed alignment and consequently, increasing the space of candidate matches, penalizes unsupervised methods, as it also increases the probability of wrong alignments. On the other hand, the size of the seed alignment is positively correlated with the performance of supervised and semi-supervised methods, as a large number of entity pairs in seed alignment implies larger KGs 18  , and by extension more training data. Although BERT_INT is a supervised method, in the new datasets (which are smaller, but exhibit higher factual similarity than the OpenEA datasets), it achieves higher performance than in OpenEA datasets, resulting in a negative correlation with the seed alignment size.
Q2. What is the improvement in the effectiveness of embedding-based entity alignment methods, if we consider not only the structural relations of entities, but also their attribute values? The improvement in the effectiveness (see Table  8 ) of embedding-based EA methods that exploit the factual information of entities depends on the characteristics of the input KGs. In the dense KGs of the OpenEA datasets, where the similarity of factual information (e. g. , textual descriptions and entity names) is low, exploiting only the structural neighborhood of entities seems sufficient for a method to achieve a competitive performance. However, when the similarity of factual information is high, the effectiveness of the methods in the dense KGs is significantly improved, approaching the perfect matching.
In the sparse KGs of the new datasets containing a high number of textual descriptions and very similar literals and predicates, the attribute-based methods outperform most relation-based methods, especially BERT_INT that is the overall winning method. In particular, among all embedding-based methods, both versions of RREA seem to achieve the best performance in dense KGs, when the similarity of factual information (e. g. , textual descriptions and entity names) is low, since RREA exploits multi-hop neighborhoods and utilizes attention mechanism that weights relevant and important neighbors. In contrast, when the similarity of factual information is high, BERT_INT is the best method, since it seems to properly handle it, achieving very high performance. However, in sparse KGs, RREA is outperformed by BERT_INT that achieves the best performance among all methods in the new datasets, as in these datasets, the literals similarity is higher than in the OpenEA datasets.
10 ). Moreover, from Fig. 11 , we conclude that BERT_INT reaches 90% of its highest MRR faster than the other methods, employing only 1 or 2 epochs. It is worth noticing that in dense KGs with low similarity in factual information (e. g. , textual descriptions and entity names), BERT_INT is not as effective as RREA(basic). In such KGs RREA(basic) should be preferred, although it is slower. Q4. To which characteristics of the datasets are supervised, semi-supervised and unsupervised methods sensitive? The effectiveness of supervised and semi-supervised methods is positively correlated with the size of the seed alignment and the density of the KGs (see Table  17 ). Although RDGCN is a supervised method, it is also affected by the degree of similarity between the entity names, as entity embeddings are initialized using the entity name embeddings.
Conclusions and future work In this work, we have experimentally evaluated several translational-and Graph Neural Networks-based methods for implementing EA tasks using KG embeddings, as well as, non-learning methods like PARIS. They essentially cover a wide range of supervised, to semi-supervised and unsupervised methods that exploit both relational and factual information of entity descriptions along with different negative sampling and neighborhood range (one-vs multi-hop) strategies. We have measured both the effectiveness and efficiency of all methods over a rich collection of datasets that exhibit different characteristics of entity descriptions provided by KGs. According to our analysis, negative sampling proves to be a critical factor that affects the performance of both relation-and attribute-based KG embeddings methods, as it helps distancing dissimilar entities in the embedding space.
As future work, we would like to benchmark embedding-based EA methods exploiting different Neural Network architectures, e. g. , Generative Adversarial  (Vretinaris et al. 2021) , Self-Adversarial  (Sun et al. 2019) , Graph Transformer  (Cai et al. 2022). Moreover, we plan to extend our testbed with datasets proposed in OAEI 19  that exhibit a higher structural heterogeneity between the KGs to be aligned (e. g. , in terms of average degrees). Finally, recent works have revealed that KG embedding methods often exhibit direct or indirect forms of bias, leading to discrimination  (Efthymiou et al. 2022; Fisher et al. 2020). We are particularly interested in assessing the robustness of embedding-based EA methods with respect to.
is a sparse dataset constructed by BBCmusic and the BTC2012 version of DBpedia. It contains various entity types such as musicians, their birth places and bands, while also it has the highest number of 4 http:// km. aifb. kit. edu/ proje cts/ btc-2012/. 5 https:// archi ve. org/ detai ls/ kasabi. 6 https:// www. imdb. com/. 7 https:// www. thetv db. com/. 8 https:// www. themo viedb. org/. 9 http:// oaei. ontol ogyma tching. org/ 2010/ im/.
Introduction Histograms are popular non-parametric density estimators available in all statistical computing packages. They are particularly adapted for univariate visualisation (see for instance  Zubiaga and Mac Namee (2016) ) and can serve as the starting point for more complex analyses. Histograms are also quite useful because they provide a compressed representation of the distribution of a random variable. This property has been used for instance in database query optimization, as illustrated in  Oommen and Rueda (2002) ;  Ioannidis (2003). In theory, histograms require very few parameters and can adapt to any kind of distribution given enough bins. In practice however, the choice of the binning can have an unforeseeable effect on the accuracy of data estimation, on the readability of the visualisation and on the size of the representation.
Comparison with other fully automated histogram methods We compare in this section our MDL based methods with state-of-the-art fully automated methods. More precisely, the comparison includes: • NML + heuristic, the NML criterion optimised with our search heuristic; • G-Enum, our fully automated enumerative criterion; • Taut string histograms  (Davies and Kovac, 2004; Davies et al. , 2009) , as implemented in the ftnonpar R package  (Davies and Kovac, 2012) ; • RMG histograms  (Rozenholc et al. , 2010) , as implemented in the histogram R package  (Mildenberger et al. , 2019) ; • Sturges rule histograms, as implemented in Python's numpy library  (Harris et al. , 2020) ; • Freedman-Diaconis rule histograms  (Freedman and Diaconis, 1981) , as implemented in Python's numpy library; • Bayesian blocks  (Scargle et al.
Illustration on a large-scale real-world data set We focus now on the performance and practical relevance of the G-Enum method for modelling an unknown distribution from a real-world large scale data set. The Lunar Crater Database 1 contains 1. 3 million entries on lunar impact craters larger than 1 to 2 km in diameter  (Robbins, 2019). Craters were manually identified and measured from images of NASA's Lunar Reconnaissance Orbiter (LRO), taken from 2011 until 2018. The run time and histogram sizes obtained on this data set are given in Table  9. Although we ignore which law governs craters' diameter distribution, the granulated criterion produces a smoothlooking histogram with only K * = 75 intervals for the 1. 3 million entries (Figure  3 , with both axis in log scale). This fairly compact representation is easy to interpret.
We notice for example, that the first 10 intervals are the most dense ones; they account for about 40% of all entries. Intervals become less dense for higher crater diameters. The last interval spans over about 1500 km of crater diameters and only accounts for 3 data entries. Overall, the shape of our irregular histogram also shows that our approach can capture a power law decrease of the densities. This is in line with astrophysics literature: power or multiple power laws are often used to fit the crater size distribution  (Wang and Zhou, 2016; Minton et al. , 2019). The same information would be hard to convey with an 1 https://astrogeology. usgs. gov/search/map/Moon/Research/Craters/lunar_crater_database_robbins_2018 The density estimation experiment on this data set also shows the scalability of our method: the granular histogram was computed in about 1.
For the next coming years, metaverse, digital twin and autonomous vehicle applications are the leading technologies for many complex applications hitherto inaccessible such as health and life sciences, smart home, smart agriculture, smart city, smart car and logistics, Industry 4. 0, entertainment (video game) and social media applications, due to recent tremendous developments in process modeling, supercomputing, cloud data analytics (deep learning, etc. ), communication network and AIoT/IIoT/IoT technologies. AIoT/IIoT/IoT is a crucial research field because it provides the essential data to fuel metaverse, digital twin, real-time Industry 4. 0 and autonomous vehicle applications. However, the science of AIoT is inherently multidisciplinary, and therefore, it is difficult for readers to understand its evolution and impacts.
Our main contribution in this article is to analyze and highlight the trends and challenges of the AIoT technology ecosystem including core hardware (MCU, MEMS/NEMS sensors and wireless access medium), core software (operating system and protocol communication stack) and middleware (deep learning on a microcontroller: TinyML). Two low-powered AI technologies emerge: TinyML and neuromorphic computing, but only one AIoT/IIoT/IoT device implementation using TinyML dedicated to strawberry disease detection as a case study. So far, despite the very rapid progress of AIoT/IIoT/IoT technologies, several challenges remain to be overcome such as safety, security, latency, interoperability and reliability of sensor data, which are essential characteristics to meet the requirements of metaverse, digital twin, autonomous vehicle and Industry 4.
Due to communication delay between front-end and back-end (upward and downward exchanges), distributed decision making is needed at each layer: IIoT/IoT devices, edge router and the IoT cloud to minimize bandwidth use, and to improve reliability, security and privacy. Moreover, metaverse, digital twin and, in particular, autonomous vehicle applications (dynamic distributed front-end devices) are time sensitive (real-time interaction). Therefore, time-sensitive dynamic IIoT/IoT devices must have the decision-making capability (AIoT device) to interact directly and locally with their environments (e. g. , actuators), and to send some defined data to the IoT cloud through an edge router for global decision making. It is evident that AIoT/IIoT/IoT technology is a crucial research area, and is the cornerstone of metaverse and digital twin as it provides the essential data to power its applications.
In general, to train an ANN, the ReLU activation function, the back propagation algorithm and the stochastic gradient descent (SGD) are used to minimize the ANN output error. The effectiveness of training ANNs with large databases in different applications has been proven; however, this is not the case with SNNs. Unlike ANNs, it is difficult to train SNNs with direct backpropagation due to the non-differentiable nature of their activation functions. One of the most popular ANN-SNN conversion models used is the linear leaky integrate-and-fire (LIF) model  [19]. It is expressed as follows  [20] : V(t) = e -t-t 0 τm t t 0 I inj (t ) C m e t -t 0 τm dt + V(t 0 ) where τ m = C m •R m is the membrane time constant, I inj(t) is the input current, C m is the membrane capacitor, Rm is the membrane resistor and V(t 0 ) is the initial membrane potential  [20].
I inj = w i N ∑ j=1 δ t -j 1 x i where δ( ) is the delta function N = x i and T w = ∑ I inj is the number of spikes in time windows T w. T w 0 I l dt = n l ∑ i=1 w i. n ∑ j=1 T w 0 δ(t -j 1 f i )dt = T. n l ∑ i=1 w i f i In  [20] , the authors demonstrated the equivalence of a linear LIF/SNN and ReLU-ANN model. This proof allowed ANN-SNN conversion and BP with surrogate gradients and direct supervised learning. Moreover, the leaky integrate-and-fire (LIF) neuron has been implemented with a few arithmetic components, such as an adder and comparator  [21]. Kashu Yamazaki et al. presented an overview of different SNN neuron models for different applications  [22]. While a spike timing-dependent plasticity (STDP) model, which is biologically plausible allows unsupervised learning, the lack of global information hinders the convergence of large SNNs to be trained with large and complex datasets.
By doing this, the LiveOS dynamic-stack scheduling performance can be improved  [50]. FreeRTOS is based on a microkernel architecture and adopts a multithreading approach where each thread or process can be interrupted and be pre-empted to execute higher priority process. Zephyr is a multithreading operating system based on a microkernel and nanokernel architecture. Therefore, Zephyr is more flexible than FreeRTOS because it may be configured to run nanokernel to meet low-end IIoT/IoT devices (Figure  7 ). Today, FreeRTOS is the most used operating system for AIoT/IIoT/IoT devices and it is integrated into the middleware software package by most MCU manufacturers: ST, NXP, Silicon Labs, Renesas, Espressif, etc. The minimum memory resource requirements are 4 KB of RAM and 9 KB of ROM. More details of resource requirement are provided in  [54].
Additionally, AWS facilitates IoT cloud integration for smart objects running Fre-eRTOS  [55, 56]. FreeRTOS and Zephyr operating systems support security and safety features. Furthermore, a microkernel architecture is more appropriate to manage processing units based on a multicore architecture, and it can be configured to adapt to the safety application requirements (fault tolerance). Today, FreeRTOS is the most used operating system for AIoT/IIoT/IoT devices and it is integrated into the middleware software package by most MCU manufacturers: ST, NXP, Silicon Labs, Renesas, Espressif, etc. The minimum memory resource requirements are 4 KB of RAM and 9 KB of ROM. More details of resource requirement are provided in  [54]. Additionally, AWS facilitates IoT cloud integration for smart objects running FreeRTOS  [55, 56].