dataesr/scientific-paragraphs-categorization huggingface dataset:
CONCLUSION Smart contracts reduce counterparty risks, but, as illustrated by the Terra debacle,  17  they do not eliminate contagion risks resulting from flaws in the economic design of their protocol. Fortunately, the transparency of blockchains offers an opportunity to develop sophisticated supervisory tools. The availability of exhaustive and exact data makes it possible to monitor the sources of systemic risks in real time with a precision that far exceeds what is possible in the traditional financial sector. Our paper, by applying classical methods for the analysis of financial networks, provides an example of such an endeavor. It characterizes how contagion might spread through Compound's network, identifying the pools that are more likely to set off or propagate a domino effect. Moving forward, we aim to further explore this research avenue by delving deeper into the trove of data accumulated during this study.
However, to date the majority of these have been developed for image classification tasks. It is necessary to study adversarial attacks in order to assess the robustness of the models, and to prevent them on critical systems. For example, Eykolt et al. [6]  Fig. 1  Scheme of adversarial attack. Time series from the BME dataset, perturbation generated with SGM, not represented at scale showed an application on real-world road sign classification, which is an obvious threat for autonomous vehicles. Fawaz et al. [10]  introduced and adapted some of them for time series classification. The main difference between adversarial attacks on images and time series lies in the visualization and the interpretation of the data. When slightly changing the value of one or few pixels, an image will always look the same and have the same appearance to the human visual system.
We will show that it outperforms BIM's performance over most of the UCR archive datasets in terms of misleading a deep learning classifier. But unfortunately this method generates perturbations that also contain spike and sawtooth patterns. We will then explain how we reduced these patterns, by enforcing a smoothness condition. First, we will show its impact visually, using a set of examples. Secondly, we introduced a function used to measure and compare the smoothness of time series. Finally, we will show how adversarial training is a good way to improve a time series classifier's robustness against smoothed perturbations.
Gradient method (GM) In order to correct the flaws of BIM, we need to design a method that, given a model, can perturb a time series while optimizing the quantity of noise according to the L2 norm. Ensuring f (x) = f (x ) can be written as a maximization problem of the KL-divergence between the two probability distributions, as follows: max D K L ( f (x), f (x )) ≡ c f (x) log f (x) f (x ) , ( 4 ) with c denoting the classes in the dataset. Generating an adversarial example can then be written as follows where the primary addition is the term (-xx 2 ) to be maximized: max μD K L ( f (x), f (x )) -x -x 2 , ( 5 ) with μ denoting a hyper-parameter to control the penalty of miss-classification. Let us consider the generated time series x = x + r. Then the maximization problem is equivalent to the following minimization problem: min {-μD K L ( f (x), f (x + r)) + r 2 } (6) We can add a hyper-parameter α in order to control the regularization of r.
Classifier and datasets As our work consists of fooling a deep learning model, we decided to use the InceptionTime  [11]  classifier in all our experiments. InceptionTime is a TS classifier, which was the state-of-the-art model on the UCR Archive  [4] , when published in 2019. During our preliminary work, we tested other deep models like ResNet or FCN. We decided not to add them to the paper because we did not notice any visual or quantitative difference from InceptionTime. As InceptionTime is still the state-of-the-art model for deep learning TSC, we decided to only focus on this model. All the weights used are the InceptionTime defaults, as used and presented in its paper. In order to demonstrate our results over several datasets, we used the well know TSC benchmark UCR Archive. The 2018 version of this archive comprises 128 univariate time series datasets.
Each dataset of the UCR Archive is split between the training and the test sets. When generating adversarial samples, we used the samples of the test set, as the model has only been trained on the training set. A single experiment is realized on the RTD dataset  [1]. It was build upon a technique called air writing. It consists of writting characters in the air, here digits, which are then recognized by a computer or a mobile phone. This dataset is a collection of 20k trajectories made by various people. A class is assigned for each one of the ten digits.
SGM benchmark Given Fig. 2 , as the dots are evenly distributed, we conclude that SGM as an overall ASR as good as BIM on the UCR archive. This also means that SGM manages to perturb  datasets that BIM cannot and vice-versa. But for an equivalent efficiency, BIM introduces an higher quantity of noise than SGM, in a majority of datasets. Figure  3  compares SGM with C&W. Regarding the ASR we notice that SGM is slightly better than C&W. The same can be concluded for the L2 norm. But when looking at the distribution of the scores, we can all SGM's scores are below 2. 5, where many C&W scores are bigger. This suggests that the C&W attack produces more noise than SGM. Figure  4  compares GM with SGM. We can see that for almost all datasets, GM has a better ASR than SGM. Intuitively, we can assert that sawtooth and spikes are patterns that help to successfully perturb a TSC.
Varying the parameter According to our previous results, the best-case scenario would be an attack with GM's ASR and SGM's smoothness. As the only difference between the two methods is the addition of the smoothness condition, it is interesting to vary the λ parameter. If λ is equal to zero, the attack is GM, and if it's equal to 1, we have SGM as we tested it previously. Figure  5  shows the impact of varying the λ parameter over two datasets, Beef and Car. As we could expect, the more we enforce the smoothness condition, the fewer samples the method manages to perturb successfully. This parameter should be tuned for each dataset in order to get the optimal trade-off between smoothness and ASR.
Visual comparison In order to remain undetectable by the naked eye, an attack performed on a time series must be as smooth as possible. We propose a visual comparison between the four methods presented, on the same test sample of the Beef dataset. To be fair, we picked a time series that is successfully perturbed by all the attacks. In this example, shown in Fig. 7 , we plotted in green the original time series, and for each method, in blue the perturbed time series and in red the perturbation. We plotted a second version of GM with a clipped perturbation in the same way as BIM. As expected, for BIM and the GM methods, the perturbations are clearly visible, in particular the parts containing sawtooth patterns that are circled in purple. The example of GM with clipping shows that clipping the noise reduces indeed the amount of noise and the visual impact, but not sufficiently enough.
SGM is the only attack that produces an adversarial sample with a perturbation that is not noticeable when judging with the naked eye. The C&W attack is the one that produces the smallest noise. The magnitude of the noise is significantly smaller than the other methods. Thus, the sawtooth patterns are less noticeable but are still present. Being closer to the eye doesn't mean being closer when using the L2 metric. Indeed, SGM's perturbation has the biggest L2 norm. This shows that, although a method is better on average for a given dataset, this is not necessarily true when we look at each sample independently.
Comparison of the smoothness In the previous subsection, we did a visual comparison. Now, using the function 11, we can compute the average for each dataset. We computed the average rank of smoothness for each attack on the UCR archive. These results are represented in Fig. 8. Like the visual comparison, SGM appears as the method that produces the smoother samples and BIM is the opposite. For GM, as expected, clipping the noise reduces its maximum amplitude, thus making it smoother. The C&W method ranks third below SGM and GM with clipped noise. Nevertheless, C&W ranks first among the methods that do not enforce the smoothness of the perturbations.
Do adversarial attacks change the underlying class of the samples? While in images human perception is a meaningful gold standard, in TSC most of the time it is not. If we classify traffic signs into stop signs and other signs, humans can easily do that. The goal of a machine is then to achieve human performance. The central argument of adversarial attacks is that while to us humans it clearly still is a stop sign, i. e. , the underlying class has not changed, the algorithm now suddenly predicts it as something else. Few time series datasets are easy to visualize and understand at a glance. RTD is one of them. When perturbing this dataset with SGM, we got an ASR equal to 0. 829. As most of the samples were successfully perturbed, we can observe in Fig. 9  that the perturbed samples do not differ visually from the originals.
But most time series datasets are not as easy to visualize as the RTD dataset. For those datasets, we assume that by doing small perturbations the underlying class will not change. But in reality, we cannot be certain that this is the case. So, the question is now what could be a good gold standard to determine a time series' underlying class? In this section, we propose to use 1 Nearest Neighbor (1NN) to determine whether the underlying classes change or not. The 1NN classifier assigns to a sample the class of its closest neighbor. Here we use 1NN combined with the Dynamic Time Warping (DTW) distance. We experimented on a subset of 22 datasets of the UCR Archive. All the results are available in our GitHub repository. By observing the confusion matrix produced, we can identify 2 groups. The first one is constituted by the datasets ArrowHead, BeetleFly, Car, Earthquake, FaceAll, FordB, FreezerSmall, Ham, InerSkate, InsectWing-beatSound, Lightning7, MoteStrain, OliveOil, ProximalPha-lanxTW, TwoPatterns, Wine, WordSynonyms and Yoga.
The second one by the datasets EOGVerticalSignal, Gun-PointOldVersusYoung, and MelbournePedestrian. In the first group, for a majority of samples, the 1NN classifier predicts the same class whether they are perturbed or not. This shows that the underlying class did not change and that the adversarial attack was conducted as expected. For the second group of datasets, the results are different. For each one of these datasets, the perturbed samples were all classified as belonging to a single class. In those cases, the underlying classes did change. Two examples, one of each group, are shown in Fig. 10. Through these examples, we have shown that, unlike computer vision, adversarial attacks on time series can change the underlying class. Nevertheless, for 19 out of the 23 datasets we tested, it is not the case.
Discriminating adversarial samples A simple way to protect a model against adversarial attacks is to filter the inputs before feeding them to the network. One can do this by training a small classifier to discriminate perturbed samples. If an example is classified as an attack, then it will not be fed to the second classifier. In order to know if a classifier is indeed able to do so, we trained a small FCN model. For each dataset, the samples have been split into a training and a test set: 80% of the samples in the training set, and the remaining 20% in the test set. Then, the samples of each set have been perturbed and added back to the set. Thus, we obtained, in the end, datasets balanced between original and perturbed samples. Table  1  shows the classification results of the five methods presented previously.
Adversarial training Adversarial training aims to improve the robustness of a model but without the disadvantages of the previous method. Figure  11  presents the results of adversarial training using SGM adversarial samples. The scatter plot shows that adversarial training highly improves the robustness of the model. For 122 datasets, the ASR is highly lower, and for the remaining 6 datasets, the ASR is the same. In fact, for 72 datasets, after adversarial training, the attack did not manage to perturb any sample, leading to an ASR of zero. This shows the effectiveness of adversarial training against SGM attacks. We can explain that adversarial training is very effective against SGM attacks because the strength of smoothed attacks is also their weakness. The adversarial samples generated by SGM are close to the original samples.
This highlights the current trade-off between having a stealth attack and an effective one. Finally, we proposed two methods to counter adversarial attacks. The first one leverages a FCN classifier by training it to discriminate perturbed time series. We showed that the FCN model is better at discriminating perturbed samples from BIM, GM, or C&W attacks than SGM. This proves that generating smooth perturbations is not only better at fooling humans but also deep classifiers. Then, we showed that adversarial training is an effective way of countering SGM attacks. After adversarial training, for 72 datasets out of the 128 datasets of the UCR archives, it becomes impossible to successfully perturb InceptionTime. The main limitation encountered while working on adversarial attacks is the possible change of the underlying class.
Funding Open Access funding enabled and organized by  CAUL  and its Member Institutions. This work was funded by  ArtIC  project " Artificial Intelligence for Care"  (Grant  ANR-20-THIA-0006-01 ) and co-funded by  Région Grand Est ,  Inria Nancy -Grand Est ,  IHU of Strasbourg ,  University of Strasbourg  and the  University of Haute-Alsace.
16 The contrast between DistNums and unmarked indefinites is thus parallel to the contrast between NCIs and NPIs. The data are replicated in Romanian. In each of the (a) sentences below, the cardinal can either scope below or above the distributive quantifier in subject position. In the (b) sentences on the other hand, the distributive markers (câte in  [15]   Henderson (2014)  provides an important independent argument against  Farkas's (1997a)  and  Brasoveanu & Farkas's (2011)  attempts to subsume distributive numerals under the analysis of unmarked dependent indefinites: in Kaqchikel unmarked numerals cannot scope below a pluractional operator whereas distributive numerals can. [16] This does not mean, however, that the semantic dependencies characteristic of dependent unmarked indefinites and NPIs is the same: whereas dependent indefinites are not to be analyzed in scopal terms but rather in terms of dependent indices  (Farkas 1997a)  or Skolem functions  (Steedman 2012) , the constraint on NPIs is currently formulated in terms of scope (NPIs have to occur in the scope of downward entailing licensors).
While most of the general-purpose data-driven ML approaches take for granted the availability of good quality data and ground truth labels, in practice this is a major obstacle to the large-scale industrial deployment of AI. In this study, we focused on the automation of the end-to-end system including the data collection, preparation, labeling, and fault prediction components. Finally, to evaluate the accuracy of our failure prediction model, we used precision and recall, which are widely used measures in classification problems. Precision refers to the fraction of correctly predicted cases (true positives) among the predicted ones, while recall is the fraction of correctly predicted cases among all the failure cases. As already noted, we are dealing with imbalanced classes; the ratio between normal and faulty sites will still be significant even after the reduction procured by the layered approach.
Data This section presents the process of building a labeled dataset for training and evaluating the fault prediction algorithms. The data come from a large network and consists of network-wise alarm events, daily snapshots of sites, the list of deployed components on the site, and the list of components sent for repair. Building a labeled dataset from such heterogeneous inputs requires a data consolidation followed by a procedure for attribution of data samples to normal or faulty behaviors. Finally, we put in place a data summarization method allowing us to obtain reusable alarm patterns comparable across different components and sites. These three steps are described in the next subsections.
An alarm a is identified by a triplet < ResourceInstance, ResourceType, AlarmType >. The first term, ResourceInstance, identifies the logical entity on which the alarm was raised. Here, a logical entity refers to a physical or virtual network resource such as an antenna, a cell, or a fan. The second term, ResourceType, identifies the type of logical entity. For example, Cell-1 and Cell-2 are two resource instances of the same resource type Cell. Finally, the third term, AlarmType, encodes the meaning of the alarm, e. g. , high temperature, or connection loss. It is provided using numerical codes which are composed of 1 to 3 integers. Given the sensitive nature of the dataset, the examples in this paper will contain only the numerical codes of alarm types. For each alarm a i identified by the triplet < resInstance i , resType i , alarmType i >, we created a binary time series representing the sequence of its daily states.
Data Labeling In order to train the fault prediction algorithms, we needed a labeled dataset, i. e. , examples of alarm bundles that precede faults and examples of bundles not followed by a fault in the (known) future, i. e. , normal behavior bundles. To obtain examples of fault cases, we first selected the components in the repair center that had been diagnosed as featuring a hardware fault. This eliminates all the components where no hardware-related issues were detected despite the fact that the component was replaced and delivered to the repair center. Such cases can correspond to configuration errors or software-related issues that do not require intervention on the hardware. Other items could have been removed as a result of wrong problem detection and must not be labeled as fault examples.
Obviously, this operation reduces the original number of time series, In our dataset, we observed a reduction by a factor of 3 on average, see for example Figure  3  in which the left side gives an example of instance time series on one base station and the right side represents the subsequent summarized time series. This summarization mechanism can be extended to build graph-based summaries instead of aggregated data tables. This can be beneficial because the functional dependen-cies between resource instances can result in alarm propagation phenomena, which can be better captured by a graph structure. In fact, by analyzing the original raw data bundles one can discover some aspects of these dependency structures. Let us define an instance graph, as G = (N, V), where N = {n i =< resInstance i , resType i , alarmType i >, i = 1,.
If, within a time-window of t consecutive days we register at least n occurrences of such error being greater that the threshold τ, we declare the site faulty. Refer to Figure  4  for an example of such method. According to the formalism introduced in Section 3, this would be equivalent to the following formulation. Define the model's hyperparameters θ site = (θ RNN , n, τ, t), where θ RNN are the weights of the RNN. Define count θ site (C) as the number of times the error measure exceeds τ for n times during a time window of t consecutive days for site C. Now, the idea is that, if count θ site (C) > 0, then it is likely that C is faulty, in other words, that P(C is faulty|θ site ) > P(C is not faulty|θ site ). We use count θ site (C) > 0 as a proxy for P(C is faulty|θ site ) > 0. 5 + ω. In our dataset, we have 737 distinct summarized alarm identifiers in the form of < ResourceType, AlarmType > pairs, therefore we encode the inputs to the RNN as 737dimensional binary vectors.
As in the previous section, in our dataset we have 737 distinct summarized alarm identifiers < ResourceType, AlarmType >; therefore, we encoded the inputs as 737-dimensional binary vectors. The RSF takes as input the 737-dimensional binary vector for a given day for a given site which corresponds to a bundle of size 1 and outputs a risk score. In time-to-event analysis, we used the information on the event status and the time duration. We added this information to the data: the event indicator and the time duration. For the normal sites, we know that the site was normal up to the last reported time, therefore.
Note that, due to the thresholding on the dependency significance, each alarm bundle may be represented by a set of connected components (summarized sub-graphs): by abuse of notation, for a summarized graph B ∈ B we have a set of summarized sub-graphs B = {s i }, where each sub-graph {s i } can carry a signature of faulty or normal behavior. The graph matching method has two phases: knowledge model building (training) and faulty component discovery (inference). Due to the layered approach, the knowledge model will be built using only the data from the faulty sites; in fact, it needs to represent the knowledge necessary for distinguishing a faulty component from the others co-located on a faulty site. During the knowledge model building phase, a contribution score g(s i ) is assigned to each summarized sub-graph s i to express the likelihood of a component γ j being faulty given the sub-graph s i appears in the set of its summarized sub-graphs B j : g(s i ) = P(γ j is faulty|s i ∈ B j ) * c(s i ), (10) where the score g(s i ) is obtained by multiplying the probability g(s i ) = P(γ j is faulty|s i ∈ B j ) by a confidence coefficient c(s i ) ∈ [0, 1] that reflects the number of times the sub-graph s i was observed within the dataset.
The higher the number of observed s i , the higher the confidence towards its contributions. Thanks to this mechanism, we avoid rare sub-graphs having a significant (and arbitrary) impact on our fault detection decisions. In this study, we have used the following confidence function: c(s i ) = 1 1 + e -(#s i -d) , (11) where #s i is the number of occurrences of the sub-graph s i in the training dataset, and d is a sensitivity parameter such that for #s i = d the conditional probability term is attenuated (1 + e) times. After optimization of the d parameter, its value was fixed at 3 for our tests. Note that the conditional probabilities, P(γ j is faulty|s i ∈ B j ), can be statistically derived from the samples of faulty sites in the training dataset. The higher the frequency of a given sub-graph in the faulty samples (and the lower in the normal ones), the higher its contribution score will be.
Data Exploration Our dataset is composed of a total of 273 alarm logs, in the form of csv files. Such alarm logs are the result of a first calculation where we pre-process the raw data coming from the network logging system. Each alarm log refers to a site, and it records its alarms' activity during a given time window. For faulty sites, such a time window is set at 43 days; for normal sites, it ranges between 90 and 130 days but most of them are either 110 or 130 days. The time granularity at which alarms are recorded is daily. As already stated, each alarm is identified by the triplet: alarm type, resource type, and resource instance. For each identifier, we have a binary time series recording; for each day, whether this alarm was on (1) or off (0). Out of 273 logs, 73 refer to faulty sites.