dataesr/scientific-paragraphs-categorization huggingface dataset:
This is done by clustering the subsequences and choosing the centroid of the cluster that represents the normality depending on criteria of frequency and coverage. The abnormal subsequences are then detected based on the distance to the centroid. There are some issues with the clustering methods; they consider the clusters to be spherical (  [9] ) and it was shown that clustering overlapping subsequences is meaningless and always produces sinusoidal centroids independently of the dataset and the clustering algorithm ([10]).
Deep features Instead of manually selecting the appropriate features, recent works make use of deep neural networks for automatic feature extraction. They are known in the literature as deep features. [23]  used the an Auto-encoder to extract features from the latent space and employed OC-SVM to detect abnormal time series. Some other works as in  [24]  replace the kernel function of the Support Vector Data Description (SVDD) by a convolutional neural network. In this setting, the loss functions Of the SVDD and the reconstruction function of the autoencoder are simultaneously optimized, which forces the features learned by the Auto-encoder to be in a minimum volume hypersphere. In contrast to classical machine learning algorithms, deep learning methods generally require large datasets for training.
9 (b). Thus, in all the problems in this works, the train dataset consists in non-overlapping subsequences. We employ the RBF kernel with the smoothing parameter γ fixed at 1 m where m is the length of the subsequences following the wellknown heuristic 1 number of features used in the classical SVMs. The ν parameter is kept at 10 -5 while the threshold τ on anomaly score of the OC-SVM model is estimated as follows: τ = F ( Ŝ) -3 × σ F ( Ŝ) (11) Where F ( Ŝ) is the mean of the anomaly scores obtained for the train subsequences and σ F ( Ŝ) refers to their standard deviation. At the test stage, we predict the anomaly score of the new subsequences using the trained model. If it is lower than τ , the subsequence is declared abnormal. We compare our approach to four other approaches. The first is similar to the one proposed by  [12]  where the mean of the distances to the k nearest neighbours is the anomaly score: NN(S, k, Ŝ) =  d(S, y ).
The decoder is composed of two transpose convolution layers with the same number of filters and kernel size. We train the autoencoder with 50 epochs and a batch size of 4. The threshold for all these approaches is estimated in the same manner as in Eq. 11 while adapting the sign of the 3 standard deviation depending on the methods. For a reliable assessment of the approaches, we use the k-folds technique with k=5; we decompose the normal dataset into 5 subsets. For each of the 5 iterations, the models are trained using 4 subsets of the normal data while the 5 th is combined with some abnormal subsequences to form the test dataset.
results We start by studying anomaly detection in the signal of the circular welding. To that end, we consider a dataset composed of 6000 and 200 abnormal subsequences of length 200. Table  1  gives the F-scores of the approaches for each of the 5 test subsets. We notice that the models have a relatively similar mean F-score. The maximum mean F-score is obtained by the OC-SVM model. We also notice that it is the fastest in terms of inference time. The second fastest method is the autoencoder. However, it has a much higher training time. The k-NN based methods do not need any training. Nevertheless they needed 6. 11 seconds for the prediction. Given that the number of the test subsequences is less than the number of subsequences of a single voltage signal, these methods might not be adequate for real time monitoring.
The minimum length of a shapelet is set at 3 while the maximum length is the length of the minimum time series in the dataset. Despite these improvements, the algorithm still had a worst case complexity of O( m4 ). This had given rise to further research works for speeding up the shapelet search and extending the concept. For instance  [38]  proposed the Fast shapelets that uses a discretization of the time series rather than the raw in order to speed up the shapelet search. [39]  introduced the Shapelet Transform that represents a time series into a vector v ∈ R k where each element is the minimum distance between the time series and the shapelet k. The goal was to overcome the shortcoming that the original Shapelet method allows only the decision tree classifier. With the new representation of the data, any classifier can be used.
e w = (w -w) • b is the bias sampled from a uniform distribution, b ∼ U (-1, 1) • d is the dilation sampled on an exponential scale d = 2 x , x ∼ U (0, A) where A = log 2 linput-1 l kernel -1. This insures that the kernel's length including dilation is at most equal to the length of the input time series. • Padding : for each random kernel, a decision is made at random if the padding will be used, if yes, zeroes are appended at the start and the end of the time series. • k The number of kernels. The higher the number the higher is the accuracy. The authors recommend to use k=10 000. • Stride is always equal to 1. Two features are extracted from each convolution: • ppv : The proportional of positive values : ppv =   To test ROCKET on our problem, we consider a dataset composed of 50 instances of each of the 6 defect classes.
To do so, we can use an epigraph reformulation to transform it into a problem of the form  (1). Our work considers the optimistic case of bilevel optimization. This means that whenever there are multiple optimal solutions for the follower problem  (2) , the one which is best for the leader is chosen, see, e. g. ,  [39]. We note that already mixed-integer bilevel linear programming (MIBLP) is p 2 -hard  [37]. The value function reformulation (VFR) of the bilevel program  (1)  is min c x + d y (3a) s. t. M x + N y ≥ h (3b) M x + Ñ y -h ∈ K (3c) Ax + By ≥ f (3d) q(y) ≤ (x) (3e) y ∈ Y (3f) (x, y) ∈ Z n , ( 3g ) where the so-called value function (x) of the follower problem (x) := min {q(y) : y ∈ F(x)}  (4)  is typically non-convex and non-continuous. Note that in the optimistic bilevel setting, the VFR is equivalent to the original bilevel program  (1).
Instances In our computations, we consider two sets of instances: the quadratic bilevel covering problem (QBCov) instances (originally studied in  [22]  and extended here with multiple linking constraints) and a new additional set of quadratic bilevel multiple knapsack problem (QBMKP) instances derived from the SAC-94 library  [26]. The instances are available at https://msinnl. github. io/pages/instancescodes. html. All instances can be described as min ĉ x + d y (14a) s. t. M x + N y ≥ ĥ (14b) y ∈ arg min{y R y : Âx + B y ≥ f , y ∈ {0, 1} n 2 } (14c) x ∈ {0, 1} n 1 , ( 14d ) where ĉ ∈ R n 1 , d ∈ R n 2 , M ∈ R m 1 ×n 1 , N ∈ R m 1 ×n 2 , ĥ ∈ R m 1 , R = V V ∈ Z n 2 ×n 2 , Â ∈ Z m 2 ×n 1 , B ∈ Z m 2 ×n 2 , and f ∈ Z m 2. The QBCov instances. In this setting, we chose m 2 = 1, in which case the problem can be seen as the covering-version of the quadratic bilevel knapsack problem studied by Zenarosa et al.
The matrix V ∈ R n 2 ×n 2 has integer entries chosen uniformly at random from the set {0, 1,. , 9}. We extended this data set from  [22]  with 40 new instances generated in the same way, by choosing m 2 = 2. The QBMKP instances. These instances were derived from the multiple knapsack (MKP) instances from SAC-94 library  [26]  which is a benchmark library containing 0/1 MKP instances. From there we chose 50 instances and generated 300 new instances of the QBMKP as follows. The instances have 2 to 10 constraints and 10 to 105 items. For each instance of this data set, we first constructed two different QBMKP instances by keeping all but the last m 2 constraints at the leader problem, where m 2 ∈ {1, 2}. The first 50 or 75% of items are associated to leader variables x, the remaining ones are associated to follower variables y.
This is only magnified when there are multiple systems. Crucially, neurons operate to condense possibility and must have bias (intended or unintended) to function. 96 Thus, they must condense all the possibility, and in the case of deep RL, approximate a reward. To trace conf lict is not to identify points of activation but to incorporate contexts from big data, their biases, and how these may be used in determining an environment for the agent. If data is used from datasets outside previous spaces of conf lict, then conf lict becomes intertwined with these, where they become unknowingly implicated into the practice of conf lict itself. Tracing conf lict is not just about the isolation of neurons within deep RL -or other AI systems -away from the big data, environments, and ecologies we live in, but the way AI systems wrap up contexts and previous weights of human decisions and computational choices in complex hybrids.
2 ome older and less effective approaches used previously, include Naive Bayes, Hidden Markov Models (HMMs), and plain Recurrent Neural Networks (RNNs). The current approach to generate synthetic, human-like text, however, is to use Large Language Models (LLMs) that are carefully pre-trained on enormous datasets. Generative Pre-trained Transformer 3 (GPT-3) created by OpenAI is a deep learning network with a capacity of 175 billion parameters, ten times more capacity than the second largest system, Microsoft's Turing NLG. 3  The main power of the OpenAI model comes from careful pre-training. To compare the scale of this process, the entire Wikipedia is part of the data used to train this model but amounts to just 3% of the hundreds of billions of words used for training. GPT-3 is able to learn rapidly from a small set of training examples due to extensive pre-training and parameter size.
The subjects correctly distinguished GPT-3 generated texts approximately 52% of the time, which is not significantly higher than a random chance. Humans interact with computers, cell phones, and other electronic devices in numerous ways. To attract customers and make their experience both natural and pleasant, developers of cell phone applications or computer software try to improve human-computer interaction (HCI) by including natural interactions such as voice user interface (VUI) and enabling dialogs resembling conversation between people. Models such as GANs and LLMs are able to reduce the cognitive barriers between machines and people. An early example of reducing cognitive barriers was designing interfaces for querying large databases using natural language. In such solutions, the user formulates a query for the type of products they are looking for in an online store, using common speech.
The speech is interpreted into text and translated into formal query language. After the query is executed, a reverse process is performed. The result is first translated to a natural language sentence, which, in turn, is emitted as a voice response. A recent and advanced development of this kind is a product developed by Microsoft called Power Apps. It provides a low-code application development process in which you describe, in plain language, what you want to do and an application is quickly created. 4 ollowing our customer of the online shop example, GANs can be used to dynamically create images of the offered products in the environment where they are naturally used. For example, instead of showing a traditionally stand-alone image of some piece of fashion, it can be adjusted to customer's needs and presented in various contexts of a person wearing it, including matches for gender, height, weight, and skin colour.
It is important to highlight that although similar solutions are available without GANs they are limited because they are always custom-made and require significant involvement of human designers. As a result, they are labour intensive and expensive to create. Another use of GANs and LLMs focuses on their core capability, the ability to generate synthetic objects that may be considered a sample from a distribution generating genuine instances. Such an application is described where GANs are used to generate synthetic orders that resemble genuine ones for e-commerce platforms such as Amazon, Alibaba, or Flipkart. Such a tool allows retailers to explore the space of plausible orders, and can provide insights into future customer orders, preferences, or price sensitivity. 5 ince LLMs are trained on enormous and unlabelled datasets, they produce models that have bias against gender and marginalized groups that are present in the digested text.
10 he ingested text will likely include inf lammatory or hateful content. As a result, LLMs might generate not only biased text but also hateful and extremist content. 11 Unfortunately, it means that GPT-3 might be weaponized to create conspiracy, extremist, misleading or hateful content with little intervention by humans. 12 It has been suggested by several researchers that the datasets used to train these models should be carefully curated to reduce this possibility. 13 ince LLMs try to "memorize" the training datasets, it might be possible to extract sensitive information from the model. For instance, with queries carefully designed for this purpose, one study extracted private information including phone numbers, full names, addresses, social media accounts, and more. 14 It may also be possible to generate sensitive national security information through a process recognized by intelligence agencies as classification by compilation.
22 There are a few approaches for detecting misinformation. Unfortunately, it is difficult to judge which is the most effective as there is no standard benchmark dataset that might be used to make a fair comparison. 23 These approaches are often complex and are beyond the scope of this chapter. We will brief ly outline them here. The content-based approach uses content of the information rather than considering how that information is propagated. For example, a click baitlike format of a headline or sentiment polarity might be an effective discriminator. This approach has advantages as it can be used for early detection algorithms. 24 The social-response-based approach investigates how users react to a post. Responses such as "fake" or "untrue" indicate that users perceive misinformation. Emotional responses such as "unbelievable!" and political term such as "PC shit!" 25 also correlate with inaccurate information.
Acknowledgement The author is grateful to Dr Bastian Giegerich and Dr Joe Burton for their constructive comments on earlier drafts of this paper. Although no accepted definition of the term exists, "middle powers" includes those countries that exercise a high degree of economic sovereignty,  1  and strongly inf luence regional developments through their political, economic, cultural, or military capabilities. Middle powers may also inf luence rulemaking in specific domains of global governance. Notably, these countries tend to favor the status quo of the liberal international order, preferring to engage multilateral regimes and seek accommodations from them as needed, rather than challenging those regimes or the broader, hegemonic interests that underpin them. 2 "Middle powers" encompasses both developed and developing countries.
Given the costs and limited availability of high-skilled workers to train healthcare professionals, lawyers, and educators, among others, emerging markets may seek to replace them with AI-enabled 'bots. ' A review of national AI strategies suggests states may pursue one of two paths to promote innovation and adoption of AI-enabled services. Some countries may choose to "make public datasets available" for market players to develop proprietary AI-enabled services, and concurrently, set up regulatory sandboxes to pilot those products. 7 Other states may opt to leverage their Digital Public Infrastructure (DPI), i. e. , public technical standards and protocols that allow third parties to access personal data of citizens or anonymized data, with clearly defined guidelines on the nature of data that can be shared, the duration of data sharing, as well as permissible use-cases.
8 uch infrastructure, which can be considered "middleware," then becomes the conduit for access of data to train AI/ML models. The "middleware" model is likely to be preferred by many states as it allows for more granular, revocable, and regulated data-sharing, rather than one-time access to public databases. Middle powers in particular may condition the use by market players of such middleware on their affordable provision of AI-enabled services in sectors such as health and education. Examples of such middleware infrastructure developed by middle powers include protocols that standardize data-sharing (for example, Personal Health Records in Japan),  9  common and interoperable railroads for digital transactions (PayNow in Singapore, 10 Unified Payments Interface in I ndia, 11 etc. ), or unique digital identifiers, often validated by biometric markers (Aadhaar in India, 12 MOSIP in Philippines, 13 e-ID system in E stonia, 14 etc.
These reasons make it probable that public, middleware infrastructure will be used to develop AI-based applications and services across the world. For instance, it is worth highlighting how the COVID-19 pandemic has accelerated the adoption of AI-enabled diagnostics that rely on data from contact-tracing applications or chest imaging databases built by states. 15 This trend will likely continue in the future, especially in middle powers that have created centralized national databases faster than advanced economies. The development of AI-enabled products and services that rely on such middleware infrastructure also creates a dilemma for states. DPIs are heavily reliant on Application Programming Interfaces (APIs) that lay down technical specifications for data collection and sharing. In some cases, the middleware in question is nothing but an API, i.
e. , a few lines of code that allow public and private players to connect with each other and share data. Payment railroads are a common example of such APIs. In other instances, APIs are necessary to allow external applications to connect to the public infrastructure and retrieve data from its servers. With respect to AI-enabled services or products, APIs will be key to ensuring that ML models developed by third parties have access to the right data parameters in order to train their algorithms. Essential as they are to the adoption and scaling of digital infrastructure, APIs are also vulnerable to sophisticated cyber attacks. By their very nature, APIs are designed to facilitate the seamless integration of third-party applications with databases, cloud services, virtual networking functions, etc.
The emphasis on ease of access has often come at the cost of secure API design. 16 In the case of AI-enabled services that run on public datasets or middleware infrastructure, this vulnerability is compounded by the fact that there is minimal human supervision of the interaction between the service and the user. As a result, attacks on AI-enabled services through APIs could seriously impair both the availability of the service as well as its predictive accuracy and effectiveness. Attacks against the availability or integrity of AI-enabled services may thus have the effect of undermining public confidence and trust in them, especially in developing countries and middle powers. To be sure, APIs are potent vectors for cyber attacks wherever they are deployed. With API 'calls' comprising over 80% of the Internet's platform-led traffic, such vectors are omnipresent.
17 The role of APIs in linking DPI to AI-enabled services, however, makes them an even more lucrative target for state and non-state actors who seek not only entry into critical national databases, but also disrupt autonomous and minimally supervised platforms that provide essential services. This chapter highlights how AI-enabled services running on public, digital infrastructure could emerge as vectors of cyber conf lict. States face a difficult choice between opening up national databases or other public infrastructure to third parties in order to promote AI innovation atop its data and risking not only the security of those databases but also that of critical, even lifesaving, services. The "middleware dilemma" is most acute for middle powers, especially large emerging markets that are undergoing rapid industrialization but find themselves constrained by resources or capital to provide essential services at scale.
18 For the same reason, APIs have been key to the rapid expansion and proliferation of federated and centralized databases across the world. The DPI developed by middle powers such as Australia, Brazil, and India, to name a few countries, too depend on APIs for their implementation and use. Indeed, regulatory tools such as the 2018 Revised Payments Directive (PSD2) in the European Union and the 2020 Consumer Data Right Act in Australia require even private actors to provide standardized APIs so that user data is interoperable and seamlessly accessible by all authorized parties. Despite their popularity, however, API security still leaves much to be desired. APIs have earned notoriety in recent years as attack surfaces for data breaches, identity theft and account takeovers, ransomware injections, IoT exploitation and DDoS attacks, among others.
19 One estimate suggests API attacks will emerge as the leading vector of cyber attacks by 2022. The security considerations involving APIs are three-fold: first, the widespread use of APIs results in a crowding of digital networks and infrastructure by third parties, making it difficult to manage or monitor the proliferating endpoints. Indeed, network administrators have no 'over-the-horizon' visibility with respect to third-party applications or devices that are constantly pinging their databases or infrastructure with API calls. Second, APIs may be developed by actors across the network, but there are no clear frameworks for accountability and remedial action in case of API-enabled attacks. 20 The poor maintenance and updating of APIs has even contributed to the phenomenon of 'zombie' APIs that continue to be functional (and potentially leak data) although their developers have long abandoned their active use.
The state in question may want to share data with market players in a bid to boost private innovation. While APIs present a relatively easy and seamless way for many states to create middleware that collects data or retrieves it from existing databases, they must balance such convenience against the risk of losing highly sensitive information. In some instances, only states have the legal imprimatur to collect certain types or categories of data from citizens, and the possibility of leakage or unauthorized exposure of such data (for example, biometric or health data) to third parties through APIs is high. The concern that APIs may be vectors for cyber attacks and indeed, cyber conf lict, is compounded in the case of AI-enabled services. States may allow the use of APIs specifically to promote AI/ML innovation on public databases or infrastructure in a number of ways.
Governments could lay down specifications and protocols for retrieving data either from databases or directly from citizens. Once data is collected through such traditional, "dumb" APIs, it is left to the third party to anonymize the data and use it to train their proprietary ML models. Alternatively, states could provide "clean rooms" or closed environments where personal and population-level data is anonymized and training models built without actual transfer of data. Such a model relies on advancements in secure multi-party computation that allows for training of algorithms without having to share private data. 24 And finally, states could open up their infrastructure to third party Machine Learning APIs (ML APIs) that are used by start-ups, enterprises, and public agencies alike. Indeed, this third option may emerge as a popular one for many market players who do not themselves have the capacity or resources to train ML models, but have innovative AI-enabled services to offer.