dataesr/scientific-paragraphs-categorization huggingface dataset:
Pre-processing the MNIST database In order to respect the constraint of having inputs in {-1, 1}, we binarized all the images with a threshold value equal to 128: any pixel whose value is smaller than the threshold is mapped to -1; the others are mapped to +1. This actually reduces the amount of information available, as each 8-bit grayscale value is clamped to a single bit, and one could wonder if this could impact the accuracy of the classification. Although this is possible, a quick visual inspection of the result shows that the digits depicted in the images are still clearly recognizable.
In order to build a DiNN, we use the simple approach described in subsection 3. 2: we (1) train a traditional neural network (i. e. , with real weights and biases), and then we (2) discretize all the values by applying the function in Equation 3. 2. For step (1) we take advantage of the library keras [C + 15] with Tensorflow[AAB + 15], which offers a simple and highly customizable framework for defining, training and evaluating even complex models of neural networks. Through a farly simple Python script and in little time, we are able to define and train our models as desired. Given its similarity with (a scaled and shifted version of) the sign function, as an activation function we used the version of hard sigmoid defined in Tensorflow. The reason behind this choice is that we know we will substitute this activation function with the true sign (x).
Classifying encrypted inputs Implementing the homomorphic evaluation of the neural network over encrypted input was more than a mere coding exercise, but allowed us to discover several interesting properties of our DiNNs. The starting point was the TFHE library by Chillotti et al. , which is freely available on GitHub [CGGI16a] and which was used to efficiently perform the bootstrapping operation. The library takes advantage of FFT processors for fast polynomial multiplication and, although not parallelized, achieves excellent timing results. We extended the code to apply this fast bootstrapping procedure to our use case. Parameters. We now present our setting of the parameters, following the notation of  [CGGI16b] , to which we refer the reader for extra details. In Table  4  we highlight the main security parameters regarding our ciphertexts, together with an estimate of the security level that this setting achieves.
Acknowledgments. Florian Bourse  was supported by the  European Research Council  under the European Community's  Seventh Framework Programme  (FP7/2007-2013 Grant Agreement no. 339563 -CryptoCloud ), and by the  French ANR  Project  ANR-16-CE39-0014   PERSOCLOUD. Part of this work was done while the author was employed by  CNRS  and visiting CryptoExperts. Michele Minelli  and  Matthias Minihold  were supported by  European Union 's  Horizon 2020 research and innovation programme  under grant agreement No  H2020-MSCA-ITN-2014-643161 ECRYPT-NET. This work was done while the.
Acknowledgements This work has been funded by the  ANR-12-BS02-007-01 TARMAC grant , the  STICAmSud  project  16STIC05   FoQCoSS  and partially supported by the  Spanish Ministerio de Educación e Innovación, MICIN-FEDER  project  FPA2014-54459-P, SEV-2014-0398  and  Generalitat Valenciana  grant  GVPROMETEOII2014-087.
The proof is deferred to the Appendix. Despite its apparent complexity, this criterion takes a favourable form, since it writes as a function of GP quantities at step n (m n , s n and k n ), which can be computed very quickly once the model is established. Besides, it does not require conditional simulations (as the criterion in  [12] ), which is a decisive advantage both in terms of computational cost and evaluation precision. Let us stress here, however, that evaluating this criterion requires a substantial computational effort, as it takes the form of an integral over X, which must be done numerically. An obvious choice here is to use the set X MC as integration points. Also, it relies on the bivariate Gaussian CDF, which also must be computed numerically. Very efficient programs can be found, such as the R package pbivnorm  [13] , which makes this task relatively inexpensive.
Two-dimensional example As an illustrating example, we use here the classical Branin test function (see  [7]  Equation 19 in Appendix). On [0, 1] 2 , the range of this function is approximately [0, 305]. We take: X 1 , X 2 ∼ U[0, 1], and search for the 85% percentile. The initial set of experiments consists of seven observations generated using Latin Hypercube Sampling (LHS), and 11 observations are added sequentially using both SUR strategies. The GP models learning, prediction and update is performed using the R package DiceKriging  [19]. The covariance is chosen as Matérn 3/2 and the mean as a linear trend. For X MC , we used a 1000-point uniform sample on [0, 1] 2. For simplicity purpose, the search of x n+1 is performed on X MC , although a continuous optimizer algorithm could have been used here.
I. INTRODUCTION U NMANNED aerial vehicles (UAVs) have initially been utilized in military applications to engage in air-toground combats, surveillance, and target tracking in hostile environments. Surveillance primarily concerns collection, analysis, and management of critical information in critical sites (airport area, nuclear site, etc. ). Tracking is the operation of following mobile targets (suspected persons or vehicles) and monitoring their behaviors. Nowadays, UAVs are also used in civil applications to explore inaccessible zones (e. g. , disaster areas) and deliver data to and from areas with no network infrastructure (3G, 4G, etc. )  [1]. An UAV network is a wireless ad-hoc network that facilitates UAV-to-UAV and/or UAV-to-ground communications in order to deliver vital information for environmental monitoring, emergency, rescue and recovery operations, and disaster assistance.
Within this network, it is impossible to have persistent connections between UAVs or between UAV and ground station unless we put drones and ground stations everywhere across the field, which will, however, incur insurmountable cost as well as safety concerns (the ground stations cannot be deployed everywhere in a disaster area, e. g. , the middle of a fire, tsunamis). In time-sensitive applications, a DTN based on "store-carry-andforward" mechanism  [2]  is used to decrease the packets loss in a hop-by-hop manner. When neither UAVs nor ground stations are available, data are buffered in the UAV until a neighbor becomes available. The "store-carry-and-forward" mechanism is also called opportunistic data forwarding  [14] , where nodes store, carry, and forward packets to target nodes. A number of DTN mechanisms have been proposed to address communications disconnections and improve the network delay  [14] ,  [15].
g. , in the middle of a fire, tsunamis, or volcano even if (2) holds LIV = v i ∈ V D i < TR. (2) 2) Data Forwarding: When the UAV detects an event (e. g. , critical disaster), it forwards a message that contains the event type to the final destination (i. e. , control station). This forwarding is done hop by hop using a greedy forwarding mechanism  [18] -  [20]. In the latter, the next-hop (UAV or ground station) is selected as the farthest one from the sender  [20]. In case neither other UAVs nor ground stations are available, the UAV stores and carries the message for a period of Ts. In case when Ts has elapsed and no nexthop is available, the message is removed from the UAV's buffer. The packet forwarding in UAV network is illustrated in Algorithm 1.
B. Common UAVs Cyber-Attacks Owing to the nature of wireless medium and relevant data handled by UAVs, securing UAV networks becomes vital to protect them against lethal cyber-attacks. In this paper, we focus on mitigating two types of cyber-attacks: 1) integrity and 2) denial of service (DoS) attacks. 1) Integrity Attacks: Such attacks aim to fabricate false information, i. e. , altering GPS coordinates or disseminating false information. a) GPS spoofing attacks: The GPS receiver of a UAV can be spoofed by an attacker, thus leading to a false estimate of the drone position. For instance, the spoofed signal can lead to the situation where the receiver estimates the UAV's position, which is different from the real position. It has been claimed that the capture of a military drone aircraft by the Iranian military in December 2011 was the result of such attack  [11].
Such attack is simple to implement and inexpensive  [22]. Furthermore, according to several research works  [21] ,  [23] , it is difficult to detect such attack since the spoofer triggers no alarms on the ship's navigation component. b) False information dissemination attacks: A malicious UAV could broadcast a different physical phenomenon such as environmental conditions or forest fires to its neighbors. Such attack is defined as a false data injection attack. The ADS-B attack  [10]  is another kind of attacks that aims to disseminate false information. ADS-B is an on-board component of the UAV system that broadcasts information such as position and collision avoidance  [24]. According to  [10]  and  [24] , an ADS-B attack either aims to broadcast a false position or spoof the GPS coordinates (i.
D. Intrusion Response Mechanism The response mechanism is embedded in the ground station to evaluate the UAV's behavior and categorize each UAV according to its perceived threat into the appropriate list. UAV categorization into a well or bad behaved node and permanent exclusion of a malicious UAV can only be done by a trust entity (i. e. , ground station) to decrease the false positives and negatives. Thereby, as mentioned in Section III-A, each UDA broadcasts to the ground station located within its radio range an Intrusion Report message, which includes the suspected UAV's information. The ground station stores into its database the id of this suspected UAV, the ids of UDAs that detect the suspected UAV as an intruder and also the ids of UDAs (neighbor of suspected UAV) that do not detect it as an intruder.
The ground station stores the TL of each UAV that passes within its range in the Trust_database as follows: (UAV u i , TL i ); afterward, this information is exchanged among all ground stations within the network. At the end, each ground station computes a Total TL as shown in  (8). As explained in Section II-A, the ground stations are connected among them through a secure digital professional mobile radio standard like TETRAPOL  [16]  Total_TL i = d k=1 TL ik d ( 8 ) where d is the number of ground stations that compute the TL of UAV u i. Note that when Total_TL i < 0, the monitored UAV is considered as a malicious node. The frequent computation of the node's TL is not desirable since it is affected by noise and packets collisions  [12]. Thereby, to get an accurate TL value, we adopt the beta distribution  [41].
V. RELATED WORKS Intrusion detection is essential to protect the network against attackers since it can potentially detect the cyberattacks with a high detection and low false positive rates  [5] ,  [12] ,  [48] -  [51] , while authentication only aims to prevent the external attacker from entering the network. UAVs are attractive and easy targets for the attackers due to the relevant information handled by UAVs. However, the protection of such network has not been well investigated to the best of our knowledge; the intrusion detection scheme called BRUIDS proposed by Mitchell and Chen  [12]  is the only publicly available work that relies on detection techniques to protect the UAV network against cyber-attacks. BRUIDS aims to detect the attacks that target the integrity such as false data injection and attacks that target the availability such as malicious UAV that directs its weapon against a friendly resource, jamming, bad-and good-mouthing.
In this detection framework, the authors proposed a set of rules related to the attacks to model a normal UAV behavior. According to their simulation results, their detection system exhibits a low false negative. However, the false positive is higher (equal to 7%). In addition, this framework incurs a high overhead when the number of UAVs is large, and is thus not scalable. Kim et al. [24]  evaluated the behavior of attackers that target UAV nodes such as intruders that access to the autopilot components and intruders that inject false data into UAV on-board sensors. According to these malicious anomalies, they proposed certain detection rules to model a normal UAV behavior. They did not evaluate the performances of their approach in terms of detection accuracy and overhead. Strohmeier et al.
C. Computation Overhead Analysis The scheme is implemented in TinyOS using TinyECC Library  [15]  with the standard curve SECP160R1. For HKDF and signatures, we use the HMAC provided in the library that considers SHA-1 as hash function. We consider MicaZ sensor platform for our results. The curve point generation takes 1. 31s. For encryption, we consider 20 bytes the size of the modulus. Each device needs 0. 07s to encrypt and sign the data and the homomorphic aggregation performed by aggregator nodes takes about 0. 002s. The IDSs that are activated at node and CHj, rely on a lightweight detection technique to identify the misbehavior of resource-constrained devices. This technique is based on a rule based detection approach. So, the computation overhead at devices is negligible. Our cyber security game is launched at a powerful node, BS, since it requires a certain computation overhead to detect with a high accuracy the malicious attacks at node and CH levels.
Yogatama and Smith (2014b)  introduced a group lasso variant to utilize groups of 1 github. com/y3nk0/OMP-for-Text-Classification Best feature (word): j (k) = arg max j / ∈I X ⊤ j r (k-1) Update active set: I = I ∪ {j (k) } r (0) = y I = {} j (k).
Regularization By only minimizing the empirical risk, a model can be led to severe overfitting in the case where the number of the features (dictionary) is much higher than the number of the instances (documents) in training set. In practice, it yields models with poor generalization capabilities (i. e. , lower performances on the test set) that fit the noise contained in the training dataset instead of learning the underlying pattern we are trying to capture. Additionally, if two hypothesis lead to similar low empirical risks, one should select the "simpler" model for better generalization capabilities. Interpretation The concept of regularization encompasses all these ideas. It can be interpreted as taking into account the model complexity by discouraging feature weights from reaching large values; incorporating prior knowledge to help the learning by making prior assumptions on the feature weights and their distribution; and helping compensate ill-posed conditions.
Datasets Sentiment analysis. The sentiment analysis datasets we examined include movie reviews  (Pang and Lee, 2004; Zaidan and Eisner, 2008)   3  , floor speeches by U. S. Congressmen deciding "yea"/"nay" votes on the bill under discussion  (Thomas et al. , 2006)  3 and product reviews from Amazon (Blitzer et al. , 2007) 4. Table  1  summarizes statistics about the aforementioned datasets used in our experiments. We choose small datasets intentionally, like  Yogatama and Smith (2014b) , so that we can observe the regularization effect clearly.
Experimental setup In our setup, as features we use unigram frequency concatenated with an additional bias term. We reproduce standard regularizers like lasso, ridge, elastic and state-of-the-art structured regularizers like sentence, LDA, GoW and w2v groups  (Skianis et al. , 2016)  as baselines and compare them with the proposed OMP and GOMP. We used pre-trained Google vectors introduced by  Mikolov et al. (2013)  and apply k-means clustering  (Lloyd, 1982)  algorithm with maximum 2000 clusters. For each word belonging to a cluster, we also keep the top 5 nearest words so that we introduce overlapping groups. For the learning part we used Matlab and specifically code provided by  Schmidt et al. (2007)   and 20% for validation. All the hyperparameters are tuned on the development dataset, using accuracy for evaluation.
Results Table  2  reports the results of our experiments on the aforementioned datasets. The empirical results reveal the advantages of using OMP or GOMP for regularization in the text categorization task. The OMP regularizer performs systematically better than the baseline ones. More specifically, OMP outperforms the lasso, ridge and elastic net regularizers in all datasets, as regards to the accuracy. At the same time, the performance of OMP is quite close or even better to that of structured regularizers. Actually, in the case of electronics data, the model produced by OMP is the one with the highest accuracy. On the other hand, the proposed overlapping GOMP regularizer outperforms all the other regularizers in 3 out of 10 datasets. Another important observation is how GOMP performs with different types of groups.
As it becomes apparent, both OMP and GOMP yield super-sparse models, with good generalization capabilities. More specifically, OMP produces sparse spaces similar to lasso, while GOMP keeps a significantly lower number of features compared to the other structured regularizers. In group regularization, GOMP achieves both best accuracy and sparsity in two datasets (vote & books), while group lasso only in one (sports). In Table  4  we demonstrate the ability of OMP to produce more discriminative features compared to lasso by showing the largest weights and their respective term. Finally, in Table  5  we compare state-ofthe-art group lasso classifiers with deep learning architectures  (Kim, 2014)  with Dropout  (Srivastava et al. , 2014)  for regularization and FastText  (Joulin et al. , 2017).
Sparsity vs Accuracy Figure  3  visualizes the accuracy vs. sparsity for all datasets and all classifiers. We do that in order to identify the best models, by both metrics. The desirable is for classifiers to belong in the top right corner, offering high accuracy and high sparsity at the same time. We observe that OMP and GOMP tend to belong in the right parts of the plot, having very high sparsity, often comparable to the aggressive lasso, even when they do not achieve the best accuracies.
Number of active features (atoms) In both OMP and GOMP algorithms, the maximum desired number of active features (K, budget) was used as stopping criterion. For instance, by setting K = 1000, the proposed methods return the learned values that correspond to the first {100, 200,. , 1000} features, respectively. Thus, we exploit the feedforward feature selection structures of OMP and GOMP. Figure  4  presents the number of active features versus accuracy in the development subsets of the 20NG dataset. It can be easily observed that after selecting 1000 active atoms, the accuracy stabilizes or even drops (overfitting problem). For instance, the best number of active features are: i) science: 700, ii) sports: 1100, iii) religion: 400 and iv) computer: 1500. The reason for selecting K = 2000 as the number of features to examine was to provide a sufficient number for OMP to reach a good accuracy while providing a supersparse solution comparable to lasso.
Time complexity Although certain types of group lasso regularizers perform well, they require a notable amount of time in the learning process. OMP offers fast learning time, given the hyperparameter values and the number of atoms. For example, on the computer subset of the 20NG dataset, learning models with the best hyperparameter value(s) for lasso, ridge, and elastic net took 7, 1. 4, and 0. 8 seconds, respectively, on a 4core 3. 00GHz CPU. On the other hand, OMP requires only 4 seconds for training, making it even faster than lasso, while providing a sparser model. GOMP can have very slow learning time when adding the features as groups individually. This is due to the large number of groups that GOMP needs to explore in order to extract the most "con-  tributing" ones. If we consider GOMP without the individual features as groups, then the learning process becomes faster, with a clear decreasing effect on accuracy.
Conclusion & Future Work In this work, we introduced OMP and GOMP algorithms on the text classification task. An extension of the standard GOMP algorithm was also proposed, which is able to handle overlapping groups. The main advantages of both OMP and GOMP compared to other regularization schemes are their simplicity (greedy feedforward feature selection) and ability to produce accurate models with good generalization capabilities. We have shown that the proposed classifiers outperform standard baselines, as well as state-of-art structured regularizers in various datasets. Similar to  Mosci et al. (2010); Yen et al. (2017) ;  Xie et al. (2017) , our empirical analysis validates that regularization remains a highly important topic, especially for deep learning models  (Roth et al. , 2017).
885* 0. 911* 0. 902* vote 0. 606 0. 643 0. 616 0. 622 0. 684* 0. 658 0. 653 0. 656 0. 640 0. 651 0. 687* Sentiment movie books dvd electr. 0. 865 0. 860 0. 870 0. 875 0. 860* 0. 900 0. 895 0. 895 0. 895 0. 890 0. 750 0. 770 0. 760 0. 780 0. 800 0. 790 0. 795 0. 785 0. 790 0. 800 0. 765 0. 735 0. 770 0. 760 0. 785 0. 800 0. 805* 0. 785 0. 795* 0. 795* 0. 820* 0. 850 0. 805* 0. 790 0. 800 0. 800 0. 825 0. 830 0. 800 0. 815 0. 805 0. 820 0. 815 0. 800 kitch. 0. 760 0. 800 0. 775 0. 800 0. 825 0. 845 0. 860* 0. 855 0. 840 0. 855* 0. 830 Dataset no reg lasso ridge elastic OMP Group lasso regularizers LDA LSI sen GoW w2v GOMP science 100 1 100 63 2. 7 19 20 86 19 21 5. 8 20NG sports religion 100 100 1 1. 1 100 100 5 3 1. 8 1. 5 60 94 11 31 6. 4 99 55 10 44 85 7. 7 1. 5 computer 100 1. 6 100 7 0.
Context In historical sciences, cartography and spatial analysis are extensively used to uncover the spatial patterns at play within textual historical data. This data contains indirect textual references about location, such as place names (toponyms) or postal addresses. In order to map such data, each item needs to be geocoded, i. e. assigned with coordinates through the matching of an indirect spatial reference with entities identified in a geographical data source (e. g. a map georeferenced in a well-known coordinate reference system)  [1]. Problems emerge when such spatial references become obsolete due to the temporal gap between the data to be geocoded and the reference datasource: used as the reference dataset for these geocoders. These databases are extremely structured (hierarchy, normalization) and of high quality.
Temporal depth Common geocoding approaches cannot be used for (geo)historical data for three main reasons. To begin with, existing geocoding services do not take the temporal aspect of the query, or the dataset they rely on, into account. Indeed, they usually rely on current data, such as OpenStreetMap 5  data, which is continuously updated. As such, they implicitly work on a valid time that is the present (or possibly the interval between the beginning of the database construction and present time). The second reason is that they rely on an exhaustive, strongly hierarchical database whose accuracy can be checked against ground truth (i. e. there is always a way to check the actual location of an address, the database can therefore constitute an unambiguous and objective reference). Unfortunately, historical data cannot easily be verified: one has to compare it with different available (geo)historical sources (possibly incomplete and conflicting) and must often make assumptions or hypotheses.