dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgements F. L. and M. Z ˙. acknowledge financial support from the  European Research Council  ( Consolidator Grant COLLEXISM , grant agreement  811363 ), the  Institut Universitaire de France  and the  Programme National ''Physique et Chimie du Milieu Interstellaire'' (PCMI)  of  CNRS/INSU  with  INC/INP  co-funded by  CEA  and  CNES. We wish to acknowledge the support from the  CINES/GENCI  for awarding us access to the OCCIGEN supercomputer within the  A0090411036  project. M. Z ˙. acknowledges that calculations have been carried out at the  Wroclaw Centre for Networking and Supercomputing , Grant  218  (PSZ). M. Z ˙. thanks  B. Desrousseaux ,  C. Bop ,  S. Demes ,  A. Godard Palluet  and  P. Pirlot  for fruitful discussions. J. L. acknowledges support from  Internal Funds KU Leuven  through Grant  19-00313.
Location metonymy resolution is a study that deals with locations being used in a non-literal way that create problems in several natural language processing tasks such as Named entity recognition and Geographical parsing. Many studies were conducted attempting to accurately classify whether the location is used literally or metonymically, however, most of the approaches that performed well had to employ a considerable amount of resources along with complex machine learning models; those that reduced the resources experienced a decline in performance due to data sparseness. This study proposes a novel feature selection approach that uses bag-of-words and augments it with GloVe embeddings to obtain features that can be recognized based on the context of the sentence. We then implement a minimalist deep learning model making the entire classification task as light as possible.
Two of the major obstacles that occur when viewing metonytmy as a classification task are: Firstly: the gener-alization conundrum, where it is significantly complicated to categorize together the types of metonymy that follow approximately the same pattern (as in the case of locations and organizations) due to the endless possibilities of senses a word can take. Secondly, data sparseness: datasets for this task are relatively small compared to other NLP tasks, which causes a limitation to some extent during the feature extraction phase. Taking into account these limita-tions, researchers (particularly the participants of SemEval 2007 Shared Task 8) tend to exploit external resources such as WordNet, FrameNet, Multiple Parsers. to assist with constructing a heavily engineered features alongside a sophisticated machine learning model making the whole approach costly.
II. RELATED WORK A. DATASETS AND EVALUATION One of the well-known issues that mainly occurs when dealing with tasks that can be quantified is data invariance  [4]. Data invariance is assessing how the results can change when changing the dataset that is applied to a specific method; Metonymy Resolution is one of the NLP tasks that had the issue of non-standardized dataset which stood in the way of progressing, as a consequence, several studies tried to either construct a customized sample with a few sentences  [5] ,  [6]  or to use different corpora  [7] , which makes the results obtained by the different studies hardly comparable, thus, difficult to objectively follow the advancement of the state-of-the-art.
B. SMALL-SCALE METONYMY SYSTEMS Before the SemEval dataset was introduced in 2007, Metonymy resolution early work was largely based on violating the selectional preferences  [8] ,  [9]  ,  [10] ; yet by focusing on selectional preferences only, a considerable number of metonymic readings were skipped  [11]. On the other hand, Schematic Metonymy can be easily generalized to be a corpus-based work such as location metonymy and organization metonymy, allowing it to be suitable for using statistical approaches along with semantic understanding. The small-scale metonymy systems primarily focus on Logical Metonymy which doesn't necessarily comply with Schematic Metonymy, however, investigating the earlier approaches can yield a better understanding. One of the notable research in Logical Metonymy was done by  [7] , it highlighted the shortcomings of describing all the possible interpretations to logical metonymy and how the ranking of interpretations is not based on the likelihood of the metonymy, for those reasons  [7]  identified the sense of the metonymic words from a large corpus (British National Corpus) by extracting the collocation and co-occurrence then applying it on a probabilistic model to rank the interpretations accordingly.
Shutova and Teufel  [6]  , approached the issue of logical metonymy by presenting a statistical method, gathering col-location "synsets" and clustering verbs based on similarities, these sets of features help logical metonymy detection, however, one of the conceivable flaws  [4] , is that neither scale invariance nor dataset invariance was mentioned, their K-means clustering method was trained on five sentences and tested on five others, additionally, the sentences were created by the authors.
Nonetheless, recently, Gritta et al. [3], approach the problem from a different angle, they view that the SemEval 2007 dataset has unbalanced classes (79. 7% literal, 19. 3% metonymy, 1% othermet) which signifies that the bias is too high for the classification task, therefore, they create a new dataset "ReLocaR" with the help of linguists from Cam-bridge university that has (49% literal, 49% metonymy). Besides their newly created dataset, they develop a new approach to extract features that are around the PMW, unlike the immediate5 and immediate10, their approach uses a predicate window of 5 words "PreWin" as they believe that most of the sentence is irrelevant to the PMW, therefore, the extraction of unrelated words would have a negative effect on the trained model, after extracting the features using their novel method, they apply a deep learning approach using LSTM (long short term memory).
The results show that their method works well on the three datasets they use (SemEval 2007, ReLocaR, CoNLL) given that they implement a minimalist model compared to precedent approaches, where they achieve 83. 1% accuracy when using the SemEval 2007 dataset and 83. 6% accuracy when using ReLocaR. Due to either the insufficient coverage or the sparseness of SemEval and ReLocaR datasets, Mathews and Strube  [21] , present a new dataset for location names called WIMCOR (Wikipedia Metonymy Corpus), the samples were generated semi automatically using Wikipedia Disambiguation Pages a  (Auer et al. , 2007; Lehmann et al. , 2015)  which is a list of the different senses a one word can be used in, and offers links to articles that hold each meaning. One of the two main procedures to generate the dataset is by using metonymic pairs where the word is the same with two different concepts that are strongly connected, the intuition behind how to determine whether the concepts are highly linked is: "the more links referring to one another, the stronger the connection between the pair".
Furthermore, they use Wikipedia to extract samples and apply some constraints such as restricting the length of the sample in the range of 10 to 512 tokens. After the generation of WIMCOR dataset, the authors test the baseline models on it, however, their metrics of measurements are different from most of the studies, they use precision, recall, f1-score, whereas, accuracy is the primary evaluation metric in earlier research. Given that, most of the previous studies were released before 2019, BERT (Bidirectional Encoder Representations from Transformers) had not been released yet. Li et al. [22] , replicate Gritta et al. [3] , approach using BERT which achieved substantially higher results. PreWin using GloVe achieved 83. 1% and 83. 6% for SemEval and RelocaR re-spectively, PreWin using BERT scored 87.
Kalchbrenner et al. [24] , is among the earliest who relied on CNN for capturing relations and predicting sentiments with their Dynamic CNN (DCNN), additionally, it can be employable to all languages. The performance of their networks achieve high results on Twitter sentiments without the need of external resources (25% error reduction). Alternatively to the previous study, Kim  [25]  CNN model is less complex than DCNN where their approach uses a 1D convolution filter on the word vector that was created by one of the word embeddings algorithms such as GloVe. Kim  [25] , furthermore evaluates several approaches of word embeddings (CNN-Rand, CNN-static, CNN-non-static, CNN-multi-channel), these methods recorded to improve the state of the art in four tasks out of seven which establish that pre-training of word vectors prior to a deep learning model can significantly increase the performance.
Liu et al. [26]  , approached the problem of "Extreme Multi-Label Text Classification" (EMTC) as the first attempt that uses deep learning model, the CNN architecture that was proposed yielded notable results where it accomplished either the best or the second best across all datasets of the study. Johnson and Zhang  [27]  experiment applying CNN on high dimension text to leverage the 1D structure which led to learn embeddings of small text regions. Driven by the work of Simonyan and Zisserman  [28]  and He et al. [29]  , Conneau et al. [30]  , presented a "Very Deep -CNN" (VD-CNN) which perform convolutions and poolings on the character level, the study demonstrates the increase in performance in relation with the depth if the model. According to Duque et al. [31]  , the primary objective of most research in CNNs is to boost the accuracy by incrementing the networks' depth which resulted in creating a numerous amount parameters that negatively impacted the memory and processing during training.
Their proposed approach "Squeezed Very Deep CNN" (SVD-CNN) is designed to fit mobile platforms, it is 10 to 20 times smaller with a minor decrease in accuracy (0. 4% -1. 3%). Le et al. [32]  , investigated the impact of Deep CNN in text classification, their research shows that on the character level deep CNN out perform shallow and wide CNN, contrarily, they found that shallow and wide CNNs outperform deep CNNs when the sequence input is a word. Zhang et al. [33] , conducted a sensitivity analysis on the input passed to a CNN layer, their analysis found that one-hot vector performs poorly for a moderate to small datasets size unlike word2vec and GloVe embeddings.
III. APPROACH This section highlights the datasets used in the experiments alongside the different phases our approach consists of as shown in Fig. 1. 1-Starting with a preprocessing phase to the selected dataset in order to eliminate certain stopwords. 2-We then identify the most repeated words in each category to form the preliminary bag-of-words. 3-Using GloVe, we augment the previous BOW to contain more similar words to the initial ones. 4-We preserve the intersection of the Augmented BOW with the samples of the dataset. 5-We pass on the input to our deep learning model for training. Finally, we evaluate the results obtained using the suitable metrics of measurement. One of the obstacles that stand in the way of advancement in Location Metonymy Resolution is data sparseness, it was until 2003 when the first dataset was created by Markert and Nissim  [14]  then introduced by the same authors in the Semantic Evaluation (SemEval) 2007 conference as "SemeEval 2007 dataset".
The process executed for the dataset creation starts by extracting all country names from WordNet  [34]  and the CIA fact book (http://www. cia. gov/ci a/publications/factbook/) to form a list called "CountryList", following that, from the BNC (The British National Corpus info. ox. ac. uk/bnc) 1000 samples that include any of the countries from CountryList are taken, few of the samples are filtered out then the manual annotation begins, The PMW is categorized to either literal or metonymy, additionally, the authors divided the metonymic class into four subclasses (1-place-for-people, 2-place-for-product, 3-place-forevent 4-othermet). The reproducibility of the results has a percentage agreement of 0. 95, and a Kappa of 0. 88  [35]  [36], signifying that the corpus can be considered reliable.
2) ReLocaR Due to the shortcomings in SemEval 2007 dataset particularly the imbalanced classes and the different annotation scheme (where the political entities are considered as literal locations) according to  Gritta et al. [3]  , ReLocar was intro-duced with some improvements compared to the previous dataset. The main differences that ReLocaR comes with are: 1-the samples were extracted from Wikipedia, 2-both classes occupy 49% of the total number of samples, 3-the political entity is considered as a metonymic reading. Sim-ilar to SemEval dataset, ReLocaR was manually annotated by four trained linguists and two computational linguists from The University of Cambridge.
4) WIMCOR Unlike the earlier datasets, Mathews and Strube  [21]  , attempted to semi automate the process of generating a dataset for location metonymy resolution, using "Wikipedia Disambiguation Pages" the authors extracted the different meanings a word can have then categorizing each word with its corresponding article from DBpedia  [37]  ,  [38]. The dataset contains more than 1000 unique PMW, however, in contrast to the number of samples (206,000 samples) it is considered small, the average length of a sample is 80 tokens. The statistics of WIMCOR and the previously mentioned datasets can be observed in Table  1.
C. BAG-OF-WORDS AUGMENTATION Now that both bag-of-words are established, considering that GloVe word embeddings algorithm Pennington et al. [39] , group together the words' vectors that are similar or co-occur frequently as shown in Fig. 2  , we use it to obtain the nearest words' vectors to each word in BOW (as shown in figure  1 ) excluding those that do not require augmentation e. g. , for, by, with, in, etc). This added step is to include the words that may have the same significance as the ones obtained earlier (i. e. the most frequent words) but are not used too often in the dataset, in order to widen the range of context words for each category. augmented bag-of-words (A-BOW) have over 100 words for each category. Our feature selection approach is primarily designated to eliminate maximum noise regardless to whether the sentence's meaning is preserved or not, as the main objective is to determine the nature in which the location is used in.
Note that as a result of using GloVe for augmentation some words may not exist in the dataset, however, the feature selection starts by finding the intersection between the A-BOW and the dataset samples as shown in Table  2  , this process eliminates the possibility of having words out of the dataset's vocabulary. Prior to passing on the end results, we verify that the length of the intersection is at least two words (theoratically, one word can be used in any sense, therefore, it cannot identify the context), the samples that did not fulfill the condition are replaced by the immediate10 approach which means taking 10 words from the left and 10 words from the right of the PMW.
A. A-BOW VS BASELINE METHODS Table  3  shows that A-BOW method has substantially outperformed the baseline methods with at least +10% increase in accuracy across all datasets, given that both were trained on the same CNNs model. Note that the F1 score is significantly lower in SemEval than the other datasets due to the imbalanced classes of the mentioned dataset. The paragraph method achieved the lowest score in all datasets due to the amount of noise it contains, it prioritizes the length of the sequence over the relevance of the word, as a result, the number of irrelevant words are higher compared to the immediate methods. Unlike the paragraph, our method filters out the words that do not intersect between the A-BOW and the samples of the datasets i. e. segregating the context words that indicate the nature of the location mentioned from the rest of the words, the size of the sequence is not static as it depends on the number of words that intersect.
B. A-BOW VS PREVIOUS MODELS In comparison with the earlier Location Metonymy Resolution models, as it is shown in Fig. 3  , Our model has outperformed all of the previous models achieving the new SOTA on the SemEval dataset with an accuracy of 92. 3%. Note that GYDER and Nastase et al. [20]  , have achieved higher results(85,2% and 86. 2 respectively) than PreWin (83. 1%) due to their excessive resource usage, whereas, PreWin has substantially reduced the amount of resources consumed to extract features by solely relying on the datset and a dependency parser (to locate the head modifier of the sentence). Li et al. [22] , achieved 88. 2% using BERT along with a masking approach to exclude the PMW for being passed on as a feature in order to avoid a possible lexical memorization. Fig. 4  shows our model's performance in recent datasets, considering that, most of the Metonymy Resolution research was done before the release of these datasets (ReLocaR, CoNLL, WIMCOR), therefore, only two studies have used them so far, which are: Gritta et al.
[3] , and Li et al. [22]. Our model has achieved the highest accuracy across all three datasets 94. 2%, 96. 4%, 99. 2% in ReLocaR, CoNLL, WIMCOR respectively. Inspite of the fact that WIMCOR contains orders of magnitude more samples, it poorly transfers to either RreLocaR or SemEval with PreWin and BERT-LG-MASK; A-BOW was not hugely affected by the source dataset as the previous two, where it scored its highest accuracy in the sixth row, we hypothize that the reason of A-BOW being resistant to some degree is the nature of the method itself, where the bigger the bag-of-wrods is the more insights it carries.
D. DISCUSSION AND CONCLUSION In this paper, the main novelty is our feature selection approach. We combined the classic bag-of-words method with GloVe word embeddings. By exploiting one of GloVe properties (Similar words or words that frequently cooccur are represented close to each other), it allowed us to augment the initial BOW with words that may have the same significance of the most repeated ones in the corpus but not frequently mentioned, which resulted in widening the range of the words that can indicate the nature of the PMW. Since our feature selection algorithm depends on obtaining the intersection between the A-BOW and the dataset samples, the more words in the A-BOW, the more words intersecting with the samples, the clearer the class of the targeted location. Our approach was tested on four datasets (SemEval, ReLocar, CoNLL, WIMCOR), we trained our model on a minimalist CNNs.
The results show a remarkable improvement in the performance of Location, Metonymy Resolution classification task ,we achieved SOTA across all datasets that our approach was tested on. The model proved that it is not necessary to use a significant amount of external resources to boost the performance of the classification task (we only used GloVe and the dataset). It also showed that we can disregard the majority of the sentence and only preserve the words that can reflect the category of the PMW. Our method was inspired by PreWin method [3], as it changed the conventional way of looking at the task, they considerably reduced the external resources yet still achieved competitive results. However, their method has some drawbacks, the main idea of PreWin method is to find and extract the head modifier of the sentence and the four words that come after it, the head modifier is a good starting point to narrow the search for context words but indiscriminately taking the four following words can still result on carrying noise to the learning phase.
It was presented alongside ReLocaR [3] , Samples (locations only) from the CoNLL 2003 NER Shared Task data were annotated with a lack of quality labels, therefore, the dataset is more susceptible to noise. CoNLL dataset has a total of 7057 samples divided as 4609 literal and 2448 metonymy. Compared to SemEval 2007 and ReLocaR datasets, it is almost twice the size which can be advantageous during training.
The probability of finding the information can then be calculated as Sp = 1 -((a -h)/a) x (3) 2. When the MTD m is not of shuffle type and a method to bypass this MTD exists, the attacker's success probability is equal to the duration of the reconfiguration period of the node n divided by the time to bypass the MTD. If the duration of a period is greater, the attacker's success probability is equal to 1. 3. When the MTD m used is neither of type shuffle nor has a method to bypass, we will need to resort to an ad hoc method to estimate the success probability. Example 1. Consider an attacker trying to find the IP address of a module on an IPV4 sub-network. On this particular sub-network, 252 addresses are valid and usable by a module. In order to estimate the time an attacker needs to scan the network, we considered a well-known open-source tool for scanning networks: nmap.
In fact, nmap can be used to discover hosts and services on a network by sending packets and analyzing the responses. It can also provide further information on targets (e. g. , reverse DNS names, device types, MAC addresses, etc. ). Using nmap with highly optimized options, the time needed to scan one IP address is approximately 255 ms  [30]. Considering that the defender can change the IP address of the module sought by the attacker every 10 s, the attacker will have 10/0. 255 = 39. 21 attempts to find the correct address. Assuming that the module is the only element present on the network, according to Formula (3), the attacker's probability of success in discovering the IP address of the module is 39  (4) Sp = 1 -((a -h)/a) x Sp = 1 -((252 -1)/252) Sp = 1 -0. 856355413 = 0. 143644587.
Automotive Use Case In order to evaluate the proposed method, we set up an experimental use case, which is based on the connected car architecture presented in Section 3. Figure  3  shows a selection of ten nodes, for each of which we have selected two or three different MTD defenses. On the attacker side, there are ten attacker profiles, corresponding to two types (integrity and privacy) and five levels of expertise. We implemented a software tool that generates the MILP problem from the given input parameters and solves it using the CPLEX solver  [32]. We obtained the optimal defense strategy shown as below equation, in which the defender will defend each node and asset targeted by an attacker by spending his/her full budget per node. As can be seen, for some nodes, one of the defenses dominates (i.
Core Services Management δ n0 m 0 0. 351 δ n0 m 1 = 0. 0 δ n0 m 2 = 0. 649 δ n0idl = 0. 0 δ n1 m 0 = 0. 95 δ n1 m 1 = 0. 05 δ n1idl = 0. 0 δ n2 m 0 = 0. 0 δ n2 m 1 = 1. 0 δ n2idl = 0. 0 δ n3 m 0 = 1. 0 δ n3 m 1 = 0. 0 δ n3idl = 0. 0 δ n4 m 0 = 0. 404 δ n4 m 1 = 0. 311 δ n4 m 2 = 0. 285 δ n4idl = 0. 0 δ n5 m 0 = 0. 0 δ n5 m 1 = 1. 0 δ n5idl = 0. 0 δ n6 m 0 = 0. 0 δ n6 m 1 = 1. 0 δ n6idl = 0. 0 δ n7 m 0 = 0. 078 δ n7 m 1 = 0. 922 δ n7idl = 0. 0 δ n8 m 0 = 0. 0 δ n8 m 1 = 1. 0 δ n8idl = 0. 0 δ n9 m 0 = 0. 519 δ n9 m 1 = 0. 481 δ n9idl = 0. 0 7. 2. Scalability 7. 2. 1. Random Scenarios In order to investigate if the proposed solution scales well, we generated random scenarios of different sizes (the generating tool we made for the experimentation is available here: https://gitlab. telecom-paris.