dataesr/scientific-paragraphs-categorization huggingface dataset:
The major limitation of the BIOCASTER is its inability to detect disease outbreaks beyond the eight languages (Chinese, English, French, Japanese, Korean, Spanish, Thai, and Vietnamese) present in its ontology  (Doan et al. , 2019). Therefore, scaling the system to work across different languages requires manually updating the ontology with information for new languages. In addition, the system is not publicly available, except for the ontology. The EcoHealth Alliance Global Rapid developed the Identification Tool System (GRITS), an application that provides automatic analyses of epidemiological texts. The system extracts important information about a disease outbreak, such as the most likely disease, dates, and countries where the outbreak originates. The pipeline for GRITS entails transforming words to vectors using the term frequencyinverse document frequency (TF-IDF) method, by first extracting features using pattern-matching tools, before applying a binary relevance-based classifier to predict the presence of a disease name in the text  (Huff et al.
Dataset We extend the dataset proposed by  Mutuvi et al. (2020)  to include additional languages so that it covers news articles from several, diverse language families: Germanic: English (en), Hellenic: Greek (el), Romance: French (fr), Slavic: Russian and Polish, and Chinese that descends from the Sino-Tibetan family. The articles were obtained from different online news sources with articles relevant to disease outbreak being obtained mainly via the Program for Monitoring Emerging Disease (ProMED) 2 platform, which is a program from the International Society for Infectious Diseases that tracks infectious disease outbreaks and acute exposures to toxins, across the world. The process of gathering the data involved first, retrieval of ProMED news articles published between August 1, 2013, and August 31, 2019.
The articles clearly annotate the title, the description that captures details about the reported disease, location, date, and the source Uniform Resource Locator (URL) where the article was originally published. The source URLs were extracted and their corresponding source documents downloaded to form the relevant documents of the dataset. On the other hand, the irrelevant news articles consist of general health-related news, but without direct or indirect mentions of disease outbreaks (e. g. , plague, cholera, cough), as well as general news like politics and sports. Most of the irrelevant documents were obtained from the News Category Dataset  (Misra, 2018)    To simulate the real scenario of news reporting, we set the number of documents reporting disease outbreak (relevant documents) to be no more than 10% of the total dataset.
Language We split the data, with a total of 7, 574 articles, into training, validation, and testing sets. The training set comprises a total of 5, 074 documents, while the remaining documents were shared equally between the validation and the testing sets, that is, 1, 250 documents for validation, and 1, 250 documents for testing, stratified by language, as shown in Table  2. We also present the percent of relevant articles that refer to epidemiological news, which depicts the imbalanced nature of the dataset.
Models and Hyperparameters Baseline model: DANIEL As a baseline model, we chose DANIEL 4   (Lejeune et al. , 2015) , an unsupervised system that does not rely on any language-specific grammar analysis and considers text as a sequence of strings instead of words. Consequently, DANIEL can be easily adapted to operate on any language and extract crucial information early on, which can significantly improve the decision-making process. This is pivotal in epidemic surveillance since timeliness is key, and more than often, initial medical reports are in the vernacular language where patient zero appears  (Lejeune et al. , 2015). We did not evaluate the BIOCASTER because only the ontology is publicly available and covers a limited number of languages, while GRITS is targeted to mostly English text.
Machine Learning models We also investigate three commonly used text classification models as baselines, Logistic Regression (LR), Random Forest (RF), and Support Vector Machine (SVM), using default hyperparameters and the TF-IDF weighting measure 5. Deep Learning models Firstly, we consider two models, a CNN and a BiLSTM with FastText  (Joulin et al. , 2016)  word representations for all the languages in the dataset, with an embedding dimension of 300. For the CNN, a sequence of word embeddings is passed through a convolution of kernel size 3 and a filter size of 250. Similarly, the BiLSTM passes the word embeddings through a bi-directional LSTM with a cell size of 128. Other hyperparameters for the models are a batch size of 32, a learning rate of 1 × 10 -2 , and 15 epochs with early stopping of 3 to avoid overfitting.
Results Deep learning transfer approaches such as BERT have demonstrated the ability to outperform the stateof-the-art methods on larger datasets. However, when there exist only a few labeled examples per class, 100 to 1, 000 as is the case of the low-resourced languages present in the dataset used in this study, the choice of the most appropriate approach is less clear, with classical machine learning and deep transfer learning presenting plausible options. Results of the experiments for different machine learning and deep learning models, using the dataset splits indicated in Table  2  are presented in Table  3  and discussed below. Regarding the machine learning methods, we notice, from the results in Table  3 , that SVM outperforms by a small margin the LR and RF on precision and recall, while the RF has not only the highest precision (95.
In the case of deep transfer learning models, one can notice a great difference in the F1 score performance of BERT-based models, compared to all the other models. We can also observe that BERT-based models manage to balance recall and precision (precision remains consistent despite the increase in recall). The models benefit from the pre-trained language models that are either used as features or finetuned on the task. BERT relies on Byte Pair Encoding (BPE) based WordPiece tokenization  (Wu et al. , 2016)  which makes it more robust to handle out-of-vocabulary words. Regarding the difference between the fine-tuned BERT-based models and those that use the BERT encoder for generating features only, the performance is slightly better when BERT is fine-tuned on the task. However, in the case of additional layers on top of the BERT encoder, when fine-tuned, a considerable decrease in performance can be seen.
Models As observed in Table  4 , all the machine learning models (LR, RF, SVM) display similar trends in their unequal performance based on language by not detecting (having the F1 values of zero) the relevant documents in Polish and Chinese. This is likely due to the size of the training data for these particular languages. Similarly, for all the low-resource languages (Polish, Chinese, Russian, and Greek), unsurprisingly, the CNN and BiLSTM -based models with pre-trained FastText embeddings were not able to distinguish relevant documents from irrelevant ones, as indicated by their low F1 scores. This might be due to the low embedding coverage of the languages. The F1 values for Chinese tend to be consistent for all BERT-based models while the performance for Polish varies a lot between models.
VGCN+BERT had the highest F1 scores for the low-resourced languages Polish, Chinese, and Russian and the secondhighest for Greek. In order to analyze the influence of the documents with a larger quantity of documents (French and English, around 2, 000 news articles) over the classification of low-resource languages, we consider every language as the source language and the other five languages as target languages. At every iteration, the best performing model from the previous experiments is trained on the data in the source language and applied directly to every target language.
Effect of Training Data Size Different sizes of the training data were selected at an interval of ten percent and evaluated to ascertain the impact on the overall performance of the best model, in this case, the BERT (multilingualuncased) fine-tuned model. We observe that there is a generally positive trend for F1 score performance when trained on increasingly large datasets, as can be seen in Figure  1. When using only 10% of the data, the model achieves an F1 score performance that is comparable to that of the classical machine learning models and plateaus at 30% of the data. It is worth noting that the model achieves an F1 score of 64. 03 using 5% of the training data, which is a significant performance for such a minimal amount of data.
Discussion Out of all the models, deep learning BERT-based models were the best performing models, in terms of both F1 score and recall measures. The good performance can be attributed to the deep network architectures and large corpora used to train Transformer-based pre-trained language models (PMLs) such as BERT, which enable learning of rich text representations. Moreover, BERT fine-tuning performed better compared to the feature-based approaches, where FastText and BERT embeddings were used as input features to CNN and BiLSTM classifiers. Essentially, the PLMs end up learning universal language representations that are beneficial to downstream tasks. The high precision and low recall noted in the machine learning models suggest that the models are unable to detect the relevant class well but are highly reliable when they do.
With regard to the contribution of various document segments on performance, it was observed from the results that, the beginning and the end of the text combined had the highest recall and F1 score. This was particularly the case for models based on BERT namely, VGCN+BERT and BERT fine-tuned models. This can be explained by the fact that the beginning paragraphs in an article often capture the most important information, which informs the reader what the story is about. On the other hand, the last part of the article tends to provide a summary of the article. The performance of the model improved proportionately with training data size. This is in line with neural network models, which require large amounts of data to train and evaluate. The competitive performance even with a small amount of data results from the transfer of knowledge from the pretrained language model, trained on a large corpus, to the specific task of classifying epidemic text.
Smart borders, intelligent systems of filtering travelers by detecting suspects of crime and terrorism through interoperable national data bases and regional agreements are proliferating. In the European Union it began with the Schengen Information System. SIS-VIS-EURODAC, EES, ETIAS, ECRIS are acronyms for different realizations and projects of Entry and Exit Systems, of pre-frontiers zone, of policy checks regarding police, immigration and asylum, tracing people and scoring the degree to which they can be suspected to be illegal or criminal. Security stakes have been technologized. This is a profound reconfiguration of the different regional fields of security professionals with the emergence of a transnational guild regrouping data analysts, civil engineers on IT systems and border controls, changing de facto who is deciding the limits between security and insecurity, risk and fate.
I will study how these specialists of IT systems have become important actors in security-freedom dilemmas from the beginning of the Schengen agreements in the mid-eighties  (1985)  with the introduction of a Schengen Information System as an instrument for collaboration on external border controls and police collaboration. I will also explain its evolutions in the 1990s correlated with the enlargement to Eastern European Countries, and its reconfiguration as a series of search tools creating a function creep giving police access to data bases constructed initially only for the management of border controls or for the management of refugees inside the EU. The creation of multiple databases protected by purpose limitations in terms of access has been at the core of the functioning of the area of freedom, security and justice during the last twenty years, even if after 2004, under the pressure of the US administration some purpose limitations had been already removed.
But when registration has been transformed into logs searchable by police organizations into the SIS2, and when police succeed to have access to other data bases like Eurodac and VIS, the question of transformation of criminal police into intelligence and surveillance for prevention purposes had become crucial. Security was not seen any more as a protection of an area of freedom of movement but as a necessity to have a "continuity" of surveillance anticipating danger coming from abroad before they actualized (risk calculation of overstay, score of suspicion based on contacts with persons knowing themselves terrorists or criminals). In the second part, I analyze the key moments of this contemporary reconfiguration of the European field of power concerning the boundaries of freedom, security and justice, by looking at the different actors having a controversy around the efficiency of digital technologies and their impact on privacy, examining in detail what are the logics at stake in terms of interoperability at the EU level.
Only in this vision of the world, data analysts can control more and more the field of security, or so it seems. As I will demonstrate in the last part, cooperation on border control via digital data sharing is not anymore, a means for the goal of enhanced security to protect the area of freedom of movement of the EU as it was presented in the beginning of the Schengen agreements, but it has become a goal in itself, serving the interests of the ones developing the technologies associated with it. Importantly, this "objective", which is more the result of a field effect than a strategy by a specific dominant group, is nevertheless the drive that all professionals of security, including the "stake holders", consider crucial: i. e. to "govern" travelers globally, via data management and ironically beyond border controls  (Walters 2002)   (Amoore 2006).
The sociogenesis of the professional guild of digital technologies for security purposes and its agencification via EU-LISA To make sense of the contemporary struggles around the digital technologies in security matters, it is necessary to analyze what is contributing to or affecting the course of the social development of technologies of border controls, and not to rely in the argument that they are only the reaction, the products of the most recent threatening events. The idea of interoperability is engrained into the history of Schengen in the European Union, and especially on the practices of actors connected with the Schengen Information System. At the origins of the Schengen Information System and its competitors: building an IT system efficient and politically acceptable, using a multi-purpose data base managing crime and entry of foreigners.
The initial configuration of police data bases has been set up by Interpol in the midseventies, and many intelligence services were sharing information by fax, satellites, or encrypted phone technologies, but the routines of using information systems began with the project in the European Union (still at that time called European Community) to manage in common the borders of a series of member states (Benelux, France, Germany) by accepting freedom of movement for citizen and third country nationals living there, but to create "safeguards", "compensatory measures" by "reinforcing" together the control at the "external" borders with the countries non-members of the common area. (see article on the origins of Schengen in this issue). To connect the different police and immigration authorities it was decided to develop both police liaison officers, common police stations at the borders, and to dedicate policemen to learn informatics to share information through data bases on a bilateral or multilateral bases.
The Schengen Information System or SIS comes from the recruitment or contracting of a hybrid of persons in police and border guards having knowledge as data base skill users, and persons coming from high tech companies like Honeywell Bull, Siemens, Hewlett Packard or later on Steria (which became SOPRA-STERIA), who were IT specialists. The creation of the SIS tool was therefore also the creation of a specific field of professionals of (in)security management specialized in data management and IT security. This introduction of new players has, in the European Union, transformed the way policing and intelligence were done, and the way sovereignty and freedom were exercised. Beyond the juridical formulation of an area of freedom, security and justice, and its strong legal component, a specific logic of (in)securitization has prevailed, not only via discourse but via material design on border control technologies, visa and passports servers and biometric tools in various databases, but it was certainly not done in a monolithic way as we will see.
Second, the transformation of the SIS1 into a SIS2 via the transition of the SIS1-for-All at the period of the enlargement of Schengen to Eastern European countries is not a technical but a highly political move which has disrupted for a brief moment the trend toward preventative policing. However, SIS2 is de facto a new system which is not the follower of SIS1-for-All or SIS1, as it has developed beyond border controls checks, search tools giving the possibilities to run multiple searches and generating potential function creep in favor of police services regarding data bases on asylum, migration and borders. This is this logic which has been at the core of the discourse of the necessity of interoperability. Third, the impact of preventative arguments has justified in 2001-2004 and later on in 2015-2017 the use of large-scale IT systems based on data collection, retention and predictive profiling, putting at the very heart of the discussion on security, the digital technologies as "mediator" of the values of freedom, privacy, protection, security.
The beginning of the SIS Different researchers have considered from the very beginning of the Schengen Information System that the forms taken by policing foreigners and doing border controls were reframed by the technological modalities given priorities of the digitization of documents over face to face relations at the border. In relation with the intense debate about national sovereignty and fear of a function creep in favor of a neo-federalist view advantaging a European bureaucracy in the making, the computer scientists had to manage the degree of centralization of the management of information in a very political way (Georgakakis, Didier 2012). The vivid nationalist reactions obliged them to create a central system interconnecting the different national systems but containing no data in itself.
Tony Bunyan of Statewatch and many other NGOs have insisted from the very beginning about the problems arising from this type of computerization of data, in terms of error, of safety of the system itself, of data protection, of privacy regarding the individuals. ( Bunyan, Tony 1993)  But the message was not listened. The decoupling of state borders' logics of control from the territorial border initiated by the Schengen agreement in the name of freedom of movement of persons -tempered by the fear of the rise in crime related to this freedomexacerbated previous tensions concerning the narratives of national sovereignty and global insecurity. It destabilized the idea of what was internal and what was external to a national state inside the European Union. The internal security of a unique European space (or area) has been conceived as the coalescence of different national internal security spaces including, de facto, an external dimension for each national state that the limited effect of European citizenship has not succeeded in solving fully.
(Author 1996a;  Infantino et al. 2013; Author 2013)  The strong argument in favor of computerized information systems was to turn collaboration between different foreign services doing the same task into a routine. Collaboration needed to be an everyday task, and marginally, it was also for some countries a way to centralize the data on serious crime or terrorism in national bureau and not at the local level. Different Information Systems were discussed, on terrorism inside the Trevi group, which will give birth to Europol, on border controls, asylum, and visa (CIREA-CIREFI) groups in addition to the specific Schengen initiative. The European Information System, competitor of the SIS, collapsed, leaving the place to the private companies coming from a consortium led by Germany and France (Bull and Siemens).
The argument that the present system was a blue print for the next step, for the future (perfect) system became a mantra organizing all the discussions. Therefore, the efforts have been about maintaining and developing the system for future extensions in order to secure the full space of all the EU members, while not creating one space controlled by one centralized entity. Future oriented action has not been set up to forecast multiple futures, but to avoid at the present time a worst-case scenario imagined by the police. This trend reinforcing police powers and their willingness to pretend that they can be preventative if technologies are interoperable and data are searchable through predictive analytics has created this political move towards "interoperability". Far from being something new, post 2015 it was already the argument in order to extend the initial Schengen to all the other member states, and to accept Italy (who has many terrorist and organised crime mafia like data bases) despite the reticence of the 5 other different ministries of Interior among the initial members, and it was then the same argument to accept Spain, Greece, as well as the Nordic states which had already an area of freedom of movement between themselves (including non EU members: Norway and Iceland).
The accumulation of data in order to connect the dots of transnational crime has been the leit motiv of the permanent geographical extension as well as the functional one. This has, by the same token, isolated the UK, and only Ireland was obliged to join the UK to stay outside the Schengen area in the name of the struggle against IRA (a still dividing problem, as Ireland succeeded in other domains to have its autonomy, like with the euro). This disjunction between EU borders and "Schengenland" has nevertheless reinforced even more the fact that border controls national agencies needed to create a virtual system where they can be permanently in touch. The formalization and institutionalization of the role of data managers in the field of security professionals, the emergence of a new guild? The drive towards digitization suggested by the initial discussions but not really implemented beyond police cooperation in terms of major crime and terrorism, became central to manage borders in an "integrated" way.
Less visible than the coordination of the border guards via Frontex, the central move was to organize a virtual space managing the digital connections and all the IT apparatus necessary for the establishment of a common management, both technical and political (by avoiding centralization). Rapidly, the number of agencies by countries and the number of countries using the SIS for the purpose of border controls for TCN entering the external borders exploded. It was not anymore, an add-on, but became a central piece in the networks of data bases, especially for Germany, France and later on Italy, even if some countries like Portugal were continuing to prioritize for most of their actions Interpol links instead of the possibilities of the SIS. To convince national forces to use the SIS was a permanent fight obliging a strong lobbying.