dataesr/scientific-paragraphs-categorization huggingface dataset:
RESULTS In this section, two experiments showing the accuracy of our method are presented. The algorithm is implemented in Matlab and C++ and tested in a synthetic scene for the first experiment. Next, another experiment is produced from a real recording. In order to characterize the accuracy of our method we will consider the sum of the squared collinearity errors E k. Additionally, we adopt the following metrics in the space of rigid motions: Algorithm 1 Event-Based PnP algorithm Require: e k = (u T k , t k , p k ) T ∀k > 0 Ensure: R * , T * Initialize the parameters for every incoming event e k do Identify the point generating the event i(k) Compute the Line-of-Sight projection matrix L k using Equation (  5 ) if full method then Compute the resulting torque Ŵ k using Equation (  23 ) Compute A k and B k using Equations (  10 ) and (  11 ) else if efficient method then Update the resulting torque Ŵ k using Equation (  27 ) Update A k and B k using Equations (  13 ) and (  14 ) end if Compute the resulting rotation R k using Equation (  24 ) Compute the resulting displacement T k using Equation (  9 ) Update R * using Equation (  26 ) Update T * using Equation (  12 ) end for • The absolute estimation error in linear translation is given by the norm of the difference between the estimated translation T * and its true value T.
Real Recordings Next, our algorithm is tested on real data obtained from an ATIS sensor, where an object moves and rotates in front the camera. As an object, we use a white piece of paper in which we printed FIGURE 8 | (A) Evolution of the errors for four different values of λ r (in N -1 mm -1 ), when n = 20. After n events both errors decay toward zero, the convergence being faster for bigger values of λ r. (B) Evolution of the errors for four different values of n, with λ r = 0. 005 (in N -1 mm -1 ). After n events have elapsed, the behavior of the system is very similar for all values of n.
Computation Time The presented experiments were carried out using a conventional laptop equipped with an Intel Core i7 processor and running Debian Linux. The algorithm was implemented both in Matlab and C++. Only the computation time of the C++ implementation is discussed. The code is not parallelized and a single core was used. When applying the full method or Lu's method, the computation time depends on the value of n (the number of past events taken into account for updating the pose). We thus evaluate the evolution of both the computation time and the pose errors with the value of this parameter. We show in Figure  14  the results obtained for recordings #01, #04, and #07, where n takes values between 2 and 30. Only three recordings are shown for clarity reasons, results obtained for the remaining recordings are equivalent.
Tools and implementation Saboori used the Umdes library (Umdes, 2009) to implement his verification method for infinite-step opacity, as described in  Saboori & Hadjicostis (2011b)  and  Saboori (2011). Umdes is a library of C routines developed at the University of Michigan for studying DES modeled by finite automata. Falcone developed a specific toolbox named Takos: a Java Toolbox for the Analysis of K-Opacity of Systems  (Takos (2010) ) to implement the K-step opacity enforcement method presented in Falcone et al. (  2014 ) using delays. Finally, in Klai et al. (2014) , a symbolic observation graph-based opacity checker has been implemented in C++ using a binary decision diagram package called BuDDy (BuDDy (1998)). Results are compared with the Takos toolbox on the also well known Dinning philosophers problem.
Experimental setup Data was collected from six ant nests in Rehovot, Israel. Tests were carried out, in the field, during the summer when P. longicornis ants display cooperative transport behavior. In general, a 100 cm Â 70 cm laminated paper sheet was placed near the ants' nest on which either a Cheerio or a 1. 5 mm thick ring-shaped piece of silicon (incubated over-night in either Royal canin, Aimargues, France, or Happycat, Wehringen, Germany, brands cat food) was introduced. In some trials obstacles were introduced between the location of the food and the nest entrance. Perspex obstacles of width 20, 60, or 80 cm were coated with Fluon (Sorpol) to prevent ants from climbing over them, and had a 5 mm slit at their center such that ants could easily pass but the object cannot. Side-view movies were filmed as the ants were transporting a load while walking on an elevated surface covered by millimeter paper.
Two cameras, filming at the same frame rate, were placed above and to the side of the load's trajectory and were manually synchronized according to the ants locations on the millimeter paper. A scenario in which ants do not hold information about the direction to the nest was created by gently picking up the load together with the ants that were connected to it and laying it on a 40 cm Â 30 cm clean paper sheet, with no freely moving ants. The lack of directional information was evident by the load's random walk motion rather than the ballistic motion observed without the manipulation (for details see  Gelblum et al. , 2015). The data were filmed using either a Canon EOS 550D camera or a Panasonic HC-VX870K camcorder. A designated image processing program, developed using MATLAB programming language and described in our previous study  (Gelblum et al.
To rid the dataset of these false hits, the automated stage was followed by a second manual stage. Here, suspect marking events automatically extracted from the speed profile, were further examined by a human observer who reviewed the corresponding raw video footage, where more subtle typical movements (e. g. , slight distortion of the gaster shape, ant shadow location relative to the gaster, and the speed reversal described in Figure  1C ) are visible (see close-up visualization in Video 3). This allowed differentiating true marking events from other stopping behavior showing similar velocity profiles. To test the reliability of the entire marking detection procedure, we simultaneously filmed ants from both the top and side view (Figure  1 -figure supplement 2 and Video 2). Marking behavior was detected, as described above, from the top-view video and then verified against the corresponding side view video.
This uncertainty could be compared to the case in which no message was received such that any of the 36 possible directions is equally probable: H no message ¼ À P 1 36 Á log 2 1 36 À Á ¼ log 2 36 ð Þ. The average information, in bits per mark, is estimated as the reduction of entropy between these two cases: I absolute ¼ H no message À H absolute. We used the same dataset to estimate the amount of directional information that the load gains, on average, from a single mark. To do this, we compared two probability histograms. The first, p before i , is a histogram of the angles, q before , between the line connecting the center of the load to the scent mark (at the time of marking) and the tangent to the load's trajectory in the two seconds that precede the appearance of the mark (bottom panel of Figure  2B ).
All simulation results were obtained through a Monte Carlo simulation. For the grid, we tested the performance of the PF algorithm for different values of probability of following advice in the range [0,1] and a fixed number of nodes N = 50 Â 50 of the network. Each data point corresponds to a random sample of 1000 graphs under the studied model of unreliable advice, in which the routing process was repeated 50 times for different source nodes, assuming that a node located next to the center of the grid was the target. For the line, we varied the number of nodes in the range N 2[2 200], starting from the leftmost node and routing towards the rightmost node. For each length of the line, we sampled between 104 and 106 advice configurations under the studied advice model. The quality of a specific execution of the PF routing process between a source-target pair in a graph is represented by its stretch, defined as the ratio between the number of hops traversed when performing PF routing and the number of hops of the actual shortest source-target path in the updated graph.
Acknowledgements We thank  Gershon Elazar ,  Guy Han , and  Yosef Shopen  for technical help. This research is supported by the  European Research Council (ERC)  under the European Unions  Horizon 2020 research and innovation programme  (grant agreement No  648032 ). E. F. is the incumbent of the  Tom Beck Research Fellow Chair in the Physics of Complex Systems. A. Kos. was partially supported by  ANR (France)  Project DESCARTES and by  NCN (Poland)  grant  2015/17/B/ST6/01897. A. Kor. was partially supported by the  ANR projects DISPLEXITY  and  PROSE , and by the  INRIA  project  GANG. O. F. is the incumbent of the Shloimo and Michla Tomarin Career Development Chair, was supported by the  Israeli Science Foundation  grant  833/15  and would like to thank the  Clore Foundation  for their ongoing generosity.
It highlights issues such as the scalability of the architecture that is collecting tweets, as well as its failover mechanism. The originality of this approach is to combine concerns about data harvesting, data storage and data analysis into one platform. The resulting polyglot storage system supports a relational database, graph database or structured files and a set of tools that can provide a suitable solution for mixing different types of algorithms in order to maximize knowledge extraction. The chapter describes the software architecture of the platform and shows how it uses the Twitter APIs and how end-users can select the best data model according to their analysis. In chapter two, Dario Compagno outlines a mixed methods approach to analysing "families of practices" based on Twitter use by different French candidates during the 2014 European campaign.
In general, graph-based algorithms are near-sighted, they do not take into account contextual information. Linear algebra algorithms can be used to identify large scaled structures, for instance clusters can be found using Singular Value Decomposition (SVD) or Principal Component Analysis (PCA). Machine learning algorithms and statistical models are used to predict links or behaviors, to detect anomalies or events. In a context of massive datasets, the most efficient storage structure should be used according to the selected algorithms, knowing that different kinds of algorithms are usually mandatory. Analysis of tweets can be performed at different levels of granularity: at an individual level like influence assessment, sentiment analysis, by extraction of features that are not explicit; and at a corpus level, through emergence of groups of users exhibiting similar behavior.
New Paradigms for Data Management Systems Most enterprise business applications rely on relational database management systems (RDBMS). This technology is mature, widely understood and adapted. However, some issues have recently emerged: • RDBMS may not have adequate performance for massive datasets; • RDBMS cannot provide the scalability required by online social network applications; • The structure of the relational model can be too rigid or not relevant to deal with the variability of complex data networks; • SQL was not designed to perform explanatory data analysis queries which does not provide exact results. Explicit and implicit links between social data can also be a hindrance to the use of RDBMS if they are combined with massive data. Explicit links are usually used by path queries that require joining tables, but when the length of the path is not known, SQL often needs to be embedded in a programming language.
Implicit links can be discovered by data analysis but the schema of the relational database cannot be quickly and easily updated according to newly discovered relationships. Considering these drawbacks, a number of systems, not following the relational model paradigms, have recently emerged. They are often denoted under the umbrella term of NoSQL databases  (Moniruzzaman 2013). In general, NoSQL databases rely on schema-less data models and scale horizontally. Their common features are scalability and flexibility in the structure of data. NoSQL database management systems provide different solutions for specific problems: the volume of datasets is addressed in the column-oriented NoSQL or key-value (HBase 1 , Cassandra 2 ); documents and links management is supported by document databases (CouchDB 3  , MongoDB 4  ); high density of links, nodes and properties are taken into account in graph database management systems (GDBMS) which are also ideal for performing queries that walk down hierarchical relationships (Neo4j 5  , HypergraphDB 6  ).
XML oriented databases provide a highly extensible data model but lack scalability in the context of social networks. NoSQL databases can be accessed by different APIs and different query languages, so  Atzeni and al. (2014)  propose a common programming interface to NoSQL systems hiding the specification details of the various systems for developing applications. The TinkerPop project 7  adopts a similar approach for graph databases. It introduces a graph query language, Gremlin, which is a domainspecific language based on Groovy 8  , supported by most GDBMS. Unlike most query languages that are declarative, Gremlin is an imperative language focusing on graph traversals. The multi-paradigm principle tends to generalize these different approaches. In modelling, multi-paradigm approaches address the necessity of using multiple modelling paradigms to design complex systems  (Hodge et al.
SNFreezer Architecture In order to collect tweets during the political campaign, we have started by analysing existing solutions and we retained the project YourTwapperKeeper 9 (YTK), an open source project that claims to provide users with a tool that archives data from Twitter directly on a server. After a period of tests and code review, we identified some major drawbacks. YTK was not able to collect tweets in various languages; the choice of the database engine limits the volume of datasets; and it does not retrieve information on accounts such as the list of following/followers nor the timeline of the users. Thus, we chose to enhance YTK with a real storage layer and to add database connectors in order to allow analysis tools such as R to retrieve data directly from SNFreezer repositories.
The Storage System To address the problem of tweet storage, both in terms of performance and interoperability (i. e. easy connection of third party tools) we have specified and developed a storage layer. The proposed polyglot persistence storage layer includes relational databases (RDBMS), a graph data store (GDBMS), and a scalable document database management system (DDBMS) that can be used simultaneously (figure  1 ). The configuration of the storage layer allows duplicating information in different storage systems according to the planned analysis so Neo4j or PostgreSQL ready to use databases are available during the collect. Developers can add their specific drivers for other storage systems. The data harvesting is provided by the data ingestion module connected to the Streaming and Search APIs of Twitter using two main processes that run continuously.
Two other processes are used to collect the followers/following information at a defined frequency and to retrieve all the possible tweets from the timeline of the users. According to the needs and type of information to be analyzed, one can choose between various storage structures: • For computing global metrics (e. g. number of tweets/retweets/mentions per user), a relational schema in PostgreSQL can be used. • In the case of high traffic, it is preferable to store information in a nonnormalized database scheme (one table for tweets and a few others for followers) or in JSON files or in MongoDB. • In the case of the study of linked information (for example social network type data structure), a Neo4j graph database is suitable. The choice of storage backend can be cumulative (both relational normalized databases and JSON files for example).
Cluster Mode and Fail Over Cluster and failover modes have been developed to overcome some limitations of the Streaming and Search APIs of Twitter. For instance, one application can only retrieve tweets matching one of the conditions called source query (attribute querysource in database), in a set of 400. Each account on Twitter can define applications, but for each IP address hosting the application the previous limitations apply. However, some projects require more than 400 query sources. To fulfill this requirement we have developed a cluster management module. In the cluster mode, several virtual machines, in which instances of the data harvesting processes are deployed, can be used in a project to collect tweets using matching criteria in a set of several thousand source queries. Source queries can be defined on each virtual machine using a web interface or can be imported automatically using scripts and import files submitted to one of the virtual machines that acts as a master.
In the cluster mode, it is assumed that there will be a high volume of tweets, thus the storage layer can use a non-normalized schema on PostgreSQL (gtweets table described in section 3. 2), or can store data in MongoDB or in JSON files. Figure  4  gives an example of the cluster deployed to collect tweets during European Election campaign of 2014. In case tweet harvesting was interrupted for any reason, the process in charge of the Search API can retrieve tweets for a period of seven days. Automatic emails are sent to the administrator with a notification of the failure (storage, networks, etc. ) and when the harvesting process is restarted, the latest tweet ID is checked and the process in charge of the Search API starts collecting any lost tweets. Moreover, it is possible to include a local database on each virtual machine using MySQL as a temporary backup, and a global database using PostgreSQL.
Tools were also developed in order to monitor tweet harvesting and the instances of the different local storage in each virtual machine of the cluster. They indicate, amongst other things, the number of tweets for a given period of time, the most used hashtags or the most active users and can send alerts by email to researchers or administrators. In the case of a high volume of tweets, a replica database is used to calculate general statistics for the monitoring tools. During the TEE2014 project this architecture allowed to harvest more than 50M tweets and the resulting database with non-normalized schema is about 50GB without indexes. For the French corpus, experiments have been performed Alex Frame, Arnaud Mercier, Gilles Brachotte and Caja  Téléchargé de PubFactory à06/10/2021 11:57:11AM via free access using Neo4j.
Connection with Analysis Tools and Web Applications A specific data exchange layer is dedicated to application services, and third party tools are plugged in according to the expected analysis. Connectors have been developed to analyse data with third party software such as R and igraph 11 and to display results with D3. js 12. If a connector is not available, the layer provides export files for the different classes of algorithms such as files containing adjacency matrix, graph triple, multidimensional array or CSV (Comma-Separated Values) for spreadsheet. In the following subsection, we give some key elements for starting analysis. In order to perform explanatory analysis on the corpus we have developed some web tools. For example, figure  5  is a component that displays the timeline of a user and shows the frequency of tweets or retweets in a period.
The difference in hashtag frequencies can be regarded as a first approximation of the semantic event description. A part of the program source code (detection of local minima and maxima of temporal density) is available on github 13. 11 http://igraph. org/ 12 http://d3js. org/ 13 https://github. com/kerzol/changepoint-pelt-vs-timedensity Alex Frame, Arnaud Mercier, Gilles Brachotte and Caja  Téléchargé de PubFactory à06/10/2021 11:57:11AM via free access.
Corpus and methodology The corpus used in this study was extracted from the TEE2014 database  14. The database collects all the tweets produced by Belgian, English, French, German, Italian and Spanish candidates to the European Elections 2014 during the campaign. Our corpus includes all messages tweeted by French candidates. In total, around 72. 000 tweets by 467 users have been collected. We decided not to consider for our analysis 106 candidates who tweeted less than 15 tweets during the campaign, and 4 candidates who tweeted more than 1000 tweets, leaving a more homogeneous corpus of 357 candidates. Data treatment, analysis and plots in this study have been realized with the open source statistical software R (R Core Team 2015), including the dplyr (Wickham / Francois 2015), ggplot2  (Wickham 2009)  and cluster  (Maechler et al.
2015)  packages  15. Our study focuses on candidates instead of on individual tweets. Therefore we began by summarizing for each candidate the proportion of tweets including an operator interpreted by Twitter  (Thimm 2012 , Einspänner / Dang-Anh / Thimm 2013). To calculate these proportions we have preferentially considered the meta-data present in the TEE2014 database, which in turn are based on the Twitter REST API  (Twitter Inc. 2015)  and on dedicated extraction algorithms. However we have recalculated these meta-data whenever needed by the ends of our research. The operators used in this analysis are the following: 1. The retweet operator "RT": identifies retweets; 2. The hashtag operator "#": identifies tweets containing hashtags; 3. The hyperlink operator "http": identifies tweets containing links to webpages; 4.
The direct message or reply operator "reply", referred to as DM/reply operator from now on: identifies tweets starting with an "@" character; 5. The source operator "via": identifies tweets ending with "via @" followed by a twitter account; 6. The mention operator "@": identifies tweets including mentions in their text that are no retweet, DM/reply or source operator. Other meta-data considered in the study are the candidate's party name and the tweeting interface most often used by him or her. The choice to focus just on operators to perform a clusterisation of the candidates in our corpus has been determined by past works on the TEE2014 database (Mercier / Villa 2015, Thimm / Einspänner / Gautier 2015) and by a previous exploratory research confirming the importance of operators as features to differentiate among the candidates' tweeting practices  (Compagno 2015).
Classification of tweets based on textual analysis In the last step, we wanted to know if our clusters could be further characterized on the basis of the discourses realized by the candidates in their tweets. Thanks to the software Iramuteq (Ratinaud 2009), we performed to this end a classification of the tweets' texts, based on the co-occurrence of words  (Reinert 1983). As a result of this classification, the tweets in our corpus have been preferentially related to one of two ample classes of co-occurring words. Similitude diagrams for the two classes, realized with Iramuteq, are shown in figure  9. The left half of the figure describes the first of the two classes, that we have called "Political Issues". The words in this class are mainly related to political topics, such as the relationship between France and the European Union, the conduct of the French prime minister François Hollande, the relationships between the three main French parties PS, UMP and FN.