dataesr/scientific-paragraphs-categorization huggingface dataset:
Discussion and Conclusion In this study, we explored the impact of the ENSO events of 2017 on retail sales through the lens of a massive transaction dataset from the greater area of Lima, Peru to understand how the population handled the aftermath of the climatic events. At the macroscopic level, despite a clear slowdown of economic activities triggered by the two main ENSO events occurring in February and March, we demonstrated that the overall economic activity recovered swiftly from the events (see Fig. 3 ). The second event appears to have had less impact on the economic activities than the first event, although its intensity was not smaller. A more detailed analysis of the events indicated that regions that registered more damage induced by the events also suffered from a long-term deficit of consumption compared with other areas (see Fig.
Transaction dataset This dataset was gathered from June 2016 to May 2017, containing approximately 1. 5 million clients, 55,000 distinct merchants, and 116. 8 million transactions from both credit and debit cards in Peru. These data are associated with customer consumption registered by credit and debit cards in stores located in Peru. The dataset is composed of the following features: 1. Features describing the clients such as anonymous ID, age, gender, and country in which the card was issued. 2. Features describing the transaction, such as the timestamp, amount spent in Peruvian currency, and the number of transactions. 3. Features associated with the bank agency, namely the region, province, and district, in which the agency of the client was located. 4. Features characterizing the merchants, such as merchant ID, merchant name, merchant address, the MCC  [20]  and the Lambert coordinates of the merchants.
Kullback-Leibler divergence (KLD) To analyze buying patterns, we used the KLD defined in  (1)  to compute the two different divergence measures D (1) and D (2). The KLD is a general measure of dissimilarity between two probability distributions. In (2), we define D (1) as the KLD between the probability distribution of the share of purchases S (t) i (k), where S (t) i (k) is the share of total expenditures allocated to expenditure category k ∈ [1, 2,. , K] and K is the total number of COICOP expenditure categories, in region i at time t. Thus, S (t) i (k) = (s i1 , s i2. , s ik ) denotes the vector of the expenditures shares in each category made by individuals living in district area i, during the time interval [t, t + w]. We compared it with the average share of purchases in each COICOP category for all transactions in our dataset S(k), which represents the average behavior of a consumer throughout the country.
Finally, if a substantial divergence suddenly appeared in our dataset, we considered it a change in the consumption pattern of an individual living in the particular area. In this study, we limited the geographic area to the 42 districts of Lima. In (3), we define the divergence measure D (2) as the average KLD between S (j) i the distribution of the share of purchases at time j in district j and S (k) i the distribution of share of purchases at time k in district j over w days. KLD(P Q) = k∈K P(k) log 2 P(k) Q(k) (1) D (1) (i,w) (j) = KLD(S (j) i S) (2) D (2) (w,i) (j) = 1 w w d=1 KLD(S (j) i S (k+d) i ) (3).
PageRank PageRank quantifies the importance of nodes (centrality) in a network by computing the dominant eigenvector of the PageRank matrix (or Google matrix  [35] ). Using the PageRank algorithm we computed the ranking c i (t) (9) of all merchants i in our dataset at each given instant t, where W(t) is the weighted adjacency matrix of the snapshot G t of transaction graph G. C(t) = αS(t) -1 W(t) T + (1 -α) 1 N 11 T (9) Here s ii (t) = N j=1 w ij (t) , 1 = [1,. , 1] T and α = 0. 85. We derived the evolution of the ranking of each merchant r i (t) by computing the PageRank of each snapshot G(t 0 ),. , G(t k ) of our dataset. Finally, we computed the normalized ranking r i (t) ∈ [0, 1] (10) of each merchant i at time t as follows: r i (t) = 1 - c i (t) -1 max i c i (t) (10) In Fig. 12 , we can observe several examples of the PageRank evolution in time for distinct merchants.
Based on the transaction dataset, we can define the transaction graph G as a list of temporal snapshots G = [G t0 ,. , G t k ] where G t (V, E(t), W(t)) is a weighted directed graph in which nodes V represent the merchant. An edge e ij (t) exists if there is at least one credit or debit card holder that completed a purchase with merchant i and then merchant j during the interval [t -4, t + 4] (in days). The weights represent w ij (t), the number of co-transactions made by different credit card holders during the interval interval [t -4, t + 4] between merchant i and merchant j. The orientation of the edge represents the temporal order of the purchase sequence. For instance, given the following purchase pattern of user l during the interval [t 0 -4, t 0 + 4] : {P.
Acknowledgments This work was conducted at the  Energy4Climate Interdisciplinary Center (E4C) of IP Paris and Ecole des Ponts ParisTech. It was supported by  3rd Programme d'Investissements d'Avenir   ANR-18-EUR-0006. It was also supported by the  STIC AM-SUD program  through the  04-2017   PEDESTAL  project. The authors thank the computing resource platform of  Telecom SudParis  and  M. Christian Bac  for his help. The authors thank  M. Latapy  for many useful discussions.
Data availability The dataset used for experiments was obtained from a Peruvian private financial entity (BBVA), the dataset was provided to us in the context of a long-standing collaboration between the Universidad del Pacífico (especially the BITMAP team) and the BBVA through a specific multidisciplinary research agreement. Although the dataset provided to us was anonymized and did not contain any personal or identity information about the bank's customers. The dataset provided contains enough information such that the anonymized ID could be subject to data reunification  [41]. In that sense, sharing the raw version of this dataset can potentially breach the privacy of bank's customers. For all these reasons, we are unable to share the raw dataset version of the dataset we have been working with.
Based on the transaction dataset, we can define the transaction graph G as a list of temporal snapshots G = [G t0 ,. , G t k ] where G t (V, E(t), W(t)) is a weighted directed graph in which nodes V represent the merchant. An edge e ij (t) exists if there is at least one credit or debit card holder that completed a purchase with merchant i and then merchant j during the interval [t -4, t + 4] (in days). The weights represent w ij (t), the number of co-transactions made by different credit card holders during the interval interval [t -4, t + 4] between merchant i and merchant j. The orientation of the edge represents the temporal order of the purchase sequence. For instance, given the following purchase pattern of user l during the interval [t 0 -4, t 0 + 4] : {P (l) u (t 0 -1), P.
The protocol developed to analyze the marine traffic of the Iroise Sea and its first results are presented in this article. It is based on a semi-automatic method aimed to clean the original data and quantify the marine traffic along synthetic routes. It includes a procedure that takes into account the temporal evolution of the traffic based on the Allen's time framework. The results proved interesting as they provide an overview of marine traffic, including all types of vessels, and may be defined for different time periods and granularity. A description of the numerical and geographic instruments created is given; all the written code is released as Open Source software and freely available for download and testing.
This paper presents a GIS Python procedure to exploit sémaphore observation data to reconstruct the spatiotemporal evolution of coastal maritime traffic. After justifying the relevance of these data and describing their original format, the article outlines the procedure developed in three stages: data cleaning and integration into a consistent and exploitable database, the extraction procedure of maritime routes, and the reconstitution of their temporal evolution. We then present our first results, coming from elaborations of traffic for five different datasets collected between 2011 and 2013, before concluding with a discussion on challenges and prospects for future work. The code written and the input data (raw and cleaned) are available as Open Source software (https://doi. org/10. 5281/zenodo.
Dataset Structure Sémaphore data are recorded daily in two spreadsheets reporting all the military, commercial, fisheries and leisure activities. Using an empty spreadsheet for each recording day, two officers take shifts of 6 h with the task of recording all the marine traffic and noting down the time, name, registration number, type and route of the boats. Although the recordings related to the military traffic are filtered out by the French Navy, all the remaining traffic (commercial, fishing and leisure) is collected for "public service" purposes and has been kindly provided by the FOSIT for use in this research. A total amount of 44,183 boat passages is examined in this work. The remaining obstacles to use these data are the lack of a completely standardized procedure and the use of a fixed linguistic classification for data collection: the observations are recorded according to different syntaxes, which could generate significant confusion in their interpretation (an example of raw data is shown in Table  1 ).
Data Cleaning and Database Integration The first step of the data processing consists of the concatenation of the daily recorded files (two files per day, thus giving 730 spreadsheets per year) in a single annual file, properly formatted for date and time information. This work was automated by writing a macro in VisualBasic in Excel. The next step aims at cleaning the annual file in order to make the data readable and exploitable for various purposes. Since each spreadsheet, for each year and for each sémaphore, can contain several thousand lines, a semi-automatic procedure was used in order to make the first language standardization for the type of boats, usage and routes. For each element, the basic assortment (i. e. , for the type of boats: "speedboat", "lifeboat", "scow", etc. ) was selected according to the occurrences encountered in the Iroise sémaphores  [44].
Later, users could expand this list to suit other regional contexts. The cleaning procedure, reported in the following flowchart (Figure  4 ), was implemented as Python code using Qt library (https://riverbankcomputing. com/software/pyqt/, accessed on 3 March 2021), and includes a graphical user interface (GUI) in order to be as user friendly as possible. As input, the code takes an annual spreadsheet file and three dictionaries containing the fundamental information: types of usage, types of vessel and routes related to the raw data occurrences for each element. These dictionaries are created automatically, verifying the occurrence of routes in a first core of data (represented by Saint-Mathieu sémaphore traffic for the year 2011), and during the routing it is possible to enlarge these dictionaries to take into account new situations.
In case of totally new elements, new values of the original keys or new keys can be specifically added. The output is made up of a clean annual data file and the updated dictionaries. Python code using Qt library (https://riverbankcomputing. com/software/pyqt/, accessed on 3 March 2021), and includes a graphical user interface (GUI) in order to be as user friendly as possible. As input, the code takes an annual spreadsheet file and three dictionaries containing the fundamental information: types of usage, types of vessel and routes related to the raw data occurrences for each element. These dictionaries are created automatically, verifying the occurrence of routes in a first core of data (represented by Saint-Mathieu sémaphore traffic for the year 2011), and during the routing it is possible to enlarge these dictionaries to take into account new situations.
This allows the quantification of the traffic at different time scales. However, in this procedure, the routes are not explicitly described. In fact, the navigation of each boat can only be reconstructed from the indications of origin and destination. The synthetic route is therefore calculated from two gates: an entry point (origin) and a way out (destination). The direction of the route is assigned automatically by the code and the route is calculated according to the principle of the shortest route (shortest path route logic from  [46] ). A GRASS Python tool was developed to calculate the shortest route (called v. createRoutes. py). The calculation performed in v. createRoutes. py can be divided into two main parts (Figure  5 ), both of which are written in Python. The first part is dedicated to data preparation; the second concerns data spatialization and is based on the use of the routine "grass_script".
In the first part of the flowchart, the original data are read and formatted: spaces are removed; the names of the gates are edited according to the routes, etc. Moreover, an analysis of the occurrences of all gates is performed in order to read the geographic coordinates of the identified gates from a text file. Basically, from sémaphore records, a gate can be represented in three ways: a specific and exact point (e. g. , a port), a zone (e. g. , a group of islands), or a direction. route is assigned automatically by the code and the route is calculated according to the principle of the shortest route (shortest path route logic from  [46] ). A GRASS Python tool was developed to calculate the shortest route (called v. createRoutes. py). The calculation performed in v. createRoutes. py can be divided into two main parts (Figure  5 ), both of which are written in Python.
This model was implemented in TGRASS from the module t. vect. createRoutes. py, and allows us to perform both an instantaneous analysis (create a timestamped vector map reporting the instantaneous passage of boats on a geometric grid) and a period analysis, with a specified granularity. In the first case, the output will be a single timestamped map reporting the instantaneous traffic situation for a specific sémaphore. In the second case, the output will be a set of timestamped maps, following each other over a user-specified period and granularity. All the vector maps produced are saved into a GRASS database in order to speed up the geographical processing, and can be exported in different formats for a wide variety of purposes. For instance, it is possible to produce an animation of the traffic evolution and save it as a.
gif file. The module was written in the Python language and involves most of the v. createRoutes. py routines; the graphical user interface is reported in Figure  10. The temporal information required for the period or moment parameter must be formatted as "dd:mm:yyyy hh:mm", and the granularity expressed in minutes. The presence (or absence) of the flag "a" determines the production of the animation. This model was implemented in TGRASS from the module t. vect. createRoutes. py, and allows us to perform both an instantaneous analysis (create a timestamped vector map reporting the instantaneous passage of boats on a geometric grid) and a period analysis, with a specified granularity. In the first case, the output will be a single timestamped map reporting the instantaneous traffic situation for a specific sémaphore.
In the second case, the output will be a set of timestamped maps, following each other over a user-specified period and granularity. All the vector maps produced are saved into a GRASS database in order to speed up the geographical processing, and can be exported in different formats for a wide variety of purposes. For instance, it is possible to produce an animation of the traffic evolution and save it as a. gif file. The module was written in the Python language and involves most of the v. createRoutes. py routines; the graphical user interface is reported in Figure  10. The temporal information required for the period or moment parameter must be formatted as "dd:mm:yyyy hh:mm", and the granularity expressed in minutes. The presence (or absence) of the flag "a" determines the production of the animation.
All the code written for this work (including both cleaning data and geoprocessing routines) has been released as Open Source software (license GNU GPL v. 3). It can be freely downloaded from https://github. com/annalisapg/maritimeTraffic/tree/v. 1. 0 (accessed on 3 March 2021) and it is published in Zenodo (https://doi. org/10. 5281/zenodo. 4420051, accessed on 3 March 2021).
Storing and Displaying Data in INDIGEO SDI In order to visualize these data, we used the scientific Spatial Data Infrastructure (SDI) INDIGEO  [52]  at the LETG (Littoral, Environnement, Télédétection, Géomatique) research unit. The functionalities of this SDI include cataloging, storage and distribution of geographic information produced by LETG and its partners. The SDI consists of a metadata catalog and a geo-referenced data server backed by a web portal with a viewer. The deployed solution is based on geOrchestra free tools (geonetwork, geoserver, openlayers. ), an initiative of GéoBretagne (a French regional SDI, http://cms. geobretagne. fr/, accessed on 3 March 2021). INDIGEO is interoperable with other regional, national and scientific SDIs through the use of Open Geospatial Consortium standards (WMS, WFS, WCS).
It also benefits from the development of an ergonomic and scalable viewer (GeoCMS) to meet the specific needs of science (series of temporal data, spatiotemporal data, etc. ) and thus make them public in accordance with the principles of the European Directive INSPIRE (http://inspire. ec. europa. eu/, accessed on 3 March 2021). Several INDIGEO functions are useful to our project: support for spatiotemporal data, an interactive and customizable way to query data, support for graphic time-series representation, a folder structure to save and share maps, and a user-friendly administration interface. The ability to navigate through the temporal dimension of a spatiotemporal dataset stored on a PostGIS database (Figure  11 ) and the visualization of statistical data through a chart module are other specificities.
It is thus possible to differentiate vessels in transit from those that are stationary. For the "period" mode, the output is represented by several timestamped pairs of vector maps (a line map of the routes and a point map of the gates), cumulating traffic over a specified period of time, following a user-defined granularity. These data can be exported from GRASS in various vector formats (e. g. , shapefile or kml) in order to be analyzed by alternative means. For example, they can be archived into a PostGIS database or imported into specific applications to be visualized in a 3D environment. Figure  13  shows the traffic evolution over four months of the year 2011, each one representative of a season, for the Saint-Mathieu sémaphore. This output shows the most frequented routes during the year and, consequently, it allows us to individualize the relative pressure on the coastal environment.
Quantitative Analysis The integration of these outputs into INDIGEO makes it possible to build simple graphical representations, which provide relevant support for their quantitative analysis. In order to correctly interpret these maps, it should be remembered that the original data are semantic; so, taking into account the potential visibility of the sémaphore observers' annotations, it would not be surprising if the Le Conquet-Westward route meant "high seas" when visibility is perfect, or "The Islands" when visibility is poor.
Discussion In summary, the model we developed firstly reads the raw data and performs the cleaning procedures through cleanDataByDicts. py; in a second step, the temporal information is added to the spreadsheet by the addTime. py tool; and, finally, the data are processed through t. vect. createRoutes. py. The output of these processes is either a couple of time stamped vector maps, if t. vect. createRoutes. py is used in "moment" mode, or a set of sequential timestamped vector maps, if the "period" mode is set. In this case, a temporal granularity must be specified in order to assign a temporal resolution. The vector maps produced in this way show the aggregation of traffic fluxes in a specific moment or over a period of time. Each map contains quantitative information about the traffic through the parameter "npass" calculated both for the line vector file of routes, and the point vector file of gates, to distinguish boats in transit from stationary ones.
At the present stage of our work, the procedure shows encouraging results: reasonable initial language standardization, cleaning procedure and representation of data over space and time. One quite innovative aspect of the methodology used is the spatialization of semantic traffic data. In fact, this spatialization technique can be re-used each time a "moving object" needs to be spatialized, and can be refined by applying more precise spatialization rules, as mentioned above. Moreover, it is possible not only to create a representation of the traffic, but also to export timestamped data in order to obtain a complete traffic dataset ready to be analyzed and processed for any further purpose. Therefore, considering that there are sémaphores positioned along the entire French coastline and that these data are readily available, an analysis at national level could now be pursued.
Additionally, the results of this study could be compared with the data from other monitoring systems, such as AIS or GPS data, with the general aim to achieve more accurately than now the "real" amount of leisure boating in the total marine traffic. In fact, in current conditions, the contribution of leisure boating to the marine traffic, and the pressure it exerts on the coastal environment entity, could only be summarily estimated since not all recreational craft (especially the small ones) carry an AIS or GPS on board. The methodology has now been widely tested over five different datasets (Saint-Mathieu, years: 2011, 2012, and 2013; Toulinguet, 2011; Cap La Chèvre, 2011). The total amount of traffic observed and processed during this period was about 44,000 passages of vessels in the Iroise Sea.
In addition, some simulations were performed using different configurations, for example, short simulations on one week of traffic, with a temporal granularity going from 30 min to one day, or long period simulations on one year, with a temporal granularity of one month. Moreover, various "instantaneous" situation maps and animations have been produced. Some statistical analyses have been conducted in order to qualify and quantify the traffic over the observed period. These data will be presented in detail in a later publication concentrating on this theme.
Conclusions This work is original in several ways. First, it is based on original, unexploited data collected from sémaphore monitoring of marine traffic. Despite their semantic nature, these data are relevant for describing synthetic marine traffic flux in the spatiotemporal dimension, especially because the observation made by the network of sémaphores is continuous in space (French coast) and in time (24/24 h, 7/7 d). In fact, the results obtained from these data and the approach we propose are able to provide information for a most comprehensive summary on maritime traffic. Indeed, much of this traffic is composed of boats that are not currently observed by any other device than the visual and radio observations made by the sémaphore network. The production of such information could open at least two interesting prospects for public policy support, including the identification of areas of potential conflict between several kinds of marine activities, and the assessment of the cumulative pressure generated by marine activities.