dataesr/scientific-paragraphs-categorization huggingface dataset:
For instance, data collected from the residence permits include details about intention and length of stay. They also require specific details on place of origin and address in the country of stay. The same applies to labour permit data. Nevertheless, in Europe where the freedom of movement and work is established, it is difficult to know flows and stocks of EU immigrants using these administrative data unless all the individuals are registered. An alternative is to use health insurance data. With these, it is possible to infer the stocks more accurately, provided the immigrants register for health insurance. In addition, registries can also collect information about asylum seekers 7 and refugees 8. However, this information is not always present in all migration data. In some countries like France, Italy, United Kingdom and so on, asylum seekers residing at least 12 months in a country are included in the data.
Alternative data sources -is nowcasting possible? In recent studies, the use of social big data in the study of immigrants' journey is increasing. A variety of data types can fall under this category. They can be data from social media, internet services, mobile phones, supermarket transaction data and more. These datasets contain detailed information about their users. Furthermore, they cover larger sets of population than some of the traditional data sources which are limited in terms of sample size. Yet, the literature points out that the data may be biased because of users' characteristics in the sample. For instance with Twitter data, it is known that the majority of the users are young and that it cannot represent the whole population. Nevertheless, various of studies state that the observed estimates of immigrants' flows and stocks extracted from these unconventional data sources can still improve the understandings of migration patterns (see for instance,  [181, 89, 127] ).
Big Data allows researchers to study immigrants' movements in real-time. Twitter data for instance, provide geo-located timestamped messages. Geo-located messages are often the key variable in estimating the flows and stocks but not the only one. In the work of  [181] , the authors infer migration patterns from Twitter data by looking at where the tweets were posted. 9 It compares statistics of both immigrants and emigrants between a set of country. The degree of underestimation of number of emigrants can be inferred by doing so. 10.
United Nations Economic Commission for Europe Other studies like  [127]  assume origins of immigrants from language used in tweets, whether the local language was used or not. These studies conclude that Twitter data allow researchers to localize the flows and stocks of immigrants and to observe recent trends even before the official statistics are published. The results of these studies are validated by matching the big data results to official data. In one of our recent works, we have also analysed geolocalised Twitter data, with the aim of quantifying diversity in communities, by computing a superdiversity index  [141]  (see also Section 3). This index correlates very well with migration stocks, hence we believe it can become an important feature in a now-casting model. A different line of work we are pursuing is that of estimating user nationality from Twitter data.
As seen above, language can be important in understanding nationality, however we believe that this can be refined by employing also the connections among users. The model can be validated with data collected through monitoring frameworks such as that presented in  [24]. Once users are assigned a nationality, we can use these for a now-casting model of migration stocks. Additionally, we can define communities on Twitted based on nationality, and study the flow of ideas among communities, and the role of migrants in the spreading of information. Furthermore, these data could enable analysis of ego-networks of migrants (see Section 3. 2. 5 bellow). Skype Ego networks data can also be used to explain international migration patterns  [103]. In this case, the IP addresses that appear when users login to their account can be used to infer the place of residence.
In the context of studying labor mobility, the availability of massive datasets of individuals' career path fostered works on predicting individuals' next jobs (outside the academia)  [157]. Paparrizos et al. [137]  build a system to recommend new jobs to people who are seeking a job, using all their past job transitions as well as their employees data. They train a predictive model to show that job transitions can be accurately predicted, significantly improving over a baseline that always predicts the most frequent institution in the data. Recently, Li et al. [114]  propose a system to predict next career moves based on profile context matching and career path mining from a real-world LinkedIn dataset. They show that their system can predict future career moves, revealing interesting insights in micro-level labor mobility.
Our recent work, conducted within the SoBigData projects, is placed on the line of conjunction of the aforementioned strands of research. In particular, we investigate how a scientist's scientific profile influences the decision to move, based on a massive dataset consisting of all the publications in journals of the American Physical Society (APS) from 1950 to 2009 -360,000 publications, 3,500 institutions and 60,000 scientists  [99]. We approach the problem by constructing a two-stage predictive model. We first predict, using data mining, which scientist will change institution in the next year. We describe a scientist's profile as a multidimensional vector of variables describing three aspects: the recent scientific career, the quality of scientific environment and the structure of the scientific collaboration network.
From the constructed predictive model, we identify the main factors influencing scientific migration. Secondly, for those scientists who are predicted to move, we predict which institution they will choose using the performance-social-gravity model, an adaptation of the gravity model of human mobility to include the above mentioned factors. A different recent line of work in the SoBigData project is to understand, by using ORCID data, what was the effect of the Brexit referendum on scientific migration in and out of the United Kingdom. Preliminary results (still unpublished) show an increase in UK researchers moving from the EU to the UK and an increase of EU researchers moving out of the UK.
Towards a novel integration index using alternative data sources While the type of studies exemplified in the previous section have been instrumental in understanding the effects of migration, the fact that they are based on traditional data makes them inherit the disadvantages of these data. Big Data can help to analyse the issues above, and others, with the advantage of producing real-time results, and enabling analysis at higher spatial resolution. For instance, retail data can help understand how immigrants adopt habits and values of the new community they live in. Mobile call data records (CDR) can be used to describe social interaction and mobility patterns of immigrants, and understand segregation. OSN data can help study various topics, such as social integration, language adoption, changes in the local language, sentiment towards immigrants, etc.
Our project owns a key data source for these analyses, composed of scanner data from a large Italian retail market chain, that are available since January 2007 for more than 1. 1 million customers holding a fidelity card. The dataset includes the price, quantity, promotional sales (if any) and the name of the good purchased out of a set of around 600,000 products. Besides this information, for each customer the country of birth is available and the date on which the fidelity card was obtained. About 7% of the customers are foreign-born, when the immigration rate in Italy is currently around 8. 5%. On average, a foreign customer is observed 5 times per month, with a mean monthly food expenditure of about EUR150. In Figure  1  we report the cumulative number of customers joining the fidelity club for Albania, Romania and France.
Call data records A large amount of work has been done using call data records (CDR) in understanding individual personal mobility. These range from empirical analyses of large CDR datasets  [78, 139, 138, 73, 179]  to proposal of theoretical mobility models  [155]. Initiatives to motivate researchers to analyse CDR data have also appeared, through data challenges such as the Data for Development (D4D) challenge in Senegal  [37]  or the recent Data for Refugees (D4R) challenge in Turkey  [2]. Readers can refer to  [36]  for a survey of works related to using CDR data for individual mobility studies and models. A recent example is the study of the flocking and mobility behavior of the population after the Haiti earthquake using CDR data  [116]. Researchers found that mobility patterns of the population after natural calamities is predictable.
It has also been shown that mobility patterns can be used for creating detailed maps of population distribution which are more accurate and recent. This approach is in particular useful for poor countries. This in turn can help in creating proper socio-economic policies for the population  [54]. However, while mobility analyses are abundant, not much work has been done to analyse the international migration phenomenon using CDR. This due to several reasons. First, CDR datasets typically span only one nation. Secondly, in general, due to privacy reasons, no information on the nationality of the customer is provided. Without these pieces of information, studying migration with these data is difficult. One exception is the above mentioned D4R challenge, where refugee status of customers is made available.
Our project has participated in this challenge, together with several other teams, concentrating on five different aspects: health, integration, unemployment, safety and security, and education. For details on result obtained by other teams please see the published collection of articles  [153]. Our objective was to analyse integration and combine the Turktelekom data with other datasets  [33]. We observed that integration seemed to increase in time for refugees, and also that the presence of refugees influenced the house market in Turkey, decreasing housing prices. Another recent example where CDR data was used to analyse transnational mobility is  [7] , using CDR data that includes mobile roaming events. Transnational population mobility can be defined as living and working in two or more countries.
[101]  have compared Twitter mobility patterns with patterns observed through other technologies, eg. CDRs. , by using individuals' spatial orbit as the measure of how far they move; Gonçalves et al. [77]  have found two global superdialects in the modern-day Spanish; and Doyle  [57]  have proposed a Bayesian method to build conditional probability distributions of the spatial extension of English dialects. Within the SoBigData project, we have analysed the concept of Superdiversity theorized by  Vertovec (2007) , and proposed a measure to quantify it  [141]. We focus on the conjunct analysis of both language and geographic dimensions starting from a Twitter dataset. Our ground hypothesis is built on the idea that different cultures use the language in different ways and, in consequence, the emotional value associated with words changes depending on the culture of the person that writes a tweet.
We introduced a Superdiversity Index (SI), that is based on the diversity of the emotional content expressed in texts of different communities. Specifically, we extract the emotional valences of words used by a community from Twitter data produced by that community. We compare the obtained valences with a standard dictionary tagged with sentiment. The distance between the community and the standard valences is a measure of superdiversity for the community. This SI measure is computed at different geographical scales based on the Classification of Territorial Units for Statistics (NUTS) for two different nations: Italy and United Kingdom, and validated with data from the above mentioned D4I challenge (see Section 2). We observe a very high correlation with immigration rates at all geographical levels.
negative). The analyses we conducted show how the framework captures the differences in positive and negative user sentiment over time and space. The resulting knowledge supports the understanding of complex dynamics by identifying variations in the perception of specific events and locations. We used the Twitter Streaming API under the Gardenhose agreement (granting access to 10% of all tweets) to collect the English tweets posted in two periods: from mid August to mid Sept 2015 for the refugees dataset, and from mid June to the beginning of July 2016 for the Brexit dataset, respectively. We filtered out the tweets not related to the specific events analyzed. The first dataset refers to the Refugees crisis and contains about 1. 2 M tweets, while the second one refers to the Brexit referendum and contains about 4.
3 M tweets. The datasets 13 are available for use through Transnational Access in the SoBigData project infrastructure. In our study we try to answer the following analytical questions: What is the evolution of the discussions about refugees migration in Twitter? What is the sentiment of users across Europe in relation to the refugee crisis? What is the evolution of the perception in the countries affected by the phenomenon? Are users more polarized in the countries that are most impacted by the migration flow? Is the polarization of the users about refugees and the Brexit referendum somehow correlated? For this purpose, we analyse the ratio between pro-and against-refugee users across Europe. For example, Figure  3  shows the geographical distribution of this ratio considering all users residing in a country, but also internal and external perception (perception of the users residing inside/outside a country C related to the refugees in C).
d4science. org/group/ resourcecatalogue/data-catalogue?path=/dataset/ hpc_twitter_dumps  178, 75, 13, 173]. For this reason, studying the properties of migrants' personal network is a particularly promising avenue of research in digital demography, in order to characterise both the journey and the stay. In this perspective, studying the evolution of personal networks over time is the ideal approach to characterise the modification of migrants' social structures (or lack thereof), due to the migration process. This is related to one of the main subjects of study in this area, i. e. , the characterisation of integration of migrants. Integration is typically measured in terms of assimilation and transnationalism. Assimilation is defined as the gradual adoption of customs and traditions from the receiving country by the migrant, and can be full  [11, 12] , partial  [74]  or segmented  [143].
e. , the effect of different emotional closeness between node pairs is not taken into account. Lamanna et al. [110]  again focus on cultural assimilation but from the spatial segregation standpoint. In this case, they use a bipartite graph structure, connecting tweet languages and cities. In  [92] , Facebook is used to study the network of teenagers in the Netherlands, concentrating on ethnicity and gender. The analysis shows that ethnicity plays a stronger role in link formation. However, the extended Facebook networks are less segregated, in general, compared to core egonetworks. To the best of our knowledge, ego networks of migrants built from OSN data have never been investigated in the related literature. This is quite surprising, as it is well-known that many facets of the human behaviour chiefly depend on the ego network structure.
To measure migrant integration, we believe that several new data types can be used to introduce novel integration indices, based on retail consumer behaviour, mobile data, OSN language, sentiment and network analysis. Research in this direction is slightly less developed, mostly due to low availability of ready-to-use datasets. Our consortium is making steps in this direction, using existing datasets, participating to data challenges or collecting new data. For the return of migrants, again reserach is limited, although potential exists in data such as retail, mobile or OSN. In all three dimensions, research has to carefully consider the issues with the data that is being used. The complexity of digital demography implies that there is no free lunch with digital traces either  [108]. One problem relates to the representativeness of the collected samples.
We implement at least one representative method of each type and compare both qualitatively and quantitatively on a large-scale dataset covering the European area at a resolution of 1/16 of an arc minute in the context of the European Marine Observations Data network (EMODnet) Bathymetry project. The results show that, although the simplification method designed for elevation data attains the best results in terms of mean error with respect to the original terrain, the other, more generic state-of-the-art 3D simplification techniques create a comparable error while providing different complexities for the triangle meshes.
Introduction Today, the rapid growth of geospatial information sources enables mapping the complete surface of the Earth at increasing levels of fidelity. Computationally-efficient visualization of such world-scale terrain datasets, especially at very-high resolution and large coverage, has been a subject of interest for some time. In this direction, many standards for web-based rendering of large 2D maps, such as the Tile Map Service (TMS)  [1]  or the Web Map Tile Service (WMTS)  [2] , have been recently developed. Several services, including OpenStreetMaps or Google Maps, have adopted them in order to render world-scale imagery quickly for their users. However, when the data is not pure imagery and one wants to transfer and display 3D geometry, the problem becomes more involved. This is the case, for instance, for the EMODnet (European Marine Observations Data network) Bathymetry project  [3] , which has, as one of its main objectives, the creation of a Europe-scale bathymetric map.
Moreover, some standards, such as the Common Database (CDB) from the Open Geospatial Consortium [9], detected the need for the said LOD data structure. However, building this TIN structure from large-scale high-resolution data presents some issues. If we tackle the problem of creating a given LOD by simplifying the whole terrain, it will likely require huge amounts of memory, preventing this process from being performed on commodity hardware. One may infer that, since our structure is tiled, the LODs may be built at the tile level, which would effectively solve memory issues. However, the problem then is to maintain the coherence between tiles, as, when put together, they should form a single continuous surface. Unfortunately, leveraging per-tile simplification with surface continuity is an open issue.
Therefore, in this paper we focus on the insights of creating a multi-resolution pyramidal tile-based data structure for very large terrains, in which all the processing is performed at the tile level, while ensuring that the borders of the tiles are coincident within an LOD. The key idea is to restrict new tiles to be computed to stick to the borders of the already triangulated ones. With this restriction in mind, we show how three different well-known simplification methods can be adapted to work at the tile level. By working on one reduced subset of data at a time, the creation of this data structure requires low computational resources, especially in terms of memory. Therefore, the proposed solution provides an accessible procedure for computing this LOD data structure, which will benefit the different domains requiring the visualization of large terrain models, including Geospatial Information Systems (GIS), simulations, and the gaming industry.
This paper is organized as follows: The next section starts by reviewing the state of the art in LOD creation for terrain data, focusing on the problems found in those methods when trying to build a data structure, such as the one we envision. Then, Section 3 will introduce the benefits of our methodology, while Section 4 will develop its implementation. Finally, Section 5 validates the proposed approach by showing its practical application on a real, large-scale dataset before concluding and enumerating possible future work in Section 6.
Related Work Rendering terrains in real time at a resolution that adaptively matches with a given point of view is a current research topic in computer graphics and in the geographical information system communities. Despite recent technological advances, a moderately-sized dataset can easily exceed the capabilities of current graphics hardware. Indeed, rendering all data points and associated triangles spanning a regularly gridded height field is computationally expensive. Early approaches tried to simplify the surface into a TIN statically, where the complexity of the mesh defining the terrain is adapted to the terrain variance  [10]. This idea has been extensively developed not only for its use in terrains (where we have a common reference XY projection plane that simplifies the problem), but for 3D surface meshes in general  [11, 12].
However, as previously noted, rendering a regular grid in 3D requires a huge amount of primitives. Moreover, the amount of data to transfer is also prohibitive. Therefore, it is preferable to use a TIN data structure to represent the tiles. One can face the problem of creating this structure in many ways. On the one hand, the TIN for the terrain at a given LOD can be first derived, and then split into tiles  [7, 8]. In that case, the bottleneck is obviously at the lower levels of the pyramid, since the task storing a large terrain in RAM, even if in TIN form, will require a large amount of memory, preventing its execution on commodity hardware. On the other hand, as each LOD is split and stored using square tiles, one can work at the tile level, and simplify only the part of the terrain that falls within its boundaries  [6, 28].
Scaled Coordinates To be independent of the coordinate system of the input terrain, we apply the simplification of a given tile in scaled coordinates. That is, each coordinate is normalized to [0. 1], according to the maximum and minimum values within the tile. This normalization is required because of the huge difference in scales for coordinates when working, for instance, with a raster dataset in the WGS84 coordinate reference system. In this case, having latitude and longitude for X and Y and the height in meters, with values that are normally different in several orders of magnitude, the triangles may degenerate and specially provide numerical stability problems. Nonetheless, most point set simplification techniques are designed to be applied to a metric space. Thus, when applying some point set simplification strategies, the data may need to be transformed again to metric Earth-Centered Earth-Fixed coordinates.
Results We validated our methods by applying them to the global DTM generated in the EMODnet Bathymetry project  [3]. The aim of the EMODnet project is to gather bathymetric datasets disseminated in multiple European data centers. Along with the inventory of the bathymetric data, the EMODnet Bathymetry portal also provides direct access to a high-resolution digital elevation model of the topography of the European seabed. Among over 20,000 individual sources of bathymetric surveys, approximately 9,000 of them have been carefully selected and merged to compose the DTM. The 2018 release of the EMODnet Bathymetry DTM  [40]  has a grid resolution of 1/16 × 1/16 arc minutes (approx. 115 × 115 meters), provided within a WGS84 geodetic system, with geographical boundaries at N90 W36 and N15 E43, which results in a 113, 892 × 108, 132 grid (approximately 12.