dataesr/scientific-paragraphs-categorization huggingface dataset:
Data availability. This study made extensive use of the CALIPSO level 2 5 km merged layer product  (Vaughan et al. , 2018 ; NASA Langley Research Center Atmospheric Science Data Center; https://doi. org/10. 5067/CALIOP/CALIPSO/LID_L2_05kmMLay-Standard-V4-10; last access: 10 July 2018), the CALIPSO level 2 5 km cloud profile product (Vaughan et al. , 2018; NASA Langley Research Center Atmospheric Science Data Center; https://doi. org/10. 5067/CALIOP/CALIPSO/LID_L2_05kmCPro-Standard-V4-10; last access: 10 July 2018), the CALIPSO level 2 5 km aerosol profile product (Vaughan et al. , 2018; NASA Langley Research Center Atmospheric Science Data Center; https://doi. org/10. 5067/CALIOP/CALIPSO/LID_L2_05kmAPro-Standard-V4-10; last access: 10 July 2018), and the CALIPSO level 2 vertical feature mask product (Vaughan et al.
, 2018; NASA Langley Research Center Atmospheric Science Data Center; https://doi. org/10. 5067/CALIOP/CALIPSO/LID_L2_VFM-Standard-V4-10; last access: 10 July 2018). All CALIPSO lidar data products are also available from the AERIS/ICARE Data and Services Center (http://www. icare. univ-lille1. fr, AERIS/ICARE; last access: 10 July 2018).
Mobile phone data are a popular source of positioning information in many recent studies that have largely improved our understanding of human mobility. These data consist of time-stamped and geo-referenced communication events recorded by network operators, on a per-subscriber basis. They allow for unprecedented tracking of populations of millions of individuals over long periods that span months. Nevertheless, due to the uneven processes that govern mobile communications, the sampling of user locations provided by mobile phone data tends to be sparse and irregular in time, leading to substantial gaps in the resulting trajectory information. In this paper, we illustrate the severity of the problem through an empirical study of a large-scale Call Detail Records (CDR) dataset. We then propose Context-enhanced Trajectory Reconstruction, a new technique that hinges on tensor factorization as a core method to complete individual CDR-based trajectories.
Call Detail Records (CDR) are the current de-facto standard for mobile phone data used in human mobility studies. As illustrated in Table  1 , CDR are logs of events generated during mobile communications, such as voice calls, text messages, and possibly mobile data traffic sessions, which are collected for billing purposes  [2]. Those events are associated with individual mobile devices and are time-stamped and geo-referenced, which makes CDR an obvious proxy for the trajectories of the mobile network subscribers. Interestingly, CDR encompass huge populations (e. g. , millions of users) over large (e. g. , citywide or nationwide) geographic regions, and cover long periods (e. g. , months or years): they thus enable human mobility investigations at unprecedented scales. A drawback of CDR-based trajectories (or simply trajectories hereafter) is that they are often largely incomplete.
Details are in Section 3. • Second, we introduce our novel approach, CTR, which leverages well-known features of human mobility to customize tensor factorization in a way that befits our problem. This original methodology lets CTR transform sparse mobile phone data into seamless individual trajectories that span the full dataset duration, for all users. Details are in Sections 4 and 5. • Third, we validate the proposed strategy with ground truth data. Comprehensive performance evaluation shows that CTR achieves full reconstruction of individual trajectories on an hourly basis, with a median displacement between 1 and 2 network cells that depends on the sparsity of the original CDR data. This effectively means that, in the reconstructed data, a user is typically placed in the correct cell or in one that is very close to it.
Time discretization A very common practice in mobile phone data analyses, time discretization reduces the temporal resolution of the data so as to make trajectories appear complete. Indeed, discretizing time allows associating a single location to each time step, implying that the location is valid across the whole interval covered by the step. We remark that time discretization is basically unavoidable in CDR, due to the limited granularity of mobile communication events. Interestingly, however, it is performed at quite different temporal resolutions by works in the literature, spanning from 15 minutes  [14]  to 1 hour  [3, 4] , or even to 2 hours  [5]. Also related to data alteration in time, some studies focus on specific periods (e. g. , work hours, commuting hours, nighttime hours), which allows them to treat the (time-discretized) data as complete even if just such target periods are covered  [13].
User filtering It is customary for works on mobile phone data to filter the available user base before further processing. User filtering can be adopted to eliminate outliers (e. g. , call centers  [15] ), but it is more often considered to retain trajectories that provide sufficient mobility information. Indeed, when extensive knowledge of the user mobility is required by the nature of the investigation, the straightforward way to deal with incomplete trajectory data is that of filtering users so as to exclude from the analysis those with insufficient location information. This is typically performed by imposing thresholds on the amount of mobility data available in CDR, though there is hardly a standard to define them. As a reference, several important works opt for a choice of users who have a minimum average sampling frequency of 0.
5 events per hour, and who are observed at two unique locations at least  [3, 4, 14]. User filtering comes with a cost in terms of below-threshold users that are excluded from the analysis. Unfortunately, in practical cases involving CDR datasets, even the loose requirements set by the thresholds mentioned above risk to lead to severe reductions in the examined user population. For instance, two very well-known studies analyze just 1. 67%  [3]  and 0. 45%  [4]  of the total subscribers available in their datasets upon filtering, leaving substantial richness in the original data untapped.
Trajectory reconstruction The objective of trajectory reconstruction is to infer the positions of the users at times during which the original mobile phone data does not provide such information. By recovering the missing location data, a successful reconstruction makes user movements more complete, hence effectively obviates the problem of discarding vast portions of users caused by filters on mobility data. The literature on trajectory reconstruction from mobile phone data, including CDR, is relatively thin, and mostly focuses on partial reconstruction. A fairly standard solution is extending the validity of instantaneous locations over longer time intervals, by assuming that users do not move for some periods (e. g. ,, in the order of hours) before and after every recorded event in the CDR dataset  [16].
However, users with such a high level of communication activity are infrequent, and indeed only 0. 07% of the total subscribers in the original CDR dataset are retained for analysis in  [19]. When applied to typical CDR data, interpolation methods are ineffective and provide no clear improvement over a naive error-prone method where users are associated with the last location they are seen at  [17]. The solution we propose in this paper also falls in the trajectory reconstruction category, and, like interpolation, aims at inferring complete individual trajectories. However, we seek an approach that effectively also operates on very sparse positioning data, and thus can be applied to entire CDR datasets.
Sparsity in CDR-based trajectory data As a preliminary step to our study, we carry out a data-driven investigation of sparsity in individual trajectories extracted from mobile phone data. Our goal is quantifying the phenomenon, so as to demonstrate the significance of the problem and to motivate the need for a dedicated solution. To this end, we employ a CDR dataset collected from 1. 8 million prepaid mobile network subscribers of a major Latin America mobile network operator. The data consists of 778 million voice call entries generated by customers during a consecutive 90-day period. The customers appear within a large geographic area of several thousands of km 2 covered by 4, 200 cell towers. As illustrated in Table  1 , each CDR entry contains the caller's and the callee's hashed identifiers, the call duration, the event time, and the identifier of the cell tower which the caller's device was connected to when the call originated.
Trajectory completeness We first quantify the amount of missing information in the individual trajectories. To this end, we assume a total observation time T and discretize it into time intervals of duration τ , which we will refer to as the temporal resolution. We define the completeness of a specific trajectory as the fraction of time intervals for which we have at least one location sample. As an example, let us assume that T = 7 days and the temporal resolution is τ = 1 hour: then, a trajectory with locations in 80 different hours has a completeness of 80/(7 × 24) = 0. 476. In practice, T corresponds to the duration of the dataset from which the trajectories are extracted, and τ to the temporal resolution required for the mobility analysis. As both values may vary based on the nature of the mobility study, we investigate their impact on completeness through comprehensive parametric analysis.
Instead, our notion of completeness is independent of how positions are determined in the presence of multiple samples in the same time interval. Hence we do not need to account for that aspect at this stage. Figure  1  portrays the cumulative distribution function (CDF) of the completeness of 1. 8 million trajectories from our reference dataset when different combinations of T and τ are considered. Each plot refers to a different temporal resolution τ , while curves identify different observation periods T. We observe the following. • Seamless trajectories with completeness equal to 1 are extremely rare in mobile phone data, under any combination of T and τ. Even with short observation periods (T = 7 days) and low resolution (τ = 2 hours), not a single complete trajectory is extracted from our reference dataset, despite the large user population.
In fact, less than 1% of users would satisfy a minimum completeness requirement of 0. 5, which allows half of the locations to be unknown; and, more than 20% of trajectories do not even meet the degree of completeness 0. 1, i. e. , miss 9 locations out of 10. The fact that such poor figures refer to the best case, i. e. , the less demanding combination of T and τ , illustrates well the severity of the sparsity problem in CDR datasets. • The duration of the observation period has a marginal impact on trajectory completeness. More precisely, and as one would expect, slightly higher completeness is recorded for trajectories that span a shorter observation period when T ∈  [30, 90]  days; however, and quite interestingly, completeness hardly varies when T ≤ 30 days. We argue that the result can be linked to the weekly periodicity of human activities (which entail a reduced difference for T ∈  [7, 30]  days), plus seasonal effects such as summer holidays that only intervene over long timescales (T ≥ 30 days in our dataset).
Overall, our results highlight that the vast majority of trajectories from a typical CDR dataset only capture a rather small portion of the complete user mobility. Based on Figure  2 , we quantify the fraction of known positions between 5% and 20% on average, depending on the temporal resolution needed for the analysis. These results are well aligned with the outcome of user filtering in previous studies that leverage mobile phone data, where only a small subset of users displays sufficient completeness and is retained for analysis  [3, 4, 19].
A statistical model of completeness As a further contribution to the characterization of the sparsity problem, we derive a data-driven model of CDR-based trajectory completeness. The model can be leveraged to obtain a quick, approximate estimate of the completeness of trajectories extracted from a CDR dataset, by merely providing the timespan of the CDR data and the desired temporal resolution. In order to derive the model, we fit theoretical distributions to the completeness probability density function (PDF) and link the distribution variables to the system parameters T and τ. Earlier studies have shown that mobile communication activity tends to follow long-tailed distributions across users  [20]. Since the usage patterns of mobile devices directly determine completeness, it makes sense to look at heavytailed theoretical distributions.
Trajectory reconstruction problem Let us consider mobile phone data spanning an observation period T. User trajectories are inferred by discretizing T into intervals of duration τ , hence T = {1, • • • , N } can be regarded as the set of all time intervals. The actual (i. e. , ground-truth) trajectory of a generic individual in the dataset is denoted by L T = {l i | i ∈ T }, where l i is the representative location of the user during the i th time slot. When a single mobile communication event is associated to a time interval i, the representative location l i maps to the geographic location associated with the event. This is the geographical position of the antenna that is the most representative of the user location. Although our discussion is general and can accommodate any definition of antenna representativeness, we opt for a strategy of selecting the antenna that handles the most communications during the target time slot i.
Nighttime trajectory reconstruction The first step in CTR aims at reconstructing missing portions of a trajectory during the night hours. The rationale is that, as seen in Section 4. 2, these are easily reconstructed from the extractable knowledge about users' home locations, which is also straightforward to derive from mobile phone data. Specifically, we assume that nighttime spans between 10 PM and 9 AM of the subsequent day: this period is adapted to the typical schedule in the Latin America country where our data is collected, but are easily adjusted to any other settings with statistical data on the local population habits. For each user in the dataset, we proceed as follows. We first identify the most frequent location visited by the user during nighttime, which we name lH. Then, we verify that lH accounts for a conservative 80% (or more) of the positions of the user during the night period.
If this is not the case, we deem that the result is not consistent enough to establish with high confidence the actual home location of the user, and we skip the nighttime trajectory reconstruction step. Otherwise, we identify lH as the home location of the user. When applied to our reference dataset of 1. 8 million subscribers, this strategy allowed assigning a home location to 0. 72 million users, equivalent to 42% of the whole population. If a home location lH is identified, each 10 PM -9 AM period is processed separately for every day. The user is considered to be at home only in periods where no location other than lH is observed in the data: in this case, all time intervals i in the period are added to a set Ω H. Otherwise, if at least one location different from lH is present during a 10 PM -9 AM period, we consider that the user may not have spent that specific night at home, and no modification is made to Ω H.
Once all days have been processed, we assign the home location to all slots in Ω H , i. e. , li = lH , ∀i ∈ Ω H. We then update the known trajectory as LΩ = {l i | i ∈ Ω} ∪ { li | i ∈ Ω H }, and the set of missing locations by Ω C = T -Ω -Ω H. In our reference dataset, an average of 82% of the nighttime slots of the aforementioned 0. 72 million users could be reconstructed this way.
Seamless trajectory reconstruction with tensor factorization The second step of CTR aims at reconstructing the trajectory during the remaining time intervals in Ω C , by tailoring state-of-the-art tensor factorization techniques to our problem. Tensor factorization exploits redundancy to recover missing data; in our context, redundancy is created by the regularity of human movement, which creates repeated patterns of visited locations over the many days and weeks covered by a CDR dataset, as discussed in Section 4. 2. We start by organizing the trajectory data of a user into a three-dimensional tensor format, as illustrated in Figure  5. The location tensor X has dimensions that reflect the weekly, daily, and instantaneous (with granularity τ ) mobility of the user. Therefore X ∈ R Nw×N d ×2Nτ , where N w is the number of weeks in the observation period, N d is the number of days in a week, and N τ is the number of time slots in a day.
1. At this stage, the location tensor includes information for time slots in Ω ∪ Ω H. However, the tensor is still relatively sparse, as the set Ω C of time intervals where locations are missing is still large. In our reference dataset, the location tensors have an average density of 0. 63 (where 1 denotes a complete tensor) when first generated. In order to complete the location tensor of an individual, we employ tensor factorization, which decomposes the tensor into hyper-parameters and then uses them to infer missing values. Factorization has proven very effective in recovering highly incomplete structural data in other contexts  [24, 25]. Formally, we formulate an equivalent tensor factorization (TF) problem as follows. First, we perform a canonical polyadic decomposition (CPD)  [26]  of the tensor X ∈ R Nw×N d ×2Nτ into three R-rank matrices A ∈ R Nw×R , B ∈ R N d ×R , and C ∈ R 2Nτ ×R , where each value X i,j,k in the tensor is approximated as X i,j,k = R δ=1 A i,δ B j,δ C k,δ.
To this end, we first design a sensible cost function for determining the suitability of a match ( li , c j ) between a generic location and a cell tower position. The cost function, denoted by f ( li , c j ), accounts for (i) the geographic distance between li and c j , and (ii) the popularity of the tower c j , as: f ( li , c j ) =    1/e j if c j , li ≤ r j +∞ otherwise. , (5) In expression  (6)  above: e j is the number of communication events that are generated by all the users in the dataset and associated to the cellular tower c j in the mobile phone data; • denotes the geographic distance between the two location parameters; and, r j is the cellular radius of tower c j , i. e. , the maximum distance between the actual tower position c j and the farthest corner of its Voronoi tessellation cell  [27]  as illustrated in Figure  7.
Validation In order to validate the proposed CTR solution, we employ a set of ground-truth trajectories with completeness equal to 1, presented in Section 6. 1. We generate incomplete trajectories from the ground-truth data, following the same sampling process observed in our reference CDR dataset, as described in Section 6. 2. Then, complete trajectories are reconstructed with CTR from the downsampled ones, and compared against the ground-truth, in Section 6. 3: the level of agreement lets us comment on the quality of the reconstruction.
Fine-grained ground-truth trajectories We have access to data about the mobile Internet sessions of a small subset of the 1. 8 million users in the CDR dataset introduced in Section 3. As more events are generated from Internet access rather than voice calls, they entail a sensibly higher frequency in location sampling. When considered jointly with CDR, rich mobile Internet data allows obtaining the complete trajectories of 1, 450 users over an observation period T of two consecutive weeks, and with a temporal resolution of τ = 1 hour. If multiple samples are recorded in the same time slot, the unique location is determined as that of the cell tower which most events are associated with. In other words, we have 1, 450 complete trajectories having observation sets Ω = T. In each of them, all the locations are known, i.
Incomplete trajectories Incomplete trajectories are generated by downsampling the ground-truth ones. For instance, a trajectory with completeness 0. 3 is derived from a two-week ground-truth trajectory having 14 × 24 = 336 time slots by keeping the locations in 101 time slots and discarding the rest. Here, the sampling process must be chosen appropriately: for instance, a uniform random selection would result in a temporal distribution of retained locations that is inconsistent with that produced by mobile communication events in CDR. We ensure that the incomplete trajectories mimic the temporal sampling properties of those extracted from CDR as follows: (i) we construct an empirical distribution of the probability that events are recorded at each time slots, based on CDR data of all 1. 8 million users in our dataset; (ii) we set the desired completeness value k that shall characterize the output incomplete trajectory; (iii) we retain a fraction k of the overall location samples in one complete trajectory according to a random selection of time slots that follows the empirical distribution above.
Revisiting key results in the literature As mentioned in Section 1, mobile phone data are the cornerstone of many seminal works on large-scale human mobility analysis. All such works (i) employ raw CDR datasets that are highly incomplete, and (ii) possibly adopt substantial user filtering. However, if and how trajectory incompleteness and user filtering effectand possibly bias -the outcome of these studies is unclear. Our aim in this section is shedding light on this matter, which also lets us demonstrate the importance of trajectory reconstruction in mobile phone data analysis. To this end, we run CTR on the reference dataset presented in Section 3 and reconstruct all trajectories which have completeness higher than 1% in the original data. Selecting such a lower threshold on completeness bounds the typical cell displacement to a small value according to the validation in Section 6, hence ensure high confidence in the quality of the estimated locations; moreover, a 1% completeness threshold allows retaining 95% of the user population.
e. , incomplete) CDR-based trajectories but is reduced in the complete data, where the cutoff occurs instead at 175 km. We ascribe the difference to the fact that completion often leads to add missing locations that are far from the linear interpolation between CDR positions, such as those generated by infrequent but recurrent long-haul trips. Our conclusion is that, CDR data sparsity risks to both underestimate long trips, and overestimate the region within which the Lévy flight behavior of human mobility occurs. • Concerning individual movements, we can reproduce the truncated power-law behavior found by Gonzalez et al. with both our sparse CDR-based trajectories and the complete ones reconstructed via CTR. This is illustrated in the bottom plots of Figure  9. In the original work, the authors find an exponent α ≈ 1.
2, and we estimate the α parameter at 1. 8 and 1. 4 for original and complete trajectories. These exponents are qualitatively consistent since they all are in the (1, 3) range that characterizes Lévy walks  [29]. • We ascribe the quantitative differences with respect to  [3]  to the inherent specificity of each dataset. Although we also use locations from voice calls as in  [3] , the data refer to very different cultural and economic settings (i. e. , a European country and a South American one) and geographical span (our country covers a territory that is four times wider than the largest country in Europe). Overall, our results corroborate the findings in  [3] , and show that the resulting conclusions hold also with seamless trajectory data from larger populations.