dataesr/scientific-paragraphs-categorization huggingface dataset:
The bias-variance trade-off is more evident, since performance improvements are made at the expense of decreasing reliability. This is especially observed for small data sets such as Bladder, Colon1, Colon2 and AOD, whose Qini metrics and standard deviations are larger for 10 percent targeting than for 100 percent targeting. On the other hand, in data sets where the observed overall effect of treatments is negative (e. g. , Bladder), MTUM techniques prove to be valuable instruments to improve treatment effectiveness. For example, the treatments considered in the Bladder, Colon1, Colon2 and AOD data sets would exhibit unfavorable effects if target assignment was not customized according to the predictions of the MTUM techniques. A Qini curve is a useful visualization tool to assess model performance.
Assessing model performance: the expected response Alternatively to the Qini metric, the expected responses of optimal targeting as predicted by the MTUM techniques are reported in Table  7. Panel A and Panel B show the expected responses of targeting the full test sample and only the top 10 percent of test cases, respectively. The largest expected responses are in bold. In large data sets such as Hillstrom, Gerber and Bank, the expected responses for both exposure segments do not differ significantly across models. However, there are slight differences in the expected responses among the MTUM approaches for small data sets. The CTS, ED, XLearner and RLearner methods are as competitive as the approaches we propose in this study. Conventional techniques, on the other hand, are clearly suboptimal.
Their estimations are also more stable, since they have smaller variations. For example, MMOALR is consistently among the best performers for the Hillstrom data set, as observed in the plots of the Qini curves and the expected responses. It is a simple, easily interpretable and computationally inexpensive approach. In summary, the primary advantage of our methods is their ease of implementation, since they are based on existing algorithms that are readily available and generally known. Moreover, they are built upon conventionally accepted binary uplift modeling approaches that have been previously evaluated by several studies. most likely to respond to particular actions. In this regard, uplift modeling serves as a tool to investigate and anticipate the effects of treatments in diverse contexts.
Generally, the study shows that the performance of an uplift multitreatment technique is highly context-dependent. On the other hand, we observe that the size of the uplift data set has implications for the capacity of a model to compute reliable estimates. Small data sets such as Bladder, Colon1, Colon2 and AOD present high volatility in the uplift predictions among different folds in the cross-validation evaluation. This study includes certain limitations, which can serve as a motivation for future research. First, optimal matching with propensity scores leads to an important loss of information when treatment groups are not of equal size. In addition, this technique is highly dependent on the correct specification of the set of observable characteristics. Other methods for correcting selection bias could offer more reliable uplift estimates.
285) 10. 85%* Recognition (363) 1. 84%* Metcbt5 (200) -6%* No. of variables 18 11 8 13 12 16 24 5 Response Visit Vote No recurrence Recurrence or death Death Subscribe No turner Reduce use Channel E-mail Mail Medication Medication Medication Call Retention Program Domain Marketing Political behavior Clinical trial Clinical trial Clinical trial Marketing Human resource Public policy Multitreatment data sets Source Hillstrom (2018) Gerber et al. (2008) Therneau (2015) Therneau (2015) Therneau (2015) Moro et al. (2014) Private organization McCaffrey et al. (2013) Table 3 Dataset Hillstrom Gerber Bladder Colon 1 Colon 2 Bank Turnover AOD.
The proposed model promotes spatial correlation between neighbouring points within a given surface using spatial point processes. Secondly, we account for different spatial and spectral subsampling schemes, which reduce the total number of measurements, without significant degradation of the reconstruction performance. In this way, the total acquisition time, memory requirements and computational time can be significantly reduced. The experiments performed using both synthetic and real single-photon Lidar data demonstrate the advantages of tailored sampling schemes over random alternatives. Furthermore, the proposed algorithm yields better estimates than other existing methods for multi-surface reconstruction using multispectral Lidar data.
M. Márquez is with the Department of Physics, Universidad Industrial de Santander. H. Arguello-Fuentes is with the Department of Systems Engineering, Universidad Industrial de Santander. J. -Y. Tourneret is with the Signal and Image Department, University of Toulouse. This work was supported by the Royal Academy of Engineering under the Research Fellowship scheme RF201617/16/31. This work was supported by EPSRC grant number EP/R033013/1. This work was partly conducted within the ECOS project 'Colored apertude design for compressive spectral imaging' supported by CNRS and Colciencias, and within the STIC-AmSud Project HyperMed. MATLAB codes can be found at https://gitlab. com/Tachella/musapop. allowing the use of laser sources of low power  [1]. The acquired range information (3D structure) has many important applications, such as self-driving cars  [2] , the study of structures behind dense forests  [3]  and environmental monitoring  [4].
Datasets containing dozens of wavelengths can be prohibitively large for practical 3D reconstruction algorithms, both in terms of memory and computing requirements. For example, a typical MSL hypercube with 32 wavelengths has more than 10 9 data voxels. To alleviate this problem, some compressive acquisition strategies have been proposed. While TCSPC technology hinders compressive techniques along the depth axis  1  , reducing the number of measurements can be achieved by integrating multiple wavelengths in a single histogram  [9]  or measuring fewer histograms (i. e. , Fig. 1. An airborne MSL system can capture multiple objects per pixel and discriminate their materials. The multi-depth capability enables the recovery of information from photons reflected off different branches of the trees, ground, pedestrians or even from the interior of a car (i.
In this work, we investigate a new pseudo-random subsampling scheme for low-photon count MSL data based on ideas from coded aperture design  [19] ,  [20]. By choosing the pixels measured for each wavelength in a more principled way, we achieve better results than the completely random schemes of Altmann et al. [16]. Furthermore, the proposed subsampling strategy can be easily implemented in many Lidar systems, reducing the total number of measurements, i. e. , the time to acquire an MSL frame. Raster-scan systems using a laser supercontinuum source  [6] ,  [7]  can be easily modified to measure only a subset of pixels per wavelength. More interestingly, single-wavelength array technology  [21]  can be combined with coded apertures  [19] , which acquire different wavelengths at each pixel. Furthermore, we propose a new method to perform 3D reconstruction from subsampled MSL data, which is capable of finding multiple surfaces per pixel.
We introduce new RJ-MCMC proposals, which take into account the additional spectral dimension and improve the acceptance ratio, compared to the ones proposed in  [15]. Moreover, we propose an empirical Bayes approach to build the prior distribution associated with the background detections, which further improves the convergence of the RJ-MCMC sampler. Contrary to multi-depth methods that require storage of dense volumetric estimates (e. g. ,  [13] ,  [14]  in the single-wavelength case and  [22]  in the multi-temporal case), the memory requirements of the proposed method are minimal (just the 3D points and spectral signatures are stored in memory), enabling the acquisition and processing of very large datasets (dozens of wavelengths and hundreds of pixels).
The main contributions of this work are • a new Bayesian 3D reconstruction algorithm for subsampled MSL data with multiple surfaces per pixel • the analysis of an appropriate subsampling scheme based on coded aperture design, which provides better results than completely random alternatives. The new algorithm is referred to as MuSaPoP, as it models MultiSpectrAl Lidar signals using POint Processes. The remainder of the paper is organized as follows. Section II recalls the classical observation model for single-photon MSL data. Sections III and IV present the Bayesian 3D reconstruction algorithm and the associated RJ-MCMC sampler. Section V details the principled subsampling strategy. Experiments performed with synthetic and real Lidar data are introduced and discussed in Section VI. Conclusions and future work are finally reported in Section VII.
III. MULTIPLE-RETURN MULTIPLE-WAVELENGTH 3D RECONSTRUCTION Recovering the position of the objects c n , their spectral signatures r n and background levels B from the subsampled MSL data Z is an ill-posed inverse problem, as many solutions can explain the observed photon counts. Thus, prior regularization is necessary to promote reconstructions in a set of feasible 3D point clouds. In this work, we adopt a Bayesian framework, which allows us to include prior knowledge about the scene through tailored prior distributions assigned to the parameters of interest.
a) Independent prior distributions: In order to simultaneously model potential spatial correlation and ensure the positivity of the background levels, ManiPoP uses a 2D gamma Markov random field, which was introduced by Dikmen et al. in  [31]. However, this prior is not well suited for MSL data as it introduces an undesired penalization for large background levels, whose negative effects are amplified when considering multiple bands (see Appendix A for details). Other alternatives such as Gaussian Markov random fields  [26]  cannot be sampled directly in closed form, requiring proposals with a rejection step, whose mixing and convergence scale badly with the dimension of the spectral cube, as shown in  [32]. To alleviate these problems, we assign independent gamma priors, i. e. ,        p(B|K, Θ) = Nr i=1 Nc j=1 L =1 p(b i,j, |k i,j, , θ i,j, ) b i,j, |k i,j, , θ i,j ∼ G(k i,j, , θ i,j ) (8) where [Θ] i,j, = θ i,j, and [K] i,j, = k i,j, are the shape and scale hyperparameters of the gamma distributions.
Assuming that we have a coarse estimate of the point cloud (this information will be obtained using the multiresolution approach detailed in Section IV-B), we build the distribution q(B) following an empirical Bayes approach, as illustrated in Fig. 4. First, one can discard almost all the signal photons in the dataset by removing the photons detected in the compact support of h (t) around each point (see Fig. 4 , central subplot). The number of bins that is not excluded in each pixel is referred to as v i,j,. Secondly, we integrate the remaining photons of each pixel, obtaining a coarse estimate of the per-pixel background photon levels, denoted by zi,j, ,b. We then define q (b ) ∝ p(z |b )p(b |α B ) with      zi,j, ,b |b i,j, ∼ P(g i,j, v i,j, b i,j, ) b i,j, = exp( bi,j, + µ ) b |α B ∼ N (0, α B D -1 ) (10) where b a vectorized image of background levels at wavelength , D ∈ R N rN c×N rN c a positive semidefinite matrix, α B is a fixed hyperparameter controlling the degree of smoothness.
d) Split and merge moves: Some pixels might present two points with overlapping impulse responses in depth. In such cases, a death move followed by two birth moves would happen with very low probability. Hence, as in ManiPoP, we propose a split move, which randomly picks a point (c n , m n ) and proposes two new points, (c k1 , m k1 ) and (c k2 , m k2 ). The 3 Intensities are normalized to belong to a fixed interval across datasets. Hence, we can fix the variance of the proposal to achieve similar acceptance ratios. log-intensity is proposed for each wavelength following the mapping      u ∼ B(η, η) m k1, = m n, + log(u ) m k2, = m n, + log(1 -u ) (20) where B(•) denotes the beta distribution and η is a fixed parameter. The new positions are determined according to                      s ∼ Be(0.
B. Full algorithm We adopt a multi-resolution approach to speed up the convergence of the RJ-MCMC algorithm, in a fashion similar to  [15]. The dataset is downsampled by integrating photon detections in patches of N bin × N bin pixels. Hence, the number of pixels is reduced by a factor of N 2 bin , meaning less points and background levels to infer with N 2 bin times more photons per pixel. The estimated point cloud at the coarse scale is upsampled using a simple nearest neighbour algorithm and used as initialization for the next (finer) scale. In all our experiments we repeat the process for K = 3 scales. The background hyperpriors K and Θ are initialized with noninformative values, i. e. , k i,j, = 0. 01 and θ i,j, = 100 for all pixels (i, j) and wavelengths. In finer scales, these hyperparameters are computed using the algorithm detailed in Section III-A3.
While many compressive sensing strategies have been proposed for measurements under this noise assumption  [36] -  [38] , MSL datasets have an additional limitation if multiple surfaces per pixel are considered: photondetections belonging to different wavelengths cannot be integrated into a single histogram, as the reconstruction problem generally becomes highly non-convex, preventing practical reconstruction algorithms  4. Indeed, summing histograms associated with different wavelengths and including multiple peaks generates histograms with even more peaks (possibly overlapping), which makes the 3D reconstruction and the reflectivity estimation more difficult. As a consequence, we only consider subsampling of depth histograms, which incorporates all of the practical sampling limitations. Following the formulation of the observation model (  2 ), the subsampling strategy consists of choosing the binary coefficients G for a given compression level W/L, with W the average number of observed band per pixel.
VI. EXPERIMENTS To illustrate the efficacy of the proposed method, the new reconstruction algorithm is compared to other alternatives (based on the work conducted in  [6] ) using a synthetic dataset. Subsequently, the new subsampling scheme is compared with other random subsampling choices for a real MSL dataset. In all the experiments, the performance was measured using the following summary statistics:   | , normalized with respect to the total number of ground truth points. The ground truth and estimated points are coupled using the probability of detection F true (τ ). Note that if a point was falsely estimated or a ground truth point was not found, then they are considered to have resulted in an error of.
A. Synthetic data We first assessed the performance of the proposed algorithm using a synthetic dataset created from the "Art" scene of the Middlebury dataset  [50] , as shown in Fig. 7. The measurements were obtained by simulating the single-photon multispectral Lidar system of  [17] , whose bin width is 0. 3 mm. The generated dataset has N r = 283 and N c = 231 pixels, T = 4500 histogram bins and L = 4 wavelengths (red, green, blue and yellow), where only W = 2 wavelengths out of 4 were sampled per pixel using the coded aperture introduced in Section V. The mean number of photons per wavelength per pixel is 10, where approximately 3. 4 photons are due to the background illumination. As in mono-static Lidar systems, the background levels are generated as a passive image of the scene (see Fig.
B. Real MSL data The proposed subsampling scheme was evaluated on a real MSL dataset  [17]. The scene consists of L = 32 wavelengths sampled at regular intervals of 10 nm from 500 nm to 810 nm, N r = N c = 198 pixels and T = 4500 histogram bins. The target is composed by a series of blocks of different types of clay and two leaves. Fig. 10  shows an RGB image of the scene and the 3D reconstruction using acquisition times up to 10 ms per wavelength per pixel. We compare the blue noise codes mentioned in Section V with the random schemes introduced in  [16] , all yielding the same total number of measurements and acquisition time: 1) Random sampling without overlap: W out of L bands per pixel are sampled without replacement (i. e. , for a given pixel, each wavelength is measured at most once).
True points found [%] Random without overlap Random with overlap Blue noise shows the execution time for acquisition times of 10, 1 and 0. 1 ms and different numbers of sensed bands. The proposed RJ-MCMC sampler has a complexity proportional to the number of photon detections in the support of the impulse response around the 3D point being modified, whereas the background update has a complexity proportional to the total number of active histogram bins in the Lidar scene. The background extraction step required around 15% of the total execution time, which could be significantly reduced if all the bands were processed in parallel instead of sequentially as it is done in the current implementation. All the experiments were performed using a Matlab R2018a implementation on a i7-3. 0 GHz desktop computer (16GB RAM).
Finally, we compared the performance of MuSaPoP with the single-depth multiple-wavelength algorithm by Altmann et al. [16]. The algorithm is referred to as Depth TV and considers total variation regularizations for the background, reflectivity and depth images. Note that this method requires a (global) depth interval where all signal photons are found, which is given manually by the user. We also considered a target detection scenario (i. e. , some pixels without surfaces), by removing the backplane of the scene and keeping only photons associated with background levels. In this case, we post-process the Depth TV estimates, removing points with a mean normalized intensity below 10%, which gave the best results across the evaluated datasets. In both experiments, we used the blue noise codes with W = 8 wavelengths per pixel out of L = 32.
g. , Hamiltonian Monte Carlo [24, Chapter 9]) do not scale well with the dimension of the problem, being inefficient when applied to large multispectral Lidar datasets. Hence, we consider proposals from a Gaussian approximation of (10) (as detailed in  [26] ) using the perturbation optimization algorithm  [52] , accepting or rejecting them according to the Metropolis-Hastings rule  [24] ,  [26]. We generate 10 3 samples { b(s) i,j, , s = 1,. , 10 3 } and compute the desired expected values as E q {b i,j, } = s exp b(s) i,j, (28) E q {log b i,j, } = s b(s) i,j,. (29) Finally, the values of the hyperparameters are obtained by setting θ i,j, = E q {b i,j, } and minimizing  (27)  with a onedimensional Newton method.
However, much of the geographic variation in invertebrate composition remains unexplained, and consequently begs for more explicit incorporation of broad-scale variables like climate. For example, extreme rainfall events lead to an inversion of the trophic pyramid of macroinvertebrates in bromeliads across seven study sites broadly distributed across the neotropics  (Romero et al. 2020). Here, we used fine-grain data on the functional traits of aquatic macroinvertebrates sampled in more than 1600 tank bromeliads across 18 climate zones throughout the Neotropics to partition the separate and combined effects of local conditions and climate on community trait composition. Based on previous studies of bromeliad macroinvertebrate ecosystems, we expect that both local conditions and climatic gradients affect trait composition  (Céréghino et al.
org/10. 5281/zenodo. 1200194>. Principal component analysis (PCA) was used to reduce trait dimensionality to significant axes of trait variation. The rank-transformed [species × trait] matrix was used to compute Spearman's rank correlations between trait modalities, which then underwent PCA with bootstrap resampling  (Pillar 1999). This procedure allowed us to test ordination stability and to interpret the significant ordination axes in light of correlations with trait states. We identified a robust set of four orthogonal and important axes of trait variation, namely trophic position, habitat use, morphological defence and life cycle complexity  (Céréghino et al. 2018). The species scores for these four PCA axes (available at <https:// doi. org/10. 5281/zenodo. 1200194>) thus represented continuous trait values, or synthetic traits, which we then used in analyses of the processes underlying functional diversity across different spatial scales in relation to environmental factors (this study).
Site sampling The data for the present study consists of the abundance of macroinvertebrates found within 1436 bromeliads in 18 field sites in eight countries (Bromeliad Working Group BWG database, Supporting information). While the Bromeliad Working Group has sampled more countries and field sites, we only use the sites where more than 15 bromeliads were sampled. Some of these field sites were visited in multiple years (Supporting information), while others were visited only once. While we acknowledge there are temporal trends in community abundance and composition, all of our sampling units (each sampled bromeliad) are a snap-shot of the community structure, so we made the simplifying assumption to treat all sampling units within a site the same, regardless of the year in which they were collected.
Bromeliad sampling The following bromeliad genera were sampled across all sites: Neoregelia, Quesnelia, Tillandsia, Guzmania, Vriesea, Aechmea and Catopsis. Each tank bromeliad was exhaustively sampled either by dissecting the rosette, or by pipetting out all of its contents. The bromeliad macroinvertebrate community is defined as all the aquatic invertebrates found by the naked eye (> 0. 5 mm) within a single plant. A total of 637 morphospecies were found. Three local conditions were collected for each plant: water volume at time of sampling (ml), total amount of detritus (mg) and canopy cover (either open or closed canopy). The data on total amount of detritus and the water volume at the time of sampling were log-transformed before the analyses. These variables are proxies of habitat size and energy inputs, which are key drivers of food web structure  (Oertli et al.
2002 , Moore et al. 2004 ). The total amount of detritus was calculated by adding the amount of small, medium and large detritus. In 25. 8% of bromeliads, total detritus was not measured directly, instead we imputed total detritus with allometric relationships using other size categories of detritus, and in a few cases with the number of leaves and the diameter of the plant. In 50. 4% of bromeliads, total water volume was not measured directly and instead we imputed total water volume using leaf size, plant species, plant height, plant diameter and number of leaves. When either of these variables were missing, we used generalized linear models with a Gaussian error distribution to impute missing values. Thus, the dataset was a combination of directly measured and estimated values.
The square root transformation de-emphasizes superabundant species (gave a more equal weight to rare species), and the community weighted means (CWM) allowed us to characterize the relative abundance of traits in the sampled bromeliad. Community weighted means were obtained using the data ver. 0. 7. 7 extracted from the BWG database in July 2017. To determine the effects of environmental conditions on the CWM computed using the species scores on the four synthetic traits, we used a permutational multivariate analysis of variance -PERMANOVA (also known as distance-based RDA) (vegan R package;  Anderson 2001). This method allows us to partition distance matrices into sources of variation, and is based on within-and between-group sums of squares computed on pairwise dissimilarities, in this case, of bromeliad communities considering the CWM trait values.