dataesr/scientific-paragraphs-categorization huggingface dataset:
Therefore, it is advisable to use PCA dimensionality reduction in both blind and non-blind situations. In the final comparison (Section 3. 4), all representation learning methods yield a fairly small completeness for the UVP5/MC target dataset. This can be explained by the fact that the MorphoCluster labeling of the UVP5 dataset is only one of possibly many valid labelings and that the simulation treats interchangeable classes as distinct. Just by training another model on the same data, a different representation space was learned that is less aligned with the UVP5/MorphoCluster labels.
Clustering In MC20, we selected the density-based HDBSCAN* clustering algorithm to generate candidate clusters for annotation  [22]. Density-based approaches come with a range of favorable properties, e. g. , the detection of clusters of differing density and arbitrary shape or noise filtering  [73]. However, few Python implementations exist, and, in preliminary experiments, only HDBSCAN* was able to deal with the size of the used plankton datasets. Here, we devised two alternatives specifically tailored to the requirements of Morpho-Cluster, S-k-means and PL-k-means. The idea of modifying k-means so that it allows samples to be classified as noise is not new. NK-means  [74]  is a preprocessing step for k-means to remove outliers from a dataset prior to clustering. However, this operation is prohibitively expensive for the size of the used datasets.
To reconcile the different modalities of deep image representations and environmental and spatiotemporal metadata, multi-view clustering methods might be necessary  [80, 81]. We found that, overall, the tested algorithms perform comparably, and any of the three can be used. The newly developed clustering algorithms S-k-means and PL-k-means are advantageous if the completeness is more important than the efficiency or runtime, or for smaller datasets. HDBSCAN*, on the other hand, runs much faster than its competitors and therefore might be better suited for real-world applications. However, being based on k-means, S-k-means and PL-k-means inherit its potential for more efficient implementations in the future. In contrast to MC20, we found that, for HDBSCAN*, the initial minimum cluster size m 0 is of lesser importance than previously assumed and that, instead, the neighborhood size k should be increased above the previous choice to k = 8.
Practicality of the Results The primary goal of MorphoCluster is the maximization of annotation efficiency to keep up with the constant inflow of new image data. In our experiments, this value was often higher than five objects per click. The best efficiency was obtained with supervised representation learning and S-k-means (8-12 objects per click). Annotation completeness, the percentage of the dataset that could be efficiently sorted, is important, as a high completeness reduces the amount of data that need to be postprocessed using different annotation approaches. According to our results, the completeness mainly depends on the training mode of the feature extractor. If labeled target data were available, up to 84% of a dataset could be sorted with high efficiency (ZooScanNet, fully supervised).
63% was achieved  [22]. In this work, we take the annotations provided with the used datasets not as ground truth but as the annotations of just another expert. The annotation precision we measure here is, therefore, actually the consistency with these gold standard labels. It reaches or even surpasses the self-consistency levels experts commonly reach under optimal circumstances  [82]  (95% to 99%) for all image representations, clustering configurations and datasets. Moreover, plankton image annotation is particularly difficult, as the captured data provide only a very limited view of the physical world and lacks spatial and temporal aspects and often even color, not to mention non-visual qualities. Therefore, image annota-tion is a priori always subject to uncertainty and interpretation.
In Appendix F, we show the outliers of two gold standard classes as an illustration. Accordingly, the small variations in the throughout high precision are less significant than completeness or efficiency, and we conclude that MorphoCluster is able to approximate the true annotation of a dataset with a high consistency in most scenarios. Like in the real MorphoCluster application, the simulation reaches this high annotation precision while also investigating only a reduced number of images for growing. This is a strong indication that the MorphoCluster approach of clustering, validation and growing can, in comparison to conventional one-by-one sorting, save manual effort by approximating the true annotations with a reduced number of images a user has to review without sacrificing precision.
These settings will be incorporated into a future version of the MorphoCluster image annotation application. We would also like to note that the simulation developed here could be integrated into the MorphoCluster process as a fully automated pre-processing step to filter out simple, very homogeneous and well known classes. By detecting dense regions in the representation space of a target dataset merged with a labeled training dataset and assigning the respective majority training label (if present), the speed of MorphoCluster could be further increased. The common problems of classifier-based approaches  [22] , where outliers and objects from previously un-annotated classes are forced into the next best class, could probably be avoided. Such data would instead remain unlabeled, whereas common objects would be labeled automatically with high confidence.
Results As apparent in Figure  A5 , the time to predict grows proportionally with the feature dimensionality. The NCC-F1 saturates at around 32 dimensions for both datasets. While a further reduction to 16, 8 or 4 dimensions decreases the runtime by up to 24%, it also considerably impairs the quality of the image representations by up to 17 percentage points. A further increase to 64 dimensions only provides a small gain of 2. 50 percentage points while taking 21% longer. For ZooScanNet, even a reduction to 16 dimensions seems to be feasible without impairing the feature quality too much (impairment by 0. 20 percentage points). We conclude that the initial choice of d = 32 is justified. The SINGLE-STAGE model performs best when trained with a learning rate of 1 × 10 -4 which matches the learning rate selected for the full-dimensionality training.
It is specific to the instance discrimination task that we use for unsupervised training, but independent from training hyperparameters. Similar to the utilized MoCo training loss, it is based on two different views of the same object. We compare the representation of the raw, unaltered image to the representation of an image that was rotated and/or flipped randomly. For an individual object i, we define the per-object instance discrimination score as s(i) = b(i) -a(i) max{a(i), b(i)} (A2) where a(i) is the distance between the raw and the perturbed representation of an object i and b(i) is the distance of object i to its nearest neighbor. (This formulation is similar to the silhouette score  [88]  but serves a different purpose. ) The instance discrimination score is then the average of the s(i) for all objects in the whole dataset.
The instance discrimination score ranges from -1 to 1 and favors representation spaces where both views of an object are more similar than an object to its nearest neighbor, i. e. , representation spaces that are discriminative of individual objects while being transformation-invariant. These properties will facilitate the later unsupervised partition of the representation space into visually distinct and homogeneous clusters, just like its supervised equivalent NCC-F1 (Section 2. 4). For practical reasons, we use an approximation of the nearest neighbor distance b(i), where each object is compared only to a small sample of the whole dataset. (In our implementation, we process the query objects in batches of 1000 and draw 50 k new neighbor candidates for each batch. We did not observe any significant fluctuations despite the stochasticity of the approach.
Kaggle (National Data Science Bowl) Figure 2. Mosaics of plankton image datasets. Objects were randomly sampled from the datasets and arranged in an aesthetically pleasing manner. UVP5 (a) © Kiko and Schröder 2020, CC BY-NC [30]. ZooScanNet (b) © Elineau et al. 2020, CC BY-NC [31]. Kaggle (c) © Cowen, Sponaugle, Robinson and Luo, Oregon State University, 2015 [56]. Reproduced with permission from the authors.
Quality of supervised and unsupervised image representations measured as the test NCC-F1 on each target dataset for each condition and source dataset, respectively. The first rows show the test scores for full feature dimensionality for reference, the following rows show the results for different methods of dimensionality reduction to d = 32. All models were trained supervised, unless unsupervised training is indicated by UNSUP. Conditions where labeled target domain data were used for training (non-blind) are underlined. The best blind and non-blind result is bold, respectively, the second best is italic.
This research was funded by the  Federal Ministry of Education and Research (BMBF)  projects  CUSCO  (grant no. 03F0813D ) and  MOSAiC 3  (grant no. 03F0917B ). Rainer Kiko  was furthermore supported via a  Make Our Planet Great Again grant  of the  French National Research Agency  within the  Programme d'Investissements d'Avenir , reference  ANR-19-MPGA-0012. Institutional Review Board Statement: Not applicable.
Results The median runtime of a training run was 29. 41 h for ZooScanNet and 9. 43 h for UVP5/EcoTaxa. Figure  A4  shows the dependence of the validation NCC-F1 on under-sampling, loss, weight decay, learning rate and sample weighting for the UVP5/EcoTaxa and ZooScanNet datasets. Throughout all trials, the most robust settings, as assessed using the NCC-F1, are a learning rate of 1 × 10 -4 , a weight decay of 3 × 10 -3 , cross-entropy loss, undersampling to up to 1000 objects per class and no sample reweighting. The models with these hyperparameters are selected for the experiments.
INTRODUCTION Spatial point patterns are datasets containing random locations of events or objects within a spatial domain. These datasets arise in a broad range of applications, for instance in modeling locations of trees in a forest, locations of disease cases in a region, etc. (Baddeley et al. , 2015; Illian et al. , 2008; Møller & Waagepetersen, 2003). The stochastic models generating such datasets are called spatial point processes, with the spatial Poisson point process as reference model. This model is the natural candidate for modeling independent random structures, that is, points with no interaction. In practice, we may observe random patterns where points interact between them. There exists several models which are suitable for modeling these dependent random structures. Gibbs point processes (GPPs) are for instance one of them.
In Section 2, we present a brief formalism to define and characterize spatial GPPs. We also present a few examples. Methodologies to infer parametric GPPs are presented in Section 3 where we, in particular, review unregularized and regularized pseudo-likelihood. Asymptotic properties are given in Section 4. Section 5 details the methodology while providing its numerical aspects and Section 6 presents a simulation study while Section 7 is devoted to the application of the proposed method to a tropical forestry dataset. Discussion and conclusion follow in Section 8. Finally, proofs are postponed to Appendices S1-S3 in Data S1.
Now, the right-hand side of Equation (  19 ) corresponds to a weighted quasi-Poisson regression model with responses y i and weights  i , which can be performed easily using standard software. Note that the previous approximation performs well when g is large and in that case including the n data points yields a very small bias. This is exploited in the spatstat R package  (Baddeley et al. , 2015)  by using the ppm function with the default method, method="mpl. " In the situation where the number of points is quite large, using (  19 ) to fit Gibbs model can be computationally intensive. To get around this drawback, one can use the logistic composite log-likelihood function proposed by  Baddeley et al. (2014)  LCL(x; ) ∑ u∈x∩D log ( 𝜆 𝜽 (u, x) 𝛿(u) + 𝜆 𝜽 (u, x) ) + ∑ u∈d∩D log ( 𝛿(u) 𝛿(u) + 𝜆 𝜽 (u, x) ) , ( 20 ) where d is a realization of a dummy point process  independent of X with known intensity function (u).
Baddeley et al. (2014)  recommend to use a homogeneous stratified binomial point process or simply centers of a regular grid with number of points equal to four times the number of observed points). Conditional on x ∪ d, (20) corresponds to the logistic regression model with responses y(u) = 1{u ∈ x} and offset termlog (u). Thus, standard software for generalized linear models may also be used to implement this logistic fitting method; and this is provided in R by calling the ppm function with option method="logi" of the spatstat package. Baddeley et al. (2014)  establish several properties for the composite likelihood (20) and the estimators derived from it. In particular, it is established that the logistic likelihood should be viewed as an approximiation of the log-pseudo-likelihood, sometimes more stable numerically (see Section 6.
Simulation setup We conduct a simulation study to evaluate the performances (prediction and selection) of the regularized pseudo-likelihood estimator in an increasing domain framework, in which the total number of covariates increases as well as the size of the spatial domain. The spatial domains are W 1 = [0, 250] × [0, 125], W 2 = [0, 500] × [0, 250], and W 3 = [0, 1000] × [0, 500]. The setting for generating the spatial covariates is similar to that of  Waagepetersen (2007) ,  Thurman et al. (2015) , and  Choiruddin et al. (2018). We use the 201 × 101 pixel images of elevation (x 1 ) and slope (x 2 ) contained in the bei datasets of spatstat library in R  (Team, 2019)  as two true covariates. Note that we standardize these two covariates, that is, centered and scaled them. We first consider a Scenario 0, in which the covariates are only x 1 and x 2 , that is, for each domain W k (k = 1, 2, 3) the total number of covariates is set to p = 2.
Note that for the implementation in R of the fitting method via the ppm function of spatstat package involving the pseudo-likelihood function, we consider two choices regarding the number of dummy points nd 2 in term of the observed number of points n to control efficiently the bias induced by the Bermann-s-Turner approximation. In the presence of regular patterns like the two Strauss models, we take nd 2 ≈ 256n. For the clustered pattern, we consider the choice nd 2 ≈ 4n, as suggested in the spatstat R package. With these scenarios, we simulate 500 point patterns from each model within each spatial domain using the Metropolis-Hastings algorithm implemented in the rmh function of the spatstat package. For each pattern in Scenario 0, we fit the conditional intensity via pseudo-likelihood and assess performance by looking at prediction properties of the estimation method: the bias, the SD and the square root of mean squared errors (RMSE), which we define by Bias = ( p n ∑ j=1 { Ê( θj ) -𝜃 j } 2 ) 1 2 , SD = ( p n ∑ j=1 σ2 j ) 1 2 , RMSE = ( p n ∑ j=1 Ê( θj -𝜃 j ) 2 ) 1 2 where Ê( θj ) and σ2 j are, respectively, the empirical mean and variance of the estimates θj , for j = 1,…, p n where n = 1, 2, 3.
For Scenarios 1 and 2, we fit a regularization path to the simulated point patterns using modified internal function in spatstat, glmnet, and ncvreg. Note that in our setting, a modification of the ncvreg R package is required to include option for weights. The optimal model from each path as stated at the end of Section 5, is chosen on the basis of the minimizer of either cBIC or cERIC respectively defined by (  22 ) and (24). To evaluate d() given by (  23 ), we need to estimate matrices H and , which can be obtained using the vcov function of the spatstat package  (Coeurjolly & Rubak, 2013). This evaluation is very fast and stable numerically for pairwise interaction GPPs. For more complex models, such as the Geyer model (which depends on pairs of points, triplets, etc), the evaluation is extremely long in particular in the spatial domains W 2 and W 3 for which the number of data points is quite large.
APPLICATION TO A REAL DATASET In order to illustrate our approach, we consider the well-known data obtained from comprehensive census program conducted on the tropical rainforest of the Barro Colorado Island (BCI), Panama. Censuses are carried out on a site of 50 ha (W = 1000 m × 500 m) where all freestanding woody stems at least 1 cm diameter at breast height were identified, measured, and mapped. This results to a very rich dataset consisting of records of more than 350,000 individual trees of over 300 species  (Condit, 1998; Hubbell et al. , 1999 Hubbell et al. , , 2005)). The focus here is to analyze the locations of 3604 individual trees of the species Beilschmiedia pendula (Lauraceae). So, it seems interesting to know how this high number of trees profits from environmental factors such as topography, soil properties, etc.
One could be tempted to test the significance of these covariates but the problem of valid postselection inference is a hot and complex topic which has never been considered for spatial point process intensity or conditional intensity estimation, even for the simplest Poisson point process model. We leave this question for future research. In the same way, we have proposed to model this dataset using a high-dimensional inhomogeneous Geyer model. Judging the quality of the selected model is a question for which we currently have no clear answer due to the selection step. It may be difficult to interpret the estimate for Fe × Mn for instance as both Fe and Mn are not selected by adaptive lasso or elastic net methods. Such conclusions may also arise in Anova models however if we want to avoid this other penalty functions have to be considered (see Section 8).
DISCUSSION AND CONCLUSION In this paper, we develop regularization methods for inhomogeneous GPPs via PPL and PLCL approaches. We impose convex as well as non-convex penalties on the composite likelihoods in order to perform variable selection and parameter estimation simultaneously in a setting where the number of covariates diverges as the volume of the observation window increases. The implementation in R of our procedure is quite simple as it consists to implement penalized generalized linear models. This is done in R by combining spatstat package with glmnet and ncvreg packages. We propose a new composite information criterion, cERIC, which extends classical composite information criteria to account explicitly the effect of the tuning parameter. Simulations show that cERIC outperforms a competing criterion, cBIC for choosing the tuning parameter; and that PLCL outperforms PPL in terms of selection properties and predictive performance.
These results are the object of Theorem 2 in the present paper and also hold in an unregularized setting (Theorem 3), that is the pseudo-likelihood estimator for a non-stationary exponential Gibbs model is consistent and asymptotically normal. Adaptive lasso, adaptive elastic net, SCAD and MC+ are among the only regularization methods we consider which satisfy Theorem 2(ii)-(iii). We apply our procedure by fitting a Geyer saturation model to the B. pendula data obtained from census programs on the Barro Corrolado Island via PLCL combined with adaptive lasso and adaptive elastic net penalties. In this dataset analysis, we include 78 covariates based on interactions between two soil nutrients and the results may not be easy to interpret. A perspective of this research could be to extend the methodology and results to methods such as group Lasso, which may be more appropriate if one wants to select an interaction between two covariates only if the two covariates are also selected.
ACKNOWLEDGMENTS We thank the editor, associate editor, and the two reviewers for their constructive comments which improved the quality of this paper. The research of  I. Ba  and  J. -F. Coeurjolly  are respectively supported by the  Natural Sciences and Engineering Research Council of Canada  and the  French National Research Agency  in the framework of the " Investissements d'avenir" program  ( ANR-15-IDEX-02 ). J. -F. Coeurjolly would like to thank  Université du Québec à Montréal  for the excellent research conditions he received these last years. The BCI soils datasets were collected and analyzed by  J. Dalling ,  R. John ,  K. Harms ,  R. Stallard  and  J. Yavitt  with support from  NSF   DEB021104 , 021115 ,  0212284 , 0212818  and  OISE 0314581 , and  STRI Soils Initiative  and CTFS and assistance from  P.
The nudging is implemented by computed the current zonally averaged velocity profiles at the coarsest level J min = 5, and then interpolating the required nudging to each active grid point. The initial geostrophically balanced density and zonal velocity profiles are shown in Figure  7. The velocity magnitude is about 3. 5 times larger than in  Soufflet et al. (2016)  due to the more intense horizontal density gradient in the narrower channel. The spherical geometry and longer zonal channel length also mean the results differ quantitatively from  Soufflet et al. (2016). WAVETRISK-OCEAN was run on 160 cores of the compute canada machine niagara. To spin up, the code was first run non-adaptively at resolution J = 5 for 300 days and then restarted from the checkpoint with the four additional adaptive levels and a relative tolerance ε = 0.
The green (land) regions use the coarsest grid. To illustrate the effect of a larger tolerance, figure  9  shows the results with ε = 0. 06. The grid is far more compressed, with large areas requiring only the coarsest grid. Overall, the ε = 0. 06 case uses about half as many grid points than the ε = 0. 02 case, while still capturing the intense small scale structures in the density and vorticity fields. For simplicity, energy spectra are computed from saved vorticity checkpoint data interpolated to fill a fine level of resolution (e. g. J max or J max -1). This non-adaptive spherical data on a non-uniform hexagonal grid is then projected onto a uniform longitude-latitude grid of equivalent resolution. The spherical harmonics energy spectrum is then computed from the latitudelongitude data using the spherical harmonics toolbox SHTOOLS  (Wieczorek and Meschede, 2018).
In addition to global energy spectra, SHTOOLS also allows the computation of local energy spectra associated to specified sub-regions of the sphere. Figure  10  shows the spherical harmonic energy spectrum computed from the vorticity field shown in Figure  8. There is a power law range of approximately k -2 extending over about a decade, from scales of about 25 km to 130 km. This scaling is similar to the results for submesoscale turbulence from  Soufflet et al. (2016) , who find a slope of about -2 from scales of about 20 km to 80 km for a resolution ∆x =2 km. This paper introduced WAVETRISK-2. 1, or WAVETRISK-OCEAN, the version of the dynamically adaptive code WAVETRISK developed specifically for global ocean modelling. The dynamical equations of WAVETRISK-OCEAN are a multi-layer rotating shallow water model with inhomogeneous density layers, but with no vertical variation of velocity and buoyancy within each layer.
Acknowledgements. NKRK acknowledges an  NSERC   Discovery grant  and computer time from  Compute Ontario  (computeontario. ca)and  Compute Canada  (www. computecanada. ca). Alistair Adcroft provided valuable advice during the initial development of WAVETRISK-OCEAN. We are grateful to  Thomas Dubos  and  Matthias Aechtner  for their essential contributions to the WAVETRISK model on which WAVETRISK-OCEAN is based. 505 27 https://doi. org/10. 5194/gmd-2021-365Preprint. Discussion started: 13 December 2021 c Author(s)  2021. CC BY 4. 0 License.