dataesr/scientific-paragraphs-categorization huggingface dataset:
However, the unexplained differences between the backscatter results processed by the tools that are widely used by scientists is a concern shared by end-users of backscatter data, agencies funding data acquisition and processing; and software solution providers  [34, 35]. Hook and Kelley  [33]  identified a lack of quality control and means of comparing software output to expected and correct results as a critical challenge to assess a software package. The current study compares some of the intermediate processing results of non-transparent processing chains in an attempt to highlight which parts of these processing chains differ the most. Only four software solution providers participated in this study, but it is expected that future efforts will include other software packages. One very positive development has been that through this study and the cooperation of the software manufacturers, each of the three commercial software packages that were studied (QPS, CARIS, and SonarScope) now have the functionality to export intermediate results that will enable future end-users to be able to assess the processing chain themselves.
Why Do Different Approaches to Reading Raw Data Exist and Which One is Correct? The results indicate that the raw data in the form of seabed image/snippets is read differently by various software to create what is termed as 'beam averaged backscatter' and was referred during this study as BL 0. The impetus to compute beam averaged backscatter value stems from the need to reduce the statistical uncertainty of seafloor backscatter  [6, 15]. Through the commercial development of MBES, different approaches have been taken for the collection and provision of backscatter data, and these differences may offer some explanation for the discrepancies found. For instance, historically, the approaches taken to compute a single representative value per beam from recorded snippets differed based on: 1. Choice of central tendency, i.
Metadata also has an essential role in providing information to end-users (e. g. , a geologist interpreting seabed sediment type) who may not be actively involved in, or have an in-depth knowledge of, backscatter processing yet whose perception of the data is influenced by the data provenance from acquisition through processing. The development and implementation of a standard metadata format for backscatter data products by the community (involving sonar manufacturers, software developers, and the users of this hardware and software across industry, academia, and government organizations) should, therefore, be a priority.
However, the data from these intermediate processing stages are currently not produced consistently by all the software developers. Therefore, the active participation of software developers will be critical to make appropriate changes in the software packages to enable the export of results from intermediate processing stages while expanding this approach to other software packages. Two intermediate processing levels were assessed during this study: the level read from the raw data files (BL 0 ) and the level after radiometric corrections but before the removal of angular dependence (BL 3 ). Software developers applied the required changes in their processing methodologies and provided data in Beam -Ping configuration with BL 0 and BL 3 reported for each beam along with incidence angle. Both BL 0 and BL 3 showed differences as high as >10 dB between the software packages.
Engine: SIPS; Source Data Type: Time series; Slant Range Correction, Beam Pattern Correction; Angular Variation Gains, Adaptive; AVG size filter, 200 samples. As of the release of the CARIS SIPS 11. 1. 3 (released March 2019), end-users have the ability to export the intermediate processing stages utilized in SIPS processing engine accessed through 'Advanced Settings' and by designating a 'Corrections Text Folder' where an ASCII file is stored that contains results of intermediate processing stages. (Figure 2).
The digital technologies developed in recent decades have considerably enriched the survey and documentation practices in the field of cultural heritage. They now raise new issues and challenges, particularly in the management of multidimensional datasets, which require the development of new methods for the analysis, interpretation and sharing of heterogeneous data. In the case of rock art sites, additional challenges are added to this context, due to their nature and fragility. In many cases, digital data alone is not sufficient to meet contextualization, analysis or traceability needs. In this context, we propose to develop an application dedicated to rock art survey, allowing 3D annotation in augmented reality. This work is a part of an ongoing project about an information system dedicated to cultural heritage documentation.
Information systems and cultural heritage Knowledge-centered approaches aim to improve content data management by defining formal structures that describe the implicit and explicit relationships of concepts linking elements together  (Doerr 2009)  Finally,  (Manuel et al. 2018 ) focuses on merging geometric, visual and semantic aspects into a single integrated documentary approach, with the objective of offering through a cloud platform an information continuum that merges the acquisition, analysis, and interpretation phases in a collaborative framework. This approach relies on an indexing phase following the photogrammetric process, which establishes a 2D/3D projective relationship by keeping in memory the link between the pixels of the different photographs in a dataset and the points of the resulting point cloud.
The user must then manually set the scale using a raycast tool to pick two points from the point cloud and indicate the distance between them. At the end of this step, the project's 3D database is ready for annotations but also for augmented reality. At each T n , i. e. at each use session of a project, the augmented reality visualization mode can be activated from the Figure  2 : Overall approach for the registration of an Aïoli project. On the left, project creation at T 0. On the right, AR registration process at T n , based on incremental photogrammetry. project window. From this moment, through SLAM, we get in real time the estimates of the successive poses of the camera of the user device's video stream, and possibly a sparse point cloud according to the SDK used for the experiment a.
During the spatial referencing process, the intrinsic and extrinsic parameters of each camera are written to a JSON a As this feature is not currently integrated into all SDKs made available through the WebVR or WebXR APIs, we have chosen not to rely on it for now. file. This file allows us, at the viewer level, to generate equivalent virtual cameras, which are used to immerse the images in 3D space and navigate through all the acquisition dataset while interacting with the point cloud. Each photograph is visualized by applying it as a texture of a plane geometry, located on the near plane of the corresponding cameras frustum (Figure  3 ). In terms of representation, images of the visible spectrum are often insufficient to visualize certain characteristics of the object of study, particularly in the case of rock art sites where environmental conditions and natural wall degradation can make observation conditions very difficult.
For our very first experiment, we conduct tests on different datasets of six pictures of the same facsimile of the Chauvet-Pont dArc cave. Projects are created using these photographs, and annotations are made upstream in the desktop version of Aïoli. We then switch to the mobile version to turn on the AR mode. In AR, all content of the project's 3D scene is rendered on top of the camera's video stream, and user controls are initialized to synchronize the movements of the camera with those of the device. At this point, the position of the scene is not coherent with reality. When the user starts the alignment, a frame is extracted, as a base 64 string and sent to the server for calibration. For this calculation, we can distinguish two cases: if the photograph to be oriented has useful metadata, the calculation can be based on the common Bundle Adjustment method (Deseilligny and Cléry 2011).
Several factors can influence this duration, mainly the definition of the new image, and its proximity to those of the initial acquisition set. Obviously, this calculation time is far too long to consider a comfortable use, but nevertheless reasonable for a first proof of concept. Moreover, even if the computation time is too long to consider a systematic use, the notion of anchor persistence allows to consider realistic operational scenarios, and interesting collaborative perspectives. By completing the alignment process by saving the space-mapping state and some key anchors from the world-tracking session, the alignment time for all subsequent AR sessions can be reduced to the loading time of the WorldMap containing all this information from the project database. In this way, our proposed registration procedure by spatial resection constitutes in fact a pre-calibration of the project that can benefit all the following sessions.
Finally, different approaches described in  (Bogoslavskyi and Stachniss 2017 ) could be used to analyze the quality of matched 3D point clouds. Nevertheless, several issues stand in the way of this perspective. The first one is the nature of the feature points extracted by ARKit. These points represent notable characteristics detected in the camera image during the session. As indicated by Apple c in the documentation, "Their positions in 3D world coordinate space are extrapolated as part of the image analysis that ARKit performs in order to accurately track the device's position, orientation, and movement. Taken together, these points loosely correlate to the contours of real-world objects in view of the camera. ARKit does not guarantee that the number and arrangement of raw feature points will remain stable between software releases, or even between subsequent frames in the same session.
It also relies on the presence of a calibration source sufficiently close to the entrance of the instrument, which is the case in VLT/SPHERE, but even so the light will not see the telescope mirrors as there is rarely (if ever) an appropriate light source in the telescope itself. Although theoretically applicable, this strategy was deemed too risky for our first on-sky tests and we adopted the first approach instead. To carry out this demonstration, ESO awarded us three technical half-nights on 2018-04-01, 2018-04-02, and 2018-04-03 1. The observing conditions were unfortunately quite unstable on April 1 and 3, and very poor on April 2. In this paper, we report on the on-sky tests of NCPA calibration with ZELDA on VLT/SPHERE with the acquired data on the first and third halfnights. We first detail our implemented improvements in the wavefront error calibration and compensation with our Zernike sensor in the instrument compared with Paper II.
Finally, we also implemented several improvements to the ZELDA analysis code. Based on the IDL code that was originally developed for Paper I, we developed a new Python-based generic code called pyZELDA that is dedicated to the analysis of data acquired with Zernike WFS  (Vigan & N'Diaye 2018). This code is public and freely available under the MIT license 2. It is easily extensible to include new instruments or laboratory test beds. All these improvements make NCPA calibration and compensation with ZELDA much more robust. As a result, NCPA calibration has been transposed into a calibration template that is executed in the SPHERE daily calibration plan to monitor the long-term evolution of NCPAs. The stability of the aberrations in SPHERE has begun to be explored in the laboratory  (Martinez et al.
From here on we use a fixed value for β but keep in mind that the value could in fact vary as a function of the spatial frequency. We see further evidence of this possibility in Sect. 4. 3. In an attempt to explain the loss of sensitivity, we performed a numerical simulation of a ZELDA NCPA reconstruction based on realistic data acquired with the instrument. The simulation is designed to model measurements taken either on the internal source or on sky. For the on-sky simulation, we use reconstructed ExAO residuals computed from SAXO real-time telemetry data over 30 s (41 400 phase screens; see Appendix B). The average residual wavefront error of the generated phase screens is 95. 9 ± 12. 4 nm rms, corresponding to a Strehl of 87% ± 3% at 1. 6 µm, which is a good match to the observing conditions encountered during our tests in Paranal and the typical SAXO performance.
Description of the tests Beyond the pure on-sky NCPA compensation, the goal of our tests was to demonstrate a quantitative contrast gain in raw coronagraphic images once the compensation is applied. For this purpose, we acquired coronagraphic data following the ZT01, ZT02, ZT03, and ZT04 tests presented in Table  1. The complete data logs for these tests are provided in Table  2. We used the SPHERE apodized-pupil Lyot coronagraph (APLC;  Soummer 2005; Carbillet et al. 2011; Guerri et al. 2011 ) and all data were acquired with the IRDIS instrument  (Dohlen et al. 2008 ) in dual-band imaging (DBI;  Vigan et al. 2010 ) mode. To present the results we focus on the images in the H2 filter of the H23 filter pair, at a wavelength of 1. 593 µm and with a width of 52 nm. For each coronagraphic test, we acquired a set of coronagraphic images on-axis, followed by an off-axis PSF taken with a neutral density filter, and finally an instrumental background obtained by either switching the lamp off (internal source) or applying a 30 telescope offset (on sky).
64 × 1/0. 8 = 1. 95 in the reconstruction. While this factor seems appropriate for the low-spatial frequencies, the value does not seem appropriate at the level of the AO cutoff frequency. This again points towards a value that would vary as a function of spatial frequency. For this first study, provided that the predictions of our simulations appear correct at small separation, we keep a fixed value for the attenuation factor. However, further study would be required to fully understand the behavior of the sensitivity loss factor. Other than the atmospheric residuals, the main change on sky is the VLT pupil. The impact of this pupil is significant on the theoretical APLC raw contrast, with the apparition of a bright peak at 150 mas as well as secondary peaks in the 200-400 mas range. In the 100-400 mas range, the data after NCPA compensation are obviously limited by these peaks.
2006). This approach presents the advantage of enabling the decomposition of the full AO-filtered PSD into its individual terms, namely the fitting error, the servo-lag error, the aliasing error, the noise error, and the differential refraction. In the case of our reconstruction, we included the stellar magnitude, zenith distance, and azimuth as input parameters. For the seeing and C 2 n we used the ESO Paranal ambient database to access the data from the MASS 3   (Kornilov et al. 2007 ), and we estimated that more than 55% of the turbulence was contained in a ground layer at altitude z = 0 km, 35% in a layer at z = 4 km, and the final 10% in a high-altitude layer at z = 16 km. The wind speed and direction of the ground layer were set in the simulation using the values that are also reported in the ambient database.
The Fourier code was then used to generate a PSD that only includes the fitting error and the aliasing error. For the aliasing, an attenuation factor of 0. 5 was used to take into account the fact that SAXO used a spatially filtered Shack-Hartmann wavefront sensor and that the spatial filter was in the MEDIUM position, which only provides a partial attenuation of the aliasing. Using the PSD, we finally generate a series of N ts random OPD maps (OPD Fourier ) that are added to the OPD maps reconstructed from the SAXO real-time telemetry: OPD ExAO = OPD SAXO + OPD Fourier. (B.
Acknowledgements. The authors are extremely grateful to  ESO  for granting them VLT technical night-time for these tests and to the  Paranal observatory staff  for their support. They also thank the referee for his insightful comments that greatly helped to improve the paper. This project has received funding from the  European Research Council (ERC)  under the European Union's  Horizon 2020 research and innovation programme  (grant agreement No. 757561 ). AV and MN acknowledge partial support from the  French national programme "Action Spécifique Haute Résolution Angulaire" (ASHRA). SPHERE is an instrument designed and built by a consortium consisting of  IPAG (Grenoble, France ),  MPIA (Heidelberg, Germany) ,  LAM (Marseille, France), LESIA (Paris, France), Laboratoire Lagrange (Nice, France) ,  INAF -Osservatorio di Padova (Italy) ,  Observatoire de Genève (Switzerland) ,  ETH Zürich (Switzerland) ,  NOVA (Netherlands) ,  ONERA (France)  and  ASTRON (Netherlands)  in collaboration with  ESO.
SPHERE was funded by  ESO , with additional contributions from  CNRS (France) ,  MPIA (Germany) ,  INAF (Italy) ,  FINES (Switzerland)  and  NOVA (Netherlands). SPHERE also received funding from the  European Commission Sixth and Seventh Framework Programmes as part of the Optical Infrared Coordination Network for Astronomy (OPTICON)  under grant number  RII3-Ct-2004-001566  for  FP6  ( 2004-2008 ), grant number  226604  for  FP7  ( 2009-2012 ) and grant number  312430  for FP7 (2013)(2014)(2015)(2016).
In biomedical signal processing, we often face the problem of artifacts that distort the original signals. This concerns also sleep recordings, such as EEG. Artifacts may severely affect or even make impossible visual inspection, as well as automatic processing. Many proposed methods concentrate on certain artifact types. Therefore, artifact-free data are often obtained after sequential application of different methods. Moreover, single-channel approaches must be applied to all channels alternately. The aim of this study is to develop a multichannel artifact detection method for multichannel sleep EEG capable of rejecting different artifact types at once. The inspiration for the study is gained from recent advances in the field of Riemannian geometry. The method we propose is tested on real datasets.
In our study, we assume that the number of clusters for clean data may be superior to one, and we introduce an unsupervised and adaptive method for constructing the clusters. It allows us to achieve better performance for complex data such as sleep EEG, where we cannot assume a unique cluster because the spatial patterns vary widely across sleep stages and because sleep data within the same stage may be captured more accurately by several clusters. Different sleep stages have their own characteristics. In the present study, we investigate the proposed method and test it on real sleep EEG data. Performance is evaluated by comparing the obtained detection to a provided expert scoring. Results are examined in contrast to the outcome of the (one-cluster) Riemannian potato artifact detector  [24]  and the fully-automated statistical thresholding for EEG artifact rejection method (FASTER)  [25] , which performs artifact detection by analyzing statistical properties in separate channels.
Dreams Dataset The open-source Dreams dataset  [26]  contains 15-min recordings of sleep data, 20 recordings in total. Channels originally referenced to A1 were re-referenced to Cz. All available EEG channels O1-Cz, O2-Cz, Fp1-Cz, and Fp2-Cz were used in the study. The sampling frequencies were 50 Hz, 100 Hz, and 200 Hz. We excluded from analysis the recordings sampled at 50 Hz due to the fact that with such a sampling rate, the frequency band-pass is too restricted for sleep data analysis  [3]. Furthermore, we excluded recording Numbers 3, 6, 10, and 16 because the proportion of artifacts exceeded 40%. Sleep stage scoring and artifact labeling have been performed by a trained clinician and provided, as well. Each recording contains a mix of sleep stages. The distribution of the sleep stages in the dataset is: wakefulness 33%, REM 25%, non-REM 1 18%, non-REM 2 23%, and other 1%.
InSleep Datasets The InSleep datasets were obtained from the National Institute of Mental Health, Czech Republic. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of the National Institute of Mental Health (Project Number 6/15). The whole-night EEG as a part of the polysomnography (PSG) was recorded from subjects suffering from insomnia. Each recording was obtained using 19 channels placed according to the 10-20 system: Pz, Cz, Fz, T6, T5, T4, T3, F8, F7, O2, O1, P4, P3, C4, C3, F4, F3, Fp2, Fp1. Channels were referenced to an average of Cz, Fz, and Pz. The sampling frequency was 250 Hz. Sleep stages were scored by a trained clinician. Each recording contained 10-20 min of data corresponding to a single sleep stage (REM, non-REM 2, or non-REM 3) or awake activity.
The Riemannian Potatoes' Artifact Detection A Riemannian potatoes artifact detection method (RPs) is proposed in this study. It relies on the assumption that on the SPD manifold, contaminated epochs are mapped far away from clusters of clean data. In contrast to the simple potato  [24] , here we take into consideration the fact that several clusters may be needed to describe the distribution of clean sleep data. In the first step, RPs restore these clusters from a recorded EEG. In the second step, we obtain continuous scoring by analyzing the data in a sliding window and comparing them to the obtained clusters. Finally, artifacts are detected by applying a threshold. The method overview is depicted in Figure  2. As in  [24] , we filtered the data before further processing. However, we used a low-pass filter under 30 Hz to save frequencies significant for sleep stages  [3].
The cluster construction process starts with splitting the filtered data into non-overlapping 1-s epochs. Such an epoch length is a standard for Riemannian geometry analysis  [24]. Covariance matrices are extracted from every epoch. Then, distances between all pairs of covariance matrices are calculated by Equation (2). The average distance d i is the mean value of distances from the ith epoch to all the other epochs. Epochs with a d value greater than a threshold are removed from further analysis. We consider the mean value of d as the threshold. Then, we cluster the remaining matrices using the k-means algorithm with the distance measure defined by Equation (2). To determine the number of clusters, we use the following procedure: the number of clusters increases iteratively from one to a maximum of 10; centroids are obtained as the geometric means; the process is finished once the constructed clusters satisfy an overall normality condition, that is when the distribution of standardized distances to the centroids is normal in all clusters.
Results Our RPs artifact detection method was implemented in Python and applied to the Dreams and InSleep datasets. The Riemannian potato  [24]  method was chosen as the benchmark. We used the original implementation of the Riemannian potato artifact detector provided by the authors in the covariance toolbox for MATLAB (github. com/alexandrebarachant/covariancetoolbox). Visual comparison of scoring obtained by RPs and by the benchmark is provided in Figures  5  and 6. Detection results were compared with scoring provided by an expert, and the three statistical metrics presented in the previous section were computed as well as percentage of data labeled as artifacts by methods. Each recording of the Dream dataset contains a mix of sleep stages and awake EEG. Some of the presented sleep stages have a short duration and do not include artifacts.