dataesr/scientific-paragraphs-categorization huggingface dataset:
DEMONSTRATION OVERVIEW The audience will be able to use DiscoPG to execute the following steps. In the static setting, one can: load and visualize property graphs, configure the parameters of our GMM-S schema discovery method, as well as log and analyze its performance, in terms of runtime and clustering quality, across various datasets. In the dynamic setting, one can modify the initial graph instances, by inserting real or synthetic nodes. Then, one can analyze the updated schemas produced with our dynamic algorithms. These are I-GMM-D (which incrementally adjusts the clusters computed in the static case) and GMM-D (which dynamically recomputes the node clusters). Finally, one can explore the discovered schemas and visualize their structure: relative cluster sizes, associated labels, properties, and interconnections, as well as the impact incurred due to updates.
Schema Discovery Static Case. Users can access DiscoPG through a GUI. The main panel allows them to configure the schema discovery parameters of GMM-S. As depicted in Figure  2 , a graph instance can first be selected from a menu containing the LDBC  [5] , the Covid19 graph  [6] , and the Connectome NeuPrint's Fib25  [11]. Note than further datasets can also be added to the system. DiscoPG then supports schema discovery for either the entire dataset or for a custom fraction that can be specified in the field "Percentage of nodes to consider". In the latter case, the remaining percentage will be used for the dynamic setting. Users can also specify the number of subclusters to be discovered at each iteration (i. e. , the number of fitted Gaussians, as explained in Section 2. 1. 1). Users can choose whether the discovered schema should include the original edge labels or only the computed subtype relationships, as well as whether to record the evaluation in a log, for further reference.
Finally, if a previous schema discovery algorithm has already been run on the chosen dataset, they can consider the most recent result. This can then be processed with the dynamic schema discovery algorithms. Dynamic Case. As real-world datasets are continuously evolving, DiscoPG allows users to modify the graph instance used for static schema discovery. Configuring the parameters for the dynamic scenario is possible through a dedicated panel (see Figure  3 ). Users can first choose between the I-GMM-D and GMM-D algorithms for dynamic schema discovery. Next, they can specify the number of nodes to be inserted and whether these should correspond to real data. In both cases, the update batch is built from the sample G  retained for updates, ((100 -)% of G, where  is the percentage of data used for static schema discovery).
If the user wants to use real data, DiscoPG will randomly select the desired number of nodes from G  and add these to the graph. If not, DiscoPG will construct the required number of synthetic nodes, by first randomly selecting labels from G  and, for each, further sampling its set of applicable properties. Users have the option to also log the evaluation. Performance Metrics. For each logged algorithm, its execution time per iteration is plotted (see Figure  4 ). Users can zoom in on the portions of the plot they find the most relevant and analyze the comparative behaviour of the algorithms. Figure  4  illustrates this on the LDBC dataset. We see that the time required to discover new clusters at most iterations is under a few microseconds. The rare jumps in execution time correspond to the computation of larger clusters and primarily concern the GMM-S algorithm, responsible for producing the schema in the static case.
The corresponding plots allow users to visually grasp which algorithm is best suited for schema discovery on the chosen dataset. As they can readily visualize the minimum number of iterations already providing a schema of satisfactory quality, the information can be used for further optimizations. On the LDBC dataset (see Figure  5 ), we notice that the quality of the schema discovered by GMM-S improves with the number of iterations. This trend is also recorded when considering the GMM-D algorithm, all the while the convergence rate is much faster, as already observed when analyzing its runtime performance. For the incremental I-GMM-D algorithm, we note that it conserves cluster quality, due to its incremental nature, thus providing a more robust alternative to GMM-D.
Schema Exploration The Schema Exploration module enables users to navigate the structure and evolution of the discovered schemas. In the static case (see Figure  6 ), users can explore the graph structure of the dataset they have chosen in the Schema Discovery module (see Section 3. 2). For example, when considering the LDBC dataset, as illustrated, users can visualize the clusters corresponding to each of the discovered node types and their labeled inter-connections. Note that the latter are produced based on the information regarding the edges attached to the cluster nodes. For each of the clusters, users can inspect their properties and visualize their respective sizes, in order to understand the relative number of nodes they contain. In the dynamic case, users can visualize the results of the I-GMM-D or GMM-D algorithms (Figure  7  and Figure  8 ).
We present the first public release of SHAPEPIPE, an open-source and modular weak-lensing measurement, analysis, and validation pipeline written in Python. We describe the design of the software and justify the choices made. We provide a brief description of all the modules currently available and summarise how the pipeline has been applied to real Ultraviolet Near-Infrared Optical Northern Survey data. Finally, we mention plans for future applications and development. The code and accompanying documentation are publicly available on GitHub.
2019) , will also aim to tighten cosmological constraints with weak lensing. It is clear that weak lensing remains at the forefront of cosmological studies, and hence tools for weak-lensing analysis will remain in demand for years to come. There are various pipelines available for simulating weaklensing data (e. g. Kiessling et al. 2011; Harnois-Déraps & van Waerbeke 2015; Petri 2016). Various data processing pipelines also exist (e. g. Bosch et al. 2018) , but these are generally very survey-specific and often private. We believe this leaves an ample gap in the market for an open-source end-to-end weak-lensing pipeline. We present the first public release of SHAPEPIPE, an opensource and modular weak-lensing measurement, analysis, and validation pipeline written in Python. An earlier version of this pipeline was applied to real survey data in  Guinot et al.
(2022)  with promising results. The current version of SHAPEPIPE starts with reduced survey images and ends by providing shear measurements along with all of the information required for calibration. It includes a novel point spread function (PSF) modelling technique and various PSF validation tools. The code has been designed to facilitate the inclusion of new or improved processing steps to adapt to advances made in the coming years. This paper is structured as follows. In the following section we describe the design of the software along with efforts we have made to ensure reproducibility and to encourage external contributions. In Sect. 3, we describe the various processing elements that comprise the current version of the pipeline. In Sect. 4, we summarise how the pipeline has been implemented thus far and what we have in mind with regards to future applications.
Modular architecture A key feature dictating the design of SHAPEPIPE was flexibility. Weak gravitational lensing is a rapidly evolving field, with new analysis techniques and tools emerging regularly. A hard-coded weak-lensing pipeline would therefore have extremely limited applications and would become quickly outdated. It was therefore essential for us that a new pipeline be able to adapt in a way that was both maintainable and allowed the easy inclusion of new tools. The SHAPEPIPE package comprises two principle subpackages: pipeline and modules. The pipeline sub-package manages job handling, logging, configuration, argument parsing, dependency handling, and the reading and writing of Flexible Image Transport System (FITS) files. This can effectively be seen as the core of SHAPEPIPE. The modules sub-package consists of a series of what we refer to as 'pipeline modules' 1 , which handle all of the data processing and analysis steps that make up a weak-lensing pipeline (see Sect.
3). Pipeline modules are simply defined by writing a 'module runner', which is a standardised SHAPEPIPE script that defines what a given module should do. This could be as simple as calling an executable binary with some options or writing a full implementation in Python. In either case, once a module runner is in place, SHAPEPIPE can call this module in conjunction with any other. Inputs can be automatically passed or taken from modules, making it possible to chain a series of operations with relative ease. This architecture will allow us to quickly add new modules to SHAPEPIPE as new tools come onto the market without disrupting the existing pipeline structure. It also means we can more easily rearrange or adapt the existing tools to new datasets. SHAPEPIPE pipelines are launched via a command line execution script.
Parallel computation SHAPEPIPE was developed with the assumption that most weaklensing survey data can be divided into tiles or patches, each of which can be processed independently. Therefore, SHAPEPIPE distributes jobs in an embarrassingly parallel way using JOBLIB (Joblib Development Team 2020) for multi-core machines, and MPI FOR PYTHON  (Dalcin et al. 2005 (Dalcin et al. , 2008 (Dalcin et al. , 2011) )  on large computer clusters. This maximises the amount of data that can be processed simultaneously with the number of computer cores available to the user without having to parallelise each of the processing steps independently. Modules can also be configured to run in serial mode if the associated task requires access to all of the data, for example, for recombining tiles or patches.
Reproducibility The ability to reproduce scientific results is a growing and continuing concern within academia. This is a particularly challenging problem in the context of a weak-lensing pipeline as there are numerous data treatment and processing steps, often involving various third-party tools that evolve over time. We have attempted to mitigate these problems as best as possible. Firstly, we provide a dedicated SHAPEPIPE development and processing environment. This consists of a CONDA 2 environment where all Python packages, compilation tools (such as CMAKE), and other third-party software packages (see Appendix A) have fixed versions for a given SHAPEPIPE release. This also extends to all of the modules that make up the weak-lensing pipeline (see Appendix B). This not only ensures consistency across different platforms on which SHAPEPIPE is installed, but also isolates the tools used for processing from those already installed on a given platform (e.
g. a different version of Python). Secondly, we have built in an extensive logging system that tracks the versions of all the packages and modules used for 1 Not to be confused with Python modules. 2 https://docs. conda. io/ a given run of SHAPEPIPE, along with all the configuration options used. The logging system catches and reports all errors raised by each module run. These measures should allow users to reprocess a given dataset in virtually identical conditions. However, we will continue to make reproducibility a priority for future releases of SHAPEPIPE.
Open-source development SHAPEPIPE is a fully open-source project. All of the code is publicly available on GitHub 3  along with extensive documentation 4. We provide example configuration files demonstrating how to use the various tools in the context of our application use cases (see Sect. 4), and we additionally provide an extensive step-by-step tutorial for running SHAPEPIPE for processing Ultraviolet Near-Infrared Optical Northern Survey (UNIONS) data (see Sect. 4. 1). The GitHub repository is configured with template issues and pull requests, and includes detailed contribution guidelines. These features are intended to encourage and facilitate contributions from the wider community. Finally, we have also included a comprehensive code of conduct to ensure that contributors can interact in a safe and supportive environment.
Masking The current masking procedure in SHAPEPIPE takes inspiration from the THELI pipeline  (Erben et al. 2005 ; Schirmer 2013) to mask out bright stars, diffraction spikes, Messier objects, and border regions. The mask module uses the CDSCLIENT software to download a catalogue of bright reference stars from version 2. 2 of the Guide Star Catalog (GSC; Space Telescope Science Institute & Osservatorio Astronomico di Torino 2001). Alternatively, a star catalogue available on disk (with the same format as the GSC) can also be used, which is particularly relevant for cluster compute nodes that do not allow internet access. The mask itself is generated using WEIGHTWATCHER  (Marmo & Bertin 2008 ). An example of a mask produced using this module on a patch of UNIONS data is shown in Fig. 1.
Source detection The identification of stars and galaxies in the survey images is performed using the community standard SEXTRACTOR  (Bertin & Arnouts 1996). SHAPEPIPE allows modules such as SEX-TRACTOR to be run multiple times with different configuration options, meaning that stars and galaxies can be targeted in different ways depending on the need. For example, one run can be made to target a large sample of galaxies for shape measurement, while another run can target a sample of stars for modelling the PSF.
Source selection The star and galaxy samples used for shape measurement can be refined using several SHAPEPIPE modules. One module implements the spread model method proposed by  Desai et al. (2012)  and  Mohr et al. (2012) , which tries to determine if a given object is more point-like or rather more extended in comparison with the PSF. Another module, called setools, can apply arbitrary cuts to the objects detected by SEXTRACTOR. Figure  2  shows some example plots of a refined star sample obtained after running the setools module in SHAPEPIPE.
PSF model SHAPEPIPE includes two independent methods for modelling the PSF of the survey. The first method is the community standard PSFEX  (Bertin 2011) , which was developed to be used in conjunction with SEXTRACTOR. PSFEX produces independent models with polynomial variations of the PSF for each charge-coupled device (CCD) in the field and each exposure. SHAPEPIPE includes an additional module for interpolating the PSFEX PSF model from the positions of the star sample to the positions of the galaxy sample. The second method is the multi-CCD (MCCD) approach of  Liaudat et al. (2021). MCCD builds a model of the PSF for the entire focal plane, capturing both variations that span several CCDs and those that are localised on individual CCDs. As with PSFEX, there is a SHAPEPIPE module for interpolating MCCD PSF models to galaxy positions.
Shape measurement Galaxy shape measurement is currently performed using NGMIX  (Sheldon 2015) , which has been extensively applied and tested on DES data  (Jarvis et al. 2016; Zuntz et al. 2018; Gatti et al. 2021). At present, NGMIX is implemented in SHAPEPIPE such that galaxy shapes are obtained from a fit of a Gaussian profile from Galsim  (Rowe et al. 2015). The NGMIX module can be run with either of the PSF modelling methods described in Sect. 3. 4. Additional NGMIX features and alternative shape measurement techniques are likely to be made available in future releases of SHAPEPIPE.
Other surveys No concrete plans have been made at this stage for applying SHAPEPIPE to other datasets. Despite designing the architecture to be as flexible as possible, significant work would still need to be carried out to appropriately tune all of the processing steps to a different survey. Large cosmological surveys such as the DES, Euclid, and the Vera C. Rubin Observatory LSST already benefit from dedicated weak-lensing analysis pipelines highly tuned to the specific needs of each survey. However, independent reprocessing of some fraction of these datasets could be valuable for cross-validation and comparison. Additionally, some smaller surveys lack the workforce necessary to build a new weak-lensing pipeline, or indeed to adapt an existing pipeline to their data. Either or both of these use cases might be well suited for future SHAPEPIPE applications.
Conclusions In this paper, we have presented the first public release of the open-source and modular weak-lensing measurement, analysis, and validation pipeline SHAPEPIPE. We have described the objectives and design of the software. In particular, we have emphasised the flexibility of the code architecture, which was designed with the intention of facilitating the inclusion of new processing steps or the improvement of existing ones. This was done with the aim of keeping up with advances made in the field and the possibility of applying SHAPEPIPE to diverse datasets. We have also mentioned steps taken to ensure the reproducibility of results obtained with SHAPEPIPE. We have provided a brief summary of the tools currently available for masking, source selection, PSF modelling, PSF validation, shape measurement, and calibration.
Acknowledgements. This work is based on data obtained as part of the  Canada-France Imaging Survey , a  CFHT large program  of the  National Research Council of Canada  and the  French Centre National de la Recherche Scientifique. Based on observations obtained with  MegaPrime/MegaCam , a joint project of CFHT and CEA Saclay, at the  Canada-France-Hawaii Telescope (CFHT)  which is operated by the  National Research Council (NRC)  of  Canada , the  Institut National des Science de l'Univers (INSU)  of the  Centre National de la Recherche Scientifique (CNRS) of France , and the  University of Hawaii. This research used the facilities of the  Canadian Astronomy Data Centre  operated by the  National Research Council of Canada  with the support of the  Canadian Space Agency. This research is based in part on data collected at Subaru Telescope, which is operated by the  National Astronomical Observatory of Japan.
We are honored and grateful for the opportunity of observing the Universe from  Maunakea , which has the cultural, historical and natural significance in  Hawaii. Pan-STARRS  is a project of the  Institute for Astronomy of the University of Hawaii , and is supported by the  NASA   SSO Near Earth Observation Program  under grants  80NSSC18K0971 ,  NNX14AM74G ,  NNX12AR65G ,  NNX13AQ47G ,  NNX08AR22G ,  YORPD20_2-0014  and by the  State of Hawaii. This work has made use of the  CANDIDE Cluster at the Institut d'Astrophysique de Paris  and made possible by grants from the  PNCG, CNES , and the  DIM-ACAV. This work was supported in part by the  Canadian Advanced Network for Astronomical Research (CANFAR)  and  Compute Canada  facilities. We gratefully acknowledge support from the  CNRS/IN2P3 Computing Center (Lyon -France ) for providing computing and data-processing resources.
Accordingly, in this study, we utilized intensive longitudinal data collection (ie, ecological momentary assessment [EMA]) to capture these rapid changes. The primary research questions, context, and hypotheses were as follows: • What is the impact of daily exposure to news about COVID-19 on mental health? Daily levels of worry and optimism about COVID-19 were assessed over an 8-week study period in the early months of the pandemic. Previous research conducted with this sample indicates that COVID-19 worry and anxiety are significantly higher than general, nonspecific anxious feelings among undergraduates  [49]. Given previous research  [35] , we hypothesized that greater daily exposure to news about COVID-19 would lead to increased worry and decreased optimism about the pandemic. • What are mechanisms through which exposure to news about COVID-19 increases mental health issues? Momentary levels of hopelessness and general worry were examined over the same 8-week period.
Participants and Procedure Data (N=546) were drawn from an ongoing study assessing stress, emotion, and behavior among a general college student sample. Recruitment took place between April and December 2020 following the initiation of stay-at-home orders. Participants were remotely recruited via the undergraduate psychology research pool and introductory courses at a large, public university. Recruitment was open to all students who met eligibility criteria, which were assessed using an online screener. Eligible individuals were aged 18+ years, had access to a smartphone compatible with EMA survey software (MetricWire), and were willing to complete multiple surveys per day across the study period. Upon completion of an informed consent protocol, eligible participants were asked to complete an online baseline assessment and were provided with instructions for downloading MetricWire, a smartphone app used to deliver EMA surveys for the 8-week period.
Analytic Strategy Across the sample, participants submitted a total of 86,626 survey responses. Individual surveys were excluded from analyses if there was no variability across responses to all items in that specific survey (SD 0 across all variables scored on a Likert scale), which removed 726 (0. 8%) survey responses. Since optimism about the coronavirus outbreak and exposure to COVID-19 news were assessed once per day, momentary variables of interest (ie, hopelessness, general worry, and worry about COVID-19) were aggregated to create daily average scores for each variable for each participant. Participants who recorded <3 days of EMA data were excluded from analyses, yielding a final sample of 19,888 days of data across 546 participants. Given the hierarchical structure of our data, multilevel mediation analyses were used to test our hypothesis that COVID-19 worry and optimism would mediate the relationship between COVID-19 news exposure and general worry and hopelessness.
Recruitment was limited to students who were willing and able to complete multiple daily surveys over 8 weeks, and our sample may have systematic differences from other students (eg, less busy schedules), potentially reducing generalizability. All participants were undergraduate students recruited from the same university, likely further limiting generalizability to other populations. The identified relationships between COVID-19 news exposure and mental health should be studied in other populations, including clinical samples with higher levels of distress. This study had several strengths. It is the largest and most fine-grained study to date on the relationship between frequency of daily news exposure and same-day and next-day mental health outcomes among college students. This study used consecutive days of data collection to test the influence of variables specifically related to COVID-19 on general distress the following day, allowing for longitudinal analysis of proximal effects.
This may explain why large fractions of patients participating in WGS cohorts do not receive a molecular diagnosis  [7, 8]. A first issue to identify candidate pathogenic variants in patients is the sheer volume of genetic variants to consider. A WGS dataset delivers 4 to 5 million variants per individual. Less than 10% are present at high frequency in the human population (more than 5% of individuals), the vast majority (>90%) are present at lower frequencies  [8] , and a few thousand are unique to each genome  [9]. Ultra-rare variants, among which a mutation causing a highly penetrant genetic disease might be sought in priority, therefore amount to tens of thousands of candidates. Secondly, the catalogue of regulatory elements in the human genome is still incomplete. Tremendous progress has been made in characterising the properties of noncoding regions using genome-wide assays in many cell types, including the ENCODE  [10]  and the Roadmap Epigenomics project  [11].