dataesr/scientific-paragraphs-categorization huggingface dataset:
The evolving epidemiology of COVID-19, the response of governments and populations to the pandemic, and academic responses to papers based on the dataset all shape the questions that are added to or removed from the app over time. The dataset is only minimally validated at source. The fields often contain data of mixed type, and different fields can be in mutual contradiction. Numeric values are only validated for type rather than sensible value ranges. Furthermore, the dataset contains multiple competing schema for the same underlying data, and the app version is tied to the schema version, so users who are using older versions of the app are still contributing to otherwise retired schema elements. As such, a considerable amount of data cleaning and processing is required in order to extract data suitable for analysis.
Schema changes The handling of COVID-19 tests in the dataset is an example of the complexity created by changes to the schema. Testing was initially reported as an assessment logging activity, but this came with a number of problems. Firstly, a test needed to be logged on the day it was taken for the assessment date to be treatable as the test date. Secondly, some users interpreted the test field as something to be logged only when they took a test or received the result, whilst other users filled in intermediate assessments with the pending status. Thirdly, this system did not allow for users to enter multiple tests unambiguously. Whilst this was not a problem in the initial months of the pandemic, the ramping up of test availability necessitated a solution. A new test table was introduced in June 2020, giving each test a unique id to allow multiple tests for each patient.
Reproducibility and algorithm immutability Reproducibility depends on the ability to reproduce a given analysis from a version of the dataset and a set of algorithms run on the dataset. For this to be possible, algorithms must be considered immutable once implemented. This allows any subsequent version of the software to generate results consistent with those of the software version in which the algorithm was introduced. ExeTera does this by requiring that a version of any given algorithm that is created is treated as immutable in the code base. This means that any target script is guaranteed to exhibit the same behaviour, provided that the following conditions hold. Firstly, any algorithms written for ExeTera are explicitly versioned. Secondly, any randomness introduced must be given consistent random seeds and, ideally, multiple sources of randomness should be given different random number generators.
Working with Python Code that is compiled and run directly in CPython (the reference Python implementation) executes in the Python interpreter. The Python interpreter is extremely slow relative to optimised code such as that generated by compiled, optimised C/C++; in many cases it is orders of magnitude slower. Python is dynamically typed, but its type system does not provide light-weight objects to represent primitive types. Even numeric values such as integers and floats are stored as full objects, and typically take up 28 bytes for a 4 byte integer value. This overhead precludes efficient memory usage when iterating over large numbers of values. Numpy  6, 7  is the Python community's main tool for circumventing such time and space inefficiencies. Amongst other features, it provides a library for space-efficient representations of multi-dimensional arrays, and a large library of time-efficient operations that can be carried out on arrays.
Column-local data format Column-local data storage enables very efficient access to a given field. All the entries for a given field are (effectively) contiguous in memory and so loading a single field can be done in an asymptotically optimal fashion. When a dataset has many fields and a given operation operates on a smaller number of those fields, scaling the operation is a far simpler proposition, technically  9, 27. The ability to load specific subsets of the data with maximal efficiency benefits both the ability to scale and the latency with which operations can be performed on a small subset of the dataset. As such, a column-local data format is a preferable format. HDF5 as the ExeTera serialised data format HDF5 10 is a data format for storing keys and their associated values in a hierarchically organised, nested collection.
In contrast with CSV, HDF5 stores data in a column-local format. It also allows for data to be stored as binary, concrete data types. HDF5 permits a user to explore the overall structure of the data without loading fields. Fields are loaded at the point that a user specifically requests the contents of a given field. This can be a direct fetch of the entire field or an iterator over the field. This makes it a suitable initial data format for ExeTera, although alternative columnar data storage formats are being considered to replace HDF5 for future development due primarily to issues of dataset fragility and shortcomings relating to concurrent reading / writing.
Sorting Sorting is one of the key operations that must scale in order to process large datasets, as imposition of a sorted order enables operations such as joins to scale. ExeTera uses several techniques to provide highly scalable sorting. Generation of a sorted index Rather than sorting data directly, ExeTera generates a sorted index that is a permutation of the original order. This is used to scale related sort operations, and implement a soft sort, where fields are stored in a natural sorted order and the permuted index applied when the field is read. Scaling multi-key sorts on long arrays Multi-key sorts are memory intensive when keys are large, and expensive due to the internal creation of tuples in the inner loops of sorts. Multi-key sorts in ExeTera are rephrased as a series of sorts on individual keys from right to left, where the output of each sort step is a sorted index that is the input to the next sort step, using a stable sort.
ExeTeraThe core functionality for ExeTera is hosted at https://github. com/KCL-BMEIS/ExeTera. git and is available through pypi via pip install exetera. ExeTeraCovidThe functionality for the Covid Symptom Study dataset, (algorithms, scripts and other resources) is hosted at https://github. com/KCL-BMEIS/ExeTeraCovid. git and is available through pypi via pip install exeteracovid. Installing exeteracovid installs exetera. ExeTeraEval Code for creating evaluation datasets, including the dataset used to evaluate join operations, is hosted at https://github. com/KCL-BMEIS/ExeTeraEval. git.
Method 2: synthesis of the entire optical spectrum We also derived stellar properties by analysing parts of the optical spectrum in a different way by comparing the normalized, co-added spectrum with modelled synthetic spectra obtained with the Spectroscopy Made Easy (SME) package  (Valenti & Piskunov 1996; Piskunov & Valenti 2017)  version 5. 22, with atomic parameters from the VALD database  (Piskunov et al. 1995). The 1-D, plane-parallel LTE synthetic spectra are calculated using stellar parameters obtained from either photometry or a visual inspection of the spectrum as a starting point. The synthetic spectrum is automatically then compared to a grid of stellar atmospheric models. The grid we used in this case is based on the MARCS models  (Gustafsson et al. 2008 ). An iterative  2 minimization procedure is followed until no improvement is achieved.
Stellar activity monitoring Two instruments were used during different time periods to monitor TOI-431 in order to investigate the rotation period of the star. This is important to disentangle the effect of stellar activity when fitting for any planets present in the system. WASP-South, located in Sutherland, South Africa, was the southern station of the WASP transit survey  (Pollacco et al. 2006 ). The data reported here were obtained while WASP-South was operating as an array of 85mm, f/1. 2 lenses backed by 2048x2048 CCDs, giving a plate scale of 32 /pixel. The observations spanned 180 days in 2012, 175 days in 2013 and 130 days in 2014. Observations on clear nights, with a typical 10-min cadence, accumulated 52 800 photometric data points. We searched the datasets for rotational modulations, both separately and by combining the three years, using the methods described by  Maxted et al.
LCOGT No further detrending to that outlined in Section 2. 1. 2 was included for the LCOGT data. Only TOI-431 d is transiting in this data, so we create a model light curve of TOI-431 d using S (as outlined above) per LCOGT dataset to produce 2 model light curves overall, as there are 2 transit events -an ingress and an egress -on separate nights. For each dataset, we use a normal prior with the model light curve as the mean and a standard deviation set to the error on the LCOGT data points, and this is then compared to the observed light curve. The best fit model for both the ingress and egress data is shown in Fig. 7  (top 2 panels).
2015). We multiply these kernels together to create our final Quasi-Periodic kernel: 𝑘 (𝑥, 𝑥 ) = 𝜂 2 exp - sin 2 (𝜋|𝑥 -𝑥 | 1 𝑇 ) 2𝑙 2 𝑝 - (𝑥 -𝑥 ) 2 2𝑙 2 𝑒. (5) We use the same GP to fit the HARPS and HIRES data together using the same hyperparameters. We use a normal distribution with a mean equal to the rotation period of the star found by WASP-South (see Section 2. 5 and Table  2 ) to set a wide prior on. To bring everything together, we add the predicted radial velocities together with the offsets, and subtract these from their respective observed radial velocity values. This is then used as the prior on the GP, which is also given a noise term that is equal to an addition of the jitters with the squared error on the RV data.
We further analyse these planets in the following section. RV data plots, where the HARPS data is denoted as grey circles, HIRES as red upside down triangles, iSHELL as pale orange triangles, FEROS as pale pink squares, and M -Australis as pale turquoise diamonds. Top plot: the RV data, showing the GP and planet models that have been fit. Top: the best-fit GP used to detrend the stellar activity in the HARPS data is shown as the green line. The green shaded areas represent the 1 and 2 standard deviations of the GP fit. Upper middle: the separate models for each planet, b (orange, offset by +6 m s -1 ), c (red), and d (blue, offset by -6 m s -1 ). Lower middle: the total model, representing the addition of the models for planets b, c, and d, is plotted in black, and over plotted is the HARPS and HIRES data.
Bottom: the residuals after the total model, GP and offsets have been subtracted from the RV data. Bottom plot: the phase folds for each planet model, b (left), c (middle), and d (right), with the RV data over plotted. The top row shows all of the RV data (where the GP has been subtracted from each data set), the middle just the HARPS and HIRES data, and the bottom the residuals when the planet models have been subtracted from the RVs.
CONCLUSION We have presented here the discovery of three new planets from the TESS mission in the TOI-431 system. Our analysis is based upon 2-min cadence TESS observations from 2 sectors, groundbased follow-up from LCOGT and NGTS, and space-based followup from Spitzer. The photometric data was modelled jointly with RV data from the HARPS spectrograph, and further RVs from iSHELL, FEROS, and M -Australis are included in our analysis. We find evidence to suggest that the host star is rotating with a period of 30. 5 days, and account for this in our joint-fit model. Nearby contaminating stellar companions are ruled out by multiple sources of high resolution imaging. TOI-431 b is a super-Earth characterised by both photometry and RVs, with an ultra-short period of 0. 49 days. It likely has a negligible envelope due to substantial atmosphere evolution via photoevaporation, and an Earth-like composition.
TOI-431 c is found in the HARPS RV data and is not seen to transit. It has a period of 4. 84 days and a minimum mass similar to the mass of TOI-431 b; extrapolating this minimum mass to a radius via the MR relation places it as a likely second super-Earth. TOI-431 d is a sub-Neptune with a period of 12. 46 days, characterised by both photometry and RVs. It has likely retained a substantial H-He envelope of about 4 per cent of its total mass. Additionally, TOI-431 b and d contribute to the TESS Level-1 mission requirement. This system is a candidate for further study of planetary evolution, with TOI-431 b and d either side of the radius valley. The system is bright, making it amenable to follow-up observations. TOI-431 b, in particular, would potentially be an interesting target for phase-curve observations with JWST.
Figure9. RV data plots, where the HARPS data is denoted as grey circles, HIRES as red upside down triangles, iSHELL as pale orange triangles, FEROS as pale pink squares, and M -Australis as pale turquoise diamonds. Top plot: the RV data, showing the GP and planet models that have been fit. Top: the best-fit GP used to detrend the stellar activity in the HARPS data is shown as the green line. The green shaded areas represent the 1 and 2 standard deviations of the GP fit. Upper middle: the separate models for each planet, b (orange, offset by +6 m s -1 ), c (red), and d (blue, offset by -6 m s -1 ). Lower middle: the total model, representing the addition of the models for planets b, c, and d, is plotted in black, and over plotted is the HARPS and HIRES data. Bottom: the residuals after the total model, GP and offsets have been subtracted from the RV data.
ACKNOWLEDGEMENTS This paper includes data collected by the TESS mission. Funding for the TESS mission is provided by the  NASA   Explorer Program. Resources supporting this work were provided by the  NASA   High-End Computing (HEC) Program  through the  NASA Advanced Supercomputing (NAS) Division at Ames Research Center  for the production of the SPOC data products. We acknowledge the use of public TESS Alert data from pipelines at the  TESS Science Office and at the TESS Science Processing Operations Center. This study is based on observations collected at the  European Southern Observatory  under  ESO  programme  1102. C-0249  ( PI: Armstrong ). Additionally the archival HARPS data dating from 2004 to 2015 were obtained under the following programmes:  072. C-0488  (PI: Mayor);  183. C-0972  (PI: Udry);  085.
We acknowledge support by  FCT -Fundação para a Ciência e a Tecnologia  through national funds and by  FEDER  through  COMPETE2020 -Programa Operacional  Competitividade e Internacionalização by these grants:  UID/FIS/04434/2019 ;  UIDB/04434/2020 ;  UIDP/04434/2020; PTDC/FIS-AST/32113/2017 & POCI-01-0145-FEDER-032113; PTDC/FIS-AST/28953/2017 & POCI-01-0145-FEDER-028953. VA, EDM and SCCB acknowledge the support from  FCT  through  Investigador FCT  contracts  IF/00650/2015/CP1273/CT0001 ,  IF/00849/2015/CP1273/CT0003 , and  IF/01312/2014/CP1215/CT0004  respectively. ODSD is supported in the form of work contract ( DL 57/2016/CP1364/CT0004 ) funded by  FCT. CD acknowledges the  SNSF Ambizione Grant   174028. SH acknowledge support by the  fellowships PD/BD/128119/2016  funded by  FCT  (Portugal).
DATA AVAILABILITY The TESS data are available from the Mikulski Archive for Space Telescopes (MAST), at https://heasarc. gsfc. nasa. gov/ docs/tess/data-access. html. The other photometry from the LCOGT, NGTS, and Spitzer as well as all of the RV data, are available for public download from the ExoFOP-TESS archive at https://exofop. ipac. caltech. edu/tess/ target. php?id=31374837. This data is labelled "Osborn+ 2021" in their descriptions. The high-resolution imaging data is also available from the ExoFOP TESS archive. The model code underlying this article will be shared on reasonable request to the corresponding author.
8 Figure9. RV data plots, where the HARPS data is denoted as grey circles, HIRES as red upside down triangles, iSHELL as pale orange triangles, FEROS as pale pink squares, and M -Australis as pale turquoise diamonds. Top plot: the RV data, showing the GP and planet models that have been fit. Top: the best-fit GP used to detrend the stellar activity in the HARPS data is shown as the green line. The green shaded areas represent the 1 and 2 standard deviations of the GP fit. Upper middle: the separate models for each planet, b (orange, offset by +6 m s -1 ), c (red), and d (blue, offset by -6 m s -1 ). Lower middle: the total model, representing the addition of the models for planets b, c, and d, is plotted in black, and over plotted is the HARPS and HIRES data. Bottom: the residuals after the total model, GP and offsets have been subtracted from the RV data.
2018a, b). (3) We conducted 12 semi-structured interviews (four in Kenya, eight in Mozambique) with LAI representatives and 20 interviews with small-scale farmers (10 in Kenya, 10 in Mozambique), and completed life-cycle assessments, including water footprint assessments, to measure environmental impacts (da Silva 2018). (  4 ) We analyzed remote-sensing data to quantify land uses and land use changes in the study areas  (Eckert et al. 2018 , Zaehringer et al. 2018a ). (  5 ) We conducted 68 semi-structured interviews with investors to survey business models  (Adalima, 2016 , unpublished manuscript, Burnod, 2017 , unpublished manuscript, Mutea et al. , 2017, unpublished manuscript). Finally, (6) we conducted key informant interviews with representatives from public and governmental organizations, development and finance organizations, project managers, farmer organizations, civil society and the private sector, and performed document analysis to collect data on governance systems  (Burnod and Andriamanalina 2017, Bourblanc and Belenfant 2018; Ralandison, 2016 , unpublished manuscript, Burnod, 2017, unpublished manuscript).
Step 4. Data analysis: identify the impact patterns To analyze this data, we applied Formal Concept Analysis (FCA) in search of recurrent impacts of LAIs and the associated conditions. Formal concept analysis is a tool for qualitative knowledge representation and inference  (Ganter and Wille 2012). It is suited for set-theoretic comparative analyses as it identifies the multiple configurations of attributes present in the truth table. In contrast to qualitative comparative analysis (QCA)  (Ragin 1987) , FCA retains factors even if their presence and absence has led to the same outcome in different cases. We used the Concept Explorer software, with the truth table of cases and their attributes serving as our input. Formal concept analysis generates a concept lattice and compiles logical implications between attributes.
"The concept lattice organizes the attributes in a hierarchical structure such that higher-tier attributes are logical implications of lower-tier attributes, while lower-tier items show distinct combinations with higher-tier attributes in the dataset"  (Oberlack et al. 2016: 157). In this way, FCA is capable of visualizing multiple configurations of LAI impacts. To identify these patterns in the 27 indicators of socio-economic, environmental, and food security impacts, we identified distinct sets of impacts profiles among the 16 LAIs using the following criteria: the sets of impact profiles are (1) consistent (i. e. , assembling cases with similar configurations of impact values); (2) crisp (where cases in one subset are as similar as possible to each other and as different as possible from cases in other subsets); (3) parsimonious (where the number of subsets is as small as possible); (  4 ) recurrent (observable in at least two cases); (  5 ) and have a high coverage (where the typology of subsets covers all cases).
The household surveys and interviews captured residents living in the study areas at the time of fieldwork in 2015-2017. We cannot rule out that some households who lost land access to LAIs left the targeted area before survey/interview data were collected. Current residents, including in-migrants, may not always report on displacements that affected previous land users. Third, we used 14 indicators to assess environmental impacts. Half of our data on environmental impacts (indicators ENV1-7) are based on the perceptions of households at the time of research. Perceptions of environmental impacts can be biased according to personal experiences and values. The other half of our data on environmental impacts (indicators ENV11-17) is based on interviews with LAI and small-scale farmers, corresponding with life-cycle assessments and expert assessments.
We used a simple sum of household assets classes. The sum does not reflect the value of assets  (Browne et al. 2014; Hendriks et al. 2016). A modified Consolidated Approach for Reporting Indicators of Food Security (CARI) console was also used for comparative analysis (WFP 2012). The CARI combines food security indicators (current status and coping capacity) into a summary called the Food Security Index (FSI), representing the overall food security status  (Butaumocho and Chitiyo 2017). The CARI used a combination of three food security indicators (i. e. food consumption score, food expenditure shares, and livelihood coping strategy). Due to a lack of livelihood coping strategy-related indicators in the database, we converted the data regarding the application of the more serious consumption-based coping strategies (i.
Socio-economic and food security impacts On socio-economic impacts and food-security impacts in Kenya, only aggregated data for the cases KE1-5 were available; we used this data as "best available data" for these LAI cases. Food security scores for Kenya were calculated without case KE6, because we were not able to include the KE6 case into formal analysis due to missing data on the indicators from other work packages. 3. Environmental impact scoring  We used 14 indicators to assess environmental impacts combining participatory and expert-based assessments of environmental impacts. Half of them (indicators ENV1-7) are based on semi-structured interviews that measure households' perceptions of chemical exposure, deforestation, overabstraction of water, water pollution with chemicals and effluents, air pollution with chemicals/respiratory problems, increase in pests and occupation of water sources.
For each indicator, we quantified the share of households perceiving this environmental impact. We classified this data into very low impacts (0% of households perceive this environmental impact); low (1-33%); medium (34-66%); and high impacts (67-100%). The other half of our data on environmental impacts (indicators ENV11-17) are based on interviews with LAI and small-scale farmers, corresponding with lifecycle assessments and expert assessments. Environmental impact scoring for the different cropping systems was based on a per surface area (one hectare) basis. Life cycle assessment (LCA) metrics for global warming potential, eutrophication potential, acidification potential, non-renewable energy consumption, and water consumption were calculated for specific case studies  (Da Silva, 2018)  and used to inform the scoring.
Acknowledgments: This research was funded jointly by the  Agence Nationale de la Recherche , France;  National Research Foundation , South Africa; and  Swiss National Science Foundation , Switzerland, via the  Belmont Forum and Joint Programming Initiative on Agriculture, Food Security and Climate Change (FACCE-JPI)  (Grant Number:  40FA40_160405 ). The Land Matrix Initiative provided support in data and data analysis. We are grateful to  Anu Lannen  for language editing. This article contributes to the  Global Land Programme  (glp. earth).
Virtual environment We used the open dataset from (ville-eurometropole de Strasbourg, 2018) which was financed by the European Union as part of a FEDER (Fonds Européen de DEveloppement Régional). It consists of a 3D mesh of a large area covering the metropole of Strasbourg. It was produced by photogrammetry using high resolution (between 4 and 7cm GSD) oblique imagery acquired by helicopter platform such that it presents details at a higher resolution than typical aerial LiDAR acquisitions. Figure  1  shows a 250m by 250m tile of this mesh.