dataesr/scientific-paragraphs-categorization huggingface dataset:
Topic Development Analysis During topic development, annotators searched live Twitter for Chinese Tweets on 76 topics and for Persian Tweets on 131 topics. After this, the document set in each language was fixed. Relevance assessment was undertaken for a topic if at least one relevant Tweet had been identified during topic development. There were seven topics in Chinese and thirty-one in Persian for which no document was found to be relevant during topic development. A minimum of three somewhat or very valuable documents was required after relevance assessment was completed for a topic to be included in the collection. We also excluded topics in which relevance assessment did not end with a negative streak of twenty non-relevant documents, because such a negative streak gave us reasonable confidence that HiCAL could not have found many more relevant documents.
Interestingly, PSQ outperforms the neural systems in Persian by both nDCG@20 and in recall. However, DPR-X and ColBERT-X are more effective than PSQ in Chinese. This is likely due to better MT models for translating MS MARCO documents into Chinese than into Persian, and to the monolingual embedding quality of high-resource Chinese vs. medium-resource Persian. SPLADE-X, which uses the masked language modeling (MLM) head to predict English tokens corresponding to foreign language Tweets is worse than PSQ in both Persian and Chinese. As SPLADE-X is essentially a token-level translation model using the MLM head, with a result format similar to that of PSQ, training PSQ with parallel data is the better approach. While the percentage judged in the top twenty retrieved documents may seem low, the utility of an evaluation collection depends on whether the set of judged documents, especially the relevant ones, is systematically biased toward certain systems.
Because Tweets become unavailable over time, the contents of the documents are checked against a hash code that is also released for each document. Any document whose hash changes will be removed from the collection. This means that scores across experimenters may become incompatible as documents retrieved by baseline systems become unavailable. To mitigate this problem and allow future researchers to compare against baselines derived from those in this paper, we 13 https://developer. twitter. com/en/developer-terms/agreement-and-policy release the run files of all baseline runs and a post hoc evaluation script. When provided with a run file and either a document file or a list of document ids, the script will produce a new run file with any missing documents removed. It also creates a new qrels file with unavailable documents removed.
Health effects of endocrine disrupting chemicals (EDCs) are challenging to detect in the general population. Omics technologies become increasingly common to identify early biological changes before the apparition of clinical symptoms, to explore toxic mechanisms and to increase biological plausibility of epidemiological associations. This scoping review systematically summarises the application of omics in epidemiological studies assessing EDCs-associated biological effects to identify potential gaps and priorities for future research. Ninety-eight human studies (2004-2021) were identified through database searches (PubMed, Scopus) and citation chaining and focused on phthalates (34 studies), phenols (19) and PFASs (17), while PAHs (12) and recently-used pesticides (3) were less studied. The sample sizes ranged from 10 to 12,476 (median = 159), involving non-pregnant adults (38), pregnant women (11), children/adolescents (15) or both latter populations studied together (23).
Search strategy and selection criteria References were identified by searches of PubMed and Scopus databases. Bibliography citations of the identified publications were also screened to identify additional articles. The automated search focused on papers published from January 1, 2004 (date of the first identified paper) to December 14, 2020, written in English and involving human participants. However, key publications that shaped the field were also included manually if missed during the automated search until February 2022. To be included in the review, papers needed to measure at least one EDC and one type of omics (targeted or untargeted). The full list of search terms can be found in Supplementary methods. The list of chemicals included in this review was revised by the research team and members of the H2020 exposome project ATHLETE  (Vrijheid et al.
Data charting process A data-charting form was developed to determine which variables to extract. We extracted data on the type and method of EDC exposure and omics assessment, study participant characteristics (e. g. country of origin, cohort name if provided, demographic characteristics), the study design (cross-sectional/longitudinal, cohort/case-control/cases only/ randomised intervention and sample size). Detailed definitions can be found in Table  1. We drew a word cloud network to visualise the number of articles in which individual EDC parents and metabolites were measured, taking into account if they were assessed together in the same study. The layout was drawn in Cytoscape 3. 9. 0 with the prefuse force directed layout function.
Results Our searches spanning from 2004 to 2020 across the databases retrieved 1411 total records. These studies were identified after using pre-set literature filters (e. g. excluding non-human studies, see Methods). Finally, 102 unique studies were selected after title/abstract screening and removal of duplicates and off-topic references such as remaining non-human studies (Fig. 1 ). Further 13 studies were excluded manually for not fitting in our framework of interest (e. g. naturally occurring phenols). Nine studies missed by our automatic search were included manually, which led to a final of 98 unique studies included in this review. The full list of studies is presented in the Supplementary Table  S1.
Study designs and populations Details on the study designs and included populations for the 98 studies are presented in the Supplementary Table  S1. The vast majority of the studies were conducted in the USA (n = 28), Europe (n = 18) and China (n = 17). Almost all studies (n = 95) were observational and only three studies on phenols involved a randomised intervention (Table  2 ). These randomised controlled trials included studies on: BPA exposure through providing canned soymilk and association with miRNA profiles  (Kim et al. , 2020)  and triclosan and triclocarban exposure from household and personal care products associated with gut microbiome diversity  (Poole et al. , 2016; Ribado et al. , 2017). In the observational studies, the data were collected most often in cohort (n = 60) and case-control studies (n = 26), with a few articles  Abbreviations: PAHs = polycyclic aromatic hydrocarbons; PFASs = per-and polyfluoroalkyl substances.
Replication issues due to inter-individual (e. g. BMI) or intra-individual (e. g. time of day of data collection, season and circadian or hormonal cycles) variability should be controlled or accounted for in the study design or statistical analysis  (Gallego-Paüls et al. , 2021). Biological validation: Validation could be achieved through evidence triangulation by identifying independent biological evidence through systematic toxicological literature search [e. g. by screening databases such as The Comparative Toxicogenomics Database (CTD, http://ctd base. org/, see example in  (Chen et al. , 2022) ) or genetic data  (Avery et al. , 2022) ] or by applying multi-omics approaches where different assays provide complementary information. Only a few studies used more than one omic platform to investigate the relationship between EDC exposure and omic markers  (Borsky et al.
-Omics assessment: to include omics sample replicates collected using standardised protocols and with the use of standardised reference materials and normalisation methods. -Reproducibility: to include biological replicates and perform multicohort studies. -Biological validation and triangulation of findings: to use several omic platforms where different assays would provide complementary information and to identify independent biological evidence through systematic toxicological literature search. -Confounding: to collect important information about confounders related to inter-and intra-individual variability. -Standardisation of result reporting: to systematically add results in designated databases to facilitate access to the full results. Finally, this review should be considered with a limitation that is the heterogeneity in definitions of omics, in particular for targeted metabolic and protein studies, such as those focusing on hormones.
Acknowledgements LM is funded by a  Juan de la Cierva-Incorporación fellowship  ( IJC2018-035394-I ) awarded by the  Spanish Ministerio de Economía, Industria y Competitividad. PJ was supported by the  French National Agency for Research (ANR , grant n•  ANR-15-IDEX-02 ). ISGlobal acknowledges support from the  Spanish Ministry of Science and Innovation through the "Centro de Excelencia Severo Ochoa   2019-2023 ".
The total exposure time was 1000 s for the H 140 images, and 4000 s for the G141 grism spectroscopy. The details of the HST data reduction are provided in  Noirot et al. (2016) , whilst  Noirot et al. (2018)  describes how the cluster candidates were confirmed using emission line cluster members detected with the grism spectroscopy. Our morphological measurements of the cluster galaxies were obtained from the WFC3 H 140 images, so we recall the important details of these data here. Each WFC3 image has a field of view of 2 × 2. 3 2 with a resolution of 0. 13 pix -1. The images were dithered and processed using the software aXe (v2. 2. 4;  Kümmel et al. 2009; Kümmel et al. 2011 ) to obtain images with a final resolution of 0. 06 pix -1. Our HST image 5σ magnitude limit within an aperture of radius of 0.
Control sample photometry from CANDELS We wish to compare our cluster findings from the CARLA data to a field galaxy baseline. For this purpose, we use the CAN-DELS wide survey, in the GOODS-S region, since the depth and resolution of the CARLA H 140 images correspond to the depth and resolution of the HST/F160W (hereafter H 160 ) CAN-DELS images. In fact, our HST image 5σ magnitude limit is H 140 = 27. 1 mag compared to CANDELS magnitude limit of H 160 = 27. 4 mag (both were calculated within an aperture of radius 0. 17 ). To assemble all the relevant information, we combined the following public catalogues: photometry and photometric redshifts from  Guo et al. (2013) , stellar masses and other galaxy parameters from  Santini et al. (2015) , and galaxy morphologies from  Kartaltepe et al.
(2015). We combined the catalogues by matching data by position and we hereafter refer to this catalogue as the 'combined CANDELS catalogue'. In this catalogue, the photometry is presented as total fluxes estimated by PSF-matching. For low resolution imaging, such as IRAC, the PSF-matching was performed using the software TFIT  (Laidler et al. 2007). The PSF-matched photometry from CANDELS was derived using a different software than used to obtain our CARLA catalogues. To ensure this would not bias our results, we re-extracted the photometry from a ≈47 2 subregion of the GOODS-S CANDELS Wide field using exactly the same T-PHOT method we performed on the CARLA fields. We re-extracted the photometry using the public ACS/F775W (hereafter i 775 ), ACS/F850W (hereafter z 850 ), IRAC1, IRAC2, H 140 , and H 160 images from the CANDELS and 3D-HST archives  (Brammer et al.
Acknowledgements. For the purpose of open access, the author has applied a creative commons attribution (CC BY) to any author accepted manuscript version arising. This work is based on observations made with the  NASA/ESA Hubble Space Telescope , obtained at the  Space Telescope Science Institute , which is operated by the  Association of Universities for Research in Astronomy Inc. , under  NASA  contract  NAS 5-26555. These observations are associated with program  GO-13740. Support for program GO-13740 was provided by  NASA  through a grant from the  Space Telescope Science Institute , which is operated by the  Association of Universities for Research in Astronomy Inc. , under  NASA  contract  NAS 5-26555. This work is based on observations made with the  Spitzer Space Telescope , which is operated by the  Jet Propulsion Laboratory, California Institute of Technology , under a contract with  NASA.
Using less than 50% of learners' action sequences, the cross-validated deep learning model achieves a squared correlation higher than 0. 8 with a relative error of less than 8%, outperforming a range of regression models like linear regression, random forest, neural networks, and support vector machines. An additional experiment showed that the validated deep learning model can also achieve high performance while tested on an independent game dataset, showing its applicability and robustness in real-world cases. Comparing the results with traditional machine learning methods revealed that, in the validation and application phases, up to 0. 30 and 0. 35 R 2 gain is achieved in favor of the deep learning model, respectively. Finally, we found that while the lengths of action sequences influence the predictive power of the traditional machine learning methods, this effect is not substantial in the deep learning model.
For instance, it can adaptively regulate or activate the feedback system, hints, and tutorials employed in the game. Additionally, early prediction can help to monitor learners' actions in the games and flag at-risk learners, potentially offering timely warnings, as well as facilitating the design of adaptive non-player characters (NPCs). Contributions of our research are manifold: i) a novel and validated predictive modelling approach that considers learners' action sequences to predict their final scores at the early stages of the game; ii) comparing a state-of-the-art prediction model (i. e. , deep learning) with several traditional machine learning methods for tackling the regression task of early prediction of learners' scores in educational games; iii) investigating the applicability of the validated models by testing them on an independent educational game dataset; iv) investigating the effect of various sequences of learners' actions on the prediction power of different regression models.
Their findings revealed that their proposed approach can to a certain extent early predict players' churn. They also mentioned that including more game-specific features has the potential to improve the performance of their predictive modelling approach. Gagnon et al. [11]  used raw clickstream data from two educational games-i. e. , the Crystal Cave and Wave Combinator-to predict quitting and the performance of learners during gameplay. The authors used data samples of 1,254 and 5,308 from anonymous internet players during the fall of 2018 and after pre-processing and feature engineering, they developed predictive models. Their findings revealed that logistic regression could successfully predict learners' quitting and performance, outperforming other machine-learning methods. Roberto et al.
Moreover, Hooshyar et al. [8]  developed a novel predictive modelling approach that combines domain knowledge with advanced machine learning methods to estimate learners' success in the next game tasks. The approach called GameDKT consumes learners' in-game data from an educational game for fostering computational thinking. Their findings revealed that learners' knowledge could be modelled or traced over time through their past activities, facilitating the prediction of their performance in the upcoming game tasks. A convolutional neural network with an accuracy of 85% was found to be the best for the prediction task. As it is apparent, there has not been much research considering in-game data of learners to early predict their final scores in educational game episodes. Moreover, the existing related works mainly ignore: i) tackling the regression problem of early prediction of learners' final score (rather deal with binary classification tasks); ii) employing deep learning models and comparing their performance with traditional machine learning methods; iii) investigating the effect of various sequences of learners' actions on the prediction power of different regression models; iv) investigating the applicability of the validated models by testing them on an independent educational game dataset.
B. THE PROPOSED GameEPM APPROACH Algorithm 1 illustrates the step-by-step process of the GameEPM approach. Additionally, Fig. 1  presents a general overview of the approach. As shown in the figure, in the first step, we first collect and pre-process learners' sequence of solution submissions. For this purpose, a combination of features that directly or indirectly deal with learners' final scores was used. For example, this can include the number of collected objects in a game, the score gained for doing a specific action, the game task IDs, learner IDs, and so on. We filtered out game data belonging to those learners who had not completed the game episode. The reason for such data exclusion is that we needed to use the learner's final score to signal their final performance. Thereafter, we normalized our data using a range transformation with min and max of 0 and 1, respectively.
To remove noise from our dataset, we implemented the outlier detection method of local outlier factors using Euclidean distance. Subsequently, we converted the sequence of learners' actions or solution submissions into varying time windows (sequence of input vectors) and 1 https://www. youtube. com/watch?v=O3K6G0i1jYU labelled them with their final scores. More specifically, for each learner, we created time windows using 30%, 40%, 50%, 60%, 70%, 80%, 90% and 100% of their action sequence. After creating the nonstationary time windows, we created multiple datasets for early prediction of the learner's final score. Because some learners completed the game with less than 20 solutions, which may have resulted in winning or losing, we used the Nearest Neighbour imputation technique to create a stationary example set for each time window (i.
Algorithm 1 Early Prediction of Learners' Performance Input: Dataset D Output: Regression results among R 1: Initialize i =0 2: X (A) = Feature Selection(A) 3: X = Normalization(X ) 4: X = Outlier detection(X ) 5: for i in N //N is the number of learners 6: u i =Time windowing T (t τ , t τ ×p ) of each u in X 7: U = KNN imputation for missing values(U ) 8: for i in N // Exponential Smoothing 9: for j in m 10: t ′ i j×1 = αt i j + (1 -α) t ′ i j 11: Apply regression methods using U 12: 13: Compare the performance of regression methods through cross-validation and independent dataset 14: Select the algorithm with the highest performance 15: Use the algorithm for early prediction of players' performance.
IV. RESULTS AND FINDINGS A. DATASETS In this research, we used log data of 427 learners from five countries (Estonia, France, South Korea, Taiwan, and South Africa) who have played the third level of the Auto-Thinking game. Learners had different backgrounds and ages. The dataset includes up to 6199 examples or solutions collected between December 2019 and April 2022. While some part of the data was collected during experimental studies (e. g. ,  [31] ,  [32] ), some learners played the game on their own. To complete the game, learners developed 20 or fewer solutions, depending on the game episode' conditions. During the gameplay, various interaction of learners is logged, for instance, the position of mouse and NPCs, task IDs, number of small and big cheese eaten, loop and conditional usage, arrow and function usage, debug and simulation usage, number of times that help was requested, frequency of given feedback and hints, frequency of hitting walls, the estimate of learner's overall knowledge of computational thinking or the quality of the developed solutions (inferred from the Bayesian network decision-making algorithm used in the game), etc.
To generate the Bayesian estimates, the game considers multiple factors such as the potential of gaining scores, the risk of being caught by the NPCs, and the quality of computational thinking skills and concepts used in solutions. Further information on the decision-making process can be found in Hooshyar et al. [33]. Of all the features, based on the design and mechanism used in the game, we opted for those that directly or indirectly deal with learners' final scores, for instance, the number of big and small cheeses eaten, score gained for actions done in a single solution, the game task IDs, learner IDs, number of times bumping into walls, overall score gained for one solution, frequency of using loop and conditional concepts, and the final score of the game episode. To validate the GameEPM approach, by splitting the original dataset using shuffled sampling, we used data from 309 learners for the validation phase and an independent 30 learners (560 solutions) for the application phase.
Additionally, to test the capability of the proposed approach in the early prediction of the learner's final performance, we developed eight different versions of each dataset (for cross-validation and application datasets), namely 30% to TABLE 2. Squared correlation, RMSE, and relative error of the models for different sequence lengths using cross-validation. 100% of the learner's action sequence. In the dataset, the min, max, and the average of the learner's final scores were 342, 3294, and 2349, respectively.
For instance, when using 50% of the sequence of the learner's action, for those learners with only 10 solutions, we used the first five solutions and filled up the missing time steps by identifying samples closest to it and averaging those nearby points to fill in the value to create 50%-time windows for the models. Furthermore, k-fold cross-validation was employed as it is considered among the most reliable validation methods for the future accuracy of a predictive model, especially when it comes to rather small datasets. Simply put, we employed cross-validation as it enables us to test how well our models can get trained by some data and then predict data it has not seen. The choice of k involves a trade-off between the error prediction's efficiency and accuracy, and k = 10 is usually preferred as it has been shown empirically to produce test error rate estimates suffering neither from excessive-high variance nor very high bias.
Besides, to investigate the practicality and applicability of our models, we tested the cross-validated models on an independent game dataset of 30 learners (including 560 solutions). We compare the performance of different models on eight different sequences of learners' actions (i. e. , 30% to 100% of solutions). To measure the performance of the models, we have employed three metrics of squared correlation (R 2 ), root mean squared error (RMSE), and relative error (RE). RMSE shows the standard deviation of prediction errors, R 2 refers to the square of the Pearson correlation coefficient between the ground truth and predicted values, and RE reveals the magnitude of the absolute error concerning the actual size of.
D. PERFORMANCE OF MODELS USING INDEPENDENT GAMES DATA: APPLICATION PHASE This section reports our findings on applying the crossvalidated models to an independent game dataset of 30 learners (including 560 solutions). To merely focus on the performance of models on the early prediction of learners' performance, we compared the performance of different models on three sequences of learner's action, which are 30%, 50%, and 70% of solutions (see Table  6  in Appendix A for model performances on other sequence lengths). Table  5  illustrates the performance of regression models using the three metrics. The deep learning model appears to have outperformed other models in all three sequence lengths with an RE of less than 7%. More specifically, in sequence length of 30, the deep learning model reached R 2 of 0.