dataesr/scientific-paragraphs-categorization huggingface dataset:
The statistical subtraction method The statistical subtraction method consists in dividing the dataset in two parts -the day and the night one -and determine the asymmetry (or its absence) by searching for a residual 7 Be signal in the subtracted spectrum (night minus day). This method has the advantage of cancelling possible long time-scale variations of the backgrounds, but it can be applied only when the period of the modulation is known a priori. Indeed, it couldn't be used for the g-mode modulations described in the next section. We produced toy datasets with different exposure and background conditions in the same way decribed in section 4. For each dataset, we create the Day and Night histograms assuming that all species rates are the same during the day and the night, except for the 7 Be neutrinos which are injected with an asymmetry A DN such that: A DN = ∆R R = 2 R N Be -R D Be R N Be + R D Be =⇒ R N Be = 2 + A DN 2 -A DN R D Be.
(6. 1) We recall that due to the regeneration of ν e in Earth, we expect a higher rate of neutrinos at night with respect to day. For each toy dataset we subtract the Day histogram from the Night one, creating in such a way the Difference dataset. We performed a frequentist hypothesis test by using ∆χ 2 = χ 2 null -χ 2 altern = χ 2 ∆R=0 -χ 2 ∆R free (6. 2) as the test statistics. On one hand, the null hypothesis implies that modulations are absent and therefore the 7 Be rate of the Difference dataset is assumed to be zero. On the other hand, according to the alternative hypothesis, we fit the Difference dataset treating the 7 Be rate as a parameter left free to vary. The optimized analysis ROI is 640 p. e. < E rec < 1200 p. e. , corresponding to 450 keV < E vis < 810 keV. An example of the Difference energy spectrum is shown in figure  13  for an exposure of 6 years and A DN = 0.
The Lomb Scargle Method The Lomb Scargle Method is a powerful tool to search for periodic variation in a time series of data  [80, 81]. It is an extension of the Fourier Trasform to treat datasets which are not evenly distributed in time and has been successfully adopted in several neutrino experiments  [18, 20, 82]. Thousands of toy datasets are built for a given day-night asymmetry A DN and a given exposure in the same way as for the statistical subtraction method. The ROI is optimized and constrained to the energy region where ) or with a peak at f = 1/T = 1 day -1 (χ 2 DN (f )). This difference is then normalized to χ 2 0 : P(f ) = χ 2 DN (f ) -χ 2 0 χ 2 0. (6. 3) The distribution of normalized LS power for thousands of toy datasets simulated by injecting a given asymmetry A DN , together with the one obtained when no asymmetry is injected, are plotted as blue and orange histograms of figure 14.
Acknowledgments We are grateful for the ongoing cooperation from the  China General Nuclear Power Group. This work was supported by the  Chinese Academy of Sciences , the  National Key R&D Program of China , the  CAS Center for Excellence in Particle Physics, Wuyi University , and the  Tsung-Dao Lee Institute of Shanghai Jiao Tong University in China , the  Institut National de Physique Nucléaire et de Physique de Particules (IN2P3) in France , the  Istituto Nazionale di Fisica Nucleare   (INFN) in Italy , the  Italian-Chinese collaborative research program   MAECI-NSFC , the  Fond de la Recherche Scientifique (F. R. S-FNRS)  and  FWO  under the " Excellence of Science -EOS" in Belgium , the  Conselho Nacional de Desenvolvimento.
7 Be neutrinos are dominant, that is 640 p. e. < Figure13. Statistical Subtraction method: difference between the Night and Day datasets when an asymmetry A DN = 0. 6% is injected (eq. 6. 1), for 6 years of data-taking, in the Low background scenario. The resulting7 Be profiles assuming the null hypothesis and the alternative hypothesis are displayed as red dashed and red solid lines, respectively. Test statistics distributions for the Statistical Subtraction Method (left) and the Lomb Scargle Method (right). The distributions are obtained from ten thousands of datasets, where a daynight asymmetry is injected (blue) and when no-asymmetry is injected (orange). In this example, A DN = 0. 6%, the background scenario is the Low one, and the exposure is 6 years. For more details on how the test statistics is built for each one of the two methods, see text.
The median value of the blue curve is shown as a vertical dashed blue line. E rec < 1040 p. e. , corresponding to 450 keV < E vis < 705 keV. Note that in this case the ROI is narrower than the one used in the statistical subtraction method, because the LS is more sensitive to backgrounds. Data are divided in 1 hour long bins: the bin time width has been optimized to contain enough statistics (about 507 Be neutrino events). For each toy dataset, the Lomb Scargle periodogram is created: it represents the spectral density histogram of a JCAP10(2023)022 signal as a function of frequency. A test statistics function P, namely the normalized LS power, is built by calculating the difference between the χ 2 , obtained when the periodogram is fitted either with a flat line (χ 2 0 p. e. 3000 700 800 900 1000 1100 1200 2000 Events / p.
e. - 1000 0 1000 Difference dataset Entries 10 2 10 -- 3000 2000 0. 45 0. 5 0. 55 Energy [MeV] Alternative hypothesis ( Be 7 R ∆ Null hypothesis ( 0. 6 0. 65 10 2 10 Modulated dataset 50% limit free) Be 7 R ∆ fixed to zero) 0. 7 0. 75 =0. 6%) DN Modulated dataset (A Unmodulated dataset Entries 0. 8 JCAP10(2023)022 1 1 0 5 10 15 20 25 30 35 40 0 0. 0002 0. 0004 0. 0006 0. 0008 0. 001 0. 0012 χ ∆ 2 Normalized LS Power Figure 14. -25 -.
In this paper we study the problem-solving ability of the Large Language Model known as GPT-3 (codename DaVinci), by considering its performance in solving tasks proposed in the "Bebras International Challenge on Informatics and Computational Thinking". In our experiment, GPT-3 was able to answer with a majority of correct answers about one third of the Bebras tasks we submitted to it. The linguistic fluency of GPT-3 is impressive and, at a first reading, its explanations sound coherent, on-topic and authoritative; however the answers it produced are in fact erratic and the explanations often questionable or plainly wrong. The tasks in which the system performs better are those that describe a procedure, asking to execute it on a specific instance of the problem. Tasks solvable with simple, one-step deductive reasoning are more likely to obtain better answers and explanations.
The strategy has a long history, since it has its roots in Shannon's seminal work  (Shannon, 1948) , but in recent years it has reached a new level of sophistication and results. One of the most popular LLM systems is GPT-3, developed by OpenAI  (Brown et al. , 2020). It is based on the GPT (Generative Pre-trained Transformer) architecture and it achieves strong performance on many benchmark datasets focused on translation, question-answering, and cloze tasks. An improved version of GPT-3 is also the basis of ChatGPT (https://chat. openai. com), a system fine-tuned for conversation that can be used to generate human-like responses. ChatGPT has demonstrated the capability of performing professional tasks such as preparing legal documents, and it is considered able to receive a B/B-grade in a Wharton MBA exam  (Terwiesch, 2023).
Although ChatGPT made the headlines for its remarkable performances, in this paper, we focus on GPT-3, that is probably less powerful, but better documented by scientific publications and available with an API, making it easier to do automated experiments. We used the version nicknamed text-davinci-003 (from now on, DaVinci), trained on 570GB of Internet texts in order to tune 1. 75 • 10 11 parameters  (Brown et al. , 2020) , thus being one of the largest LLMs publicly available. DaVinci was able to correctly answer half of the challenging questions of a common sense reasoning dataset of multiple-choice questions collected from 3rd to 9th grade science exams, and two thirds of the easy questions  (Brown et al. , 2020). Thus we wanted to test how well DaVinci would perform at tasks designed to elicit problem solving and computational thinking abilities, such the ones proposed in the Bebras contest.
LLMs and Reasoning Tasks GPT-3 has been presented in  (Brown et al. , 2020)  as a new autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model. Its performance was tested in the so called "few shot" setting, in which the model is given a few demonstrations of the task, but the neural network weights are not updated (as would happen instead in a "fine tuned" setting): a question typically has a context and a desired completion, and few-shot works by receiving 10-100 examples of context and completion, and then one final example of context, with the model expected to provide the completion. In particular, GPT-3 was tested on ARC  (Clark et al. , 2018) , a common sense reasoning dataset of multiplechoice questions collected from 3rd to 9th grade science exams.
On the 'Challenge' version of the dataset (filtered to questions which simple statistical or information retrieval methods are unable to answer correctly), GPT-3 is able to correctly answer 52% of the questions, 70% for the 'Easy' version. (Kojima et al. , 2022)  shows that a careful choice of the prompt can greatly affect the performance. In particular, adding a "chain-of-thought" prompt ("Let's think step by step:") can improve GPT-3 performance on arithmetic word problems from 18% to 79%, or even 93% with the additional help of "few shots". However, the performance on com-mon sense and other reasoning tasks datasets remains much lower (52-68%) and rather independent of the "chain-of-thought" prompt. (Terwiesch, 2023)  tested ChatGPT, an improved version of GPT-3 (GPT-3. 5) "fine-tuned" to conversational tasks (i.
e. , the model was specifically trained on conversational examples), on five questions of the Operations Management course whithin the MBA program at Wharton. ChatGPT did very well at basic operations management and process analysis questions including those based on case studies, providing correct answers and excellent explanations. However, it made surprising mistakes in relatively simple calculations at the level of 6th grade math. Moreover, Chat-GPT was not capable of handling more advanced process analysis questions, even when they were based on fairly standard templates, in particular with multiple products and problems with stochastic effects. (Saparov and He, 2022)  studied the performance of GPT-3 with respect to logical (deductive) reasoning and they found it capable of making correct individual steps, but when multiple valid deduction steps are available, GPT-3 is not able to systematically explore the different options.
OpenAI also offers a version of GPT-3 fine tuned on publicly available code from GitHub, known as Codex  (Chen et al. , 2021). Codex has been found rather effective on typical introductory programming problems, scoring within the top quartile of CS1 students  (Finnie-Ansley et al. , 2022). However, it failed on problems which placed restrictions on what features could be used in the solutions. (Sarsa et al. , 2022)  found that Codex can also be very useful in helping create programming exercises that are sensible, novel and applicable, althought the explanations provided might contain several inaccuracies. (Raman and Kumar, 2022)  starts from the assumption that LLMs will soon have a great impact on education, and especially computer science education. Thus they suggest to reduce the emphasis on code writing which characterizes current CS programs, since this skill will become less useful when LLMs will be able to write sensible code.
Selection of Tasks We submitted to DaVinci 54 tasks, written in English. All tasks were selected from the pools of tasks prepared during the International Bebras Task Workshop 2022 and 2021 (317 tasks overall). As DaVinci is a language model, we needed tasks that do not rely on graphical information. We looked for: • textual tasks, i. e. , tasks in which all information needed to solve them is expressed in the text, without referring to charts or interactive elements. Images, if present, have a decorative role only; • quasi-textual tasks, i. e. , tasks that contain an image/diagram whose informative content can be easily put into textual format (for example, in task 2022-CA-03 a simple textual description of the food present in each plate was used to replace the pictures of the plates). When we selected a task, we also tagged it according to the cognitive task it requires to be solved, using macro levels inspired by the revised Bloom taxonomy  (Anderson and Krathwohl, 2001 ): • Execution (EX): a typical EX task describes a procedure, and solvers are asked to execute it on a specific instance of a problem; • Analysis/Evaluation (AE): a typical AE task requires to analyze a situation or a procedure and draw some conclusion, e.
CONCLUSIONS As we showed, the apparent coherence of DaVinci's answers and explanations does not go along with their logical accuracy, consistency, and rigor. The perceived logical soundness of the explanations is in fact only in the "eye of the beholder"  (Bender et al. , 2021). Humans tend to attribute intentions and beliefs to any interactive entity, even when by construction the interlocutor does not have any. Indeed human-human communication is a jointly constructed activity and we use the same facilities for producing language that is intended for audiences not co-present with us (readers, listeners, watchers at a distance in time or space) and in interpreting such language when we encounter it  (Bender et al. , 2021). However, DaVinci (and in fact all LLMs) is nothing more than a "stochastic parrot" which somewhat repeats the patterns seen during its training, and yet it is almost impossible to avoid to see in its answers a straighforward reasoning process.
DaVinci's authoritative style can be deceptive and it requires a developed critical thinking attitude to debunk. On the one hand, this is worrisome because it adds to the already imperative need for critical sense and ability to research and verify sources imposed by the modern digital society, in which there is plenty of information made public to a broad audience by people with malicious intent or simply not experts in that field. On the other hand, this very need can be exploited to implement more active and constructive teaching activities, in which students are required precisely to develop such critical sense by challenging themselves in analyzing and correcting the seemingly sensible answers of an LLM. All in all, we believe DaVinci and its underlying GPT-3 system is an outstanding achievement for natural language processing, with human-like fluency and a surprising ability to generate on-topic sentences.
ACKNOWLEDGMENTS  M. Lodi 's work has been supported by the  Spoke 1 "FutureHPC & BigData " of the  Italian Research Center on High- Performance Computing, Big Data and Quantum Computing (ICSC ) funded by  MUR Missione   4 Componente 2 Investimento 1. 4:  Potenziamento strutture di ricerca e creazione di "campioni nazionali di R&S  ( M4C2-19 )" - Next Generation EU (NGEU ).
We established an easily accessible framework for training and validation of 3D registration methods, which enabled the compilation of results of over 65 individual method submissions from more than 20 unique teams. We used a complementary set of metrics, including robustness, accuracy, plausibility, and runtime, enabling unique insight into the current state-of-the-art of medical image registration. This paper describes datasets, tasks, evaluation methods and results of the challenge, and the results of further analysis of transferability to new datasets, the importance of label supervision, and resulting bias.
The EMPIRE10 challenge provided the most comprehensive evaluation including distances of manual landmark pairs, fissure segmentations, and Jacobian determinant values of the deformation field. This challenge also required (original) participants to perform live registrations during the MICCAI workshop in Beijing and therefore employed a time constraint on the computations. The Continuous Registration Challenge co-organised with WBIR 2018 aimed at combining multiple tasks from previous benchmarks (lung CT and inter-patient brain MRI). It addressed assessing registration quality as a service but is limited to algorithms that can be incorporated into the SimpleElastix framework and therefore had limited participation. Benchmark Papers: Several papers have compared multiple registration algorithms for a given dataset.
In contrast to challenges, these benchmark papers did not have (at least originally) an open workshop format that enabled wide-spread participation. Nevertheless, their findings provided meaningful insights. Starting from RIRE  [6] , which compared rigid-body alignment of head MRI (T1, T2), PET and CT, there have been several brain registration benchmarks -most notably the evaluation of 14 nonlinear iterative registration algorithms  [7]. Fewer studies analyzed abdominal registration, and included the evluation of six affine and non-linear algorithms on inter-patient registration of the "beyond the cranial vault" dataset  [8]. This study revealed large performance gaps and motivated our inclusion of this dataset to study the potential benefit of supervised (learning-based) algorithms. The DIR-Lab datasets  [9]  have been widely used to benchmark intrapatient CT lung motion estimation and provide a leaderboard for state-of-the-art comparison.
All landmarks are publicly available, which makes the dataset prone to overfitting on the test data. Survey Papers and Baseline Methods: Surveys on conventional medical image registration  [2] ,  [3]  have comprehensively reviewed typical categories of approaches including similarity metric, regulariser, and optimiser criteria. Due to the strong increase in the number of deep-learning-based registration paper in the last few years, several new surveys have been published (e. g. [4] ) extending the typical categories with deep-learning specific categories like supervision-type and network architecture. Moreover, the training data and thus the registered body region and image modality are more important for deep-learning-based methods and get more into the focus of those survey papers. While few papers have evaluated their proposed registration method on more than two different registration tasks, there is a variety of public methods SyN  [10] , Elastix  [11] , NiftyReg  [12]  and deeds  [13] , and Voxelmorph  [14]  that are commonly used as baseline or comparison methods.
B. Contributions Learn2Reg provides both datasets and an easily accessible benchmark for the first comprehensive evaluation of a widerange of methods for inter-and intra-patient, mono-and multimodal medical registration. We introduce a complementary set of metrics, including robustness, accuracy, plausibility and speed, that follows the principles defined by the BIAS group  [5]  and could become a de-facto benchmark for new algorithms. Further analysis of label bias (for supervised methods), domain transfer and statistical testing of significant differences across algorithms and types of methods highlight the complementary strength and weaknesses of learning vs. non-learning-based approaches.
The developed algorithms were used to register the scan pairs of the validation dataset. The resulting displacement fields on the validation dataset were submitted and evaluated using grand-challenge. org. Challenge participants were allowed to create five submissions per day to this phase. • Phase 2 -Test phase: Within one week after the test data release, the participants had to send either the generated displacement fields to the organisers or a Docker container containing the algorithm. A Docker submission was preferred and made more attractive by evaluating the runtime of the algorithm. To remove entry barriers for new participants with expertise in deep learning but not necessarily registration, the organisers provided pre-preprocessed data (resample, crop, pre-align, etc. ). A detailed description of the used preprocessing is given in section II-B.
B. Tasks Learn2Reg consists of six clinically relevant complementary tasks (datasets). Table  1  summarises the dataset details, and we discuss them in detail below. CuRIOUS: EASY-RESECT  [15]  is a simplified subset of the original RESECT dataset  [16] , previously used in the MICCAI CuRIOUS challenges  [17]. The dataset contains 22 training and 10 testing subjects with low-grade brain gliomas, intended to help to develop MR vs. US registration algorithms to correct tissue shift in brain tumor resection. For the Learn2Reg challenge, we included T1w and T2-FLAIR MR scans, and spatially tracked intra-operative ultrasound volumes. All scans were acquired for routine clinical care of brain tumor resection procedures at St Olavs University Hospital (Trondheim, Norway). Matching anatomical landmarks were annotated between T2-FLAIR MR and 3D ultrasound volumes  [16]  to enable evaluation of the registration accuracy.
During pre-processing, for each subject, the T1w scan is rigidly registered to the T2-FLAIR scan, and both scans are resampled to the same coordinate space as the 3D ultrasound volume yielding fixed voxel dimensions for all scans (256×256×288) at an isotropic resolution of approximately 0. 5 mm. Hippocampus MR: This dataset consists of 394 MR scans of the hippocampus region acquired in 90 healthy adults and 105 adults with a non-affective psychotic disorder taken from the Psychiatric Genotype/Phenotype Project data repository at Vanderbilt University Medical Center (VUMC). The hippocampus head and tail were manually traced in all scans. Previous to the Learn2Reg challenge, the dataset was used as part of the Medical Segmentation Decathlon  [18]. Abdomen CT-CT: This task tackles inter-patient registration of abdominal CT scans, which enables statistical modelling of variations of abdominal organs for abnormality detection, and can provide a canonical atlas space for further investigations.
The dataset contains 50 abdominal CT scans (30 for training, 20 for testing) with 13 manually labelled anatomical structures: spleen, right kidney, left kidney, gall bladder, esophagus, liver, stomach, aorta, inferior vena cava, portal and splenic vein, pancreas, left adrenal gland and right adrenal gland  [8]. The images were registered affinely in a groupwise manner and resampled to the same voxel resolution and spatial dimensions (192×160×256). Abdomen MR-CT: The dataset was compiled from public studies of the cancer imaging archive (TCIA) that contained paired scans of both MRI and CT from the same patients. In particular, 16 MRI and CT scans from the following studies, TCGA-KIRC, TCGA-KIRP, and TCGA-LIHC, are included in Learn2Reg -that cover routine diagnostic scans and followup imaging for kidney surgery (donation).
CuRIOUS Hippocampus ••••• •• ••••• TABLE I: Overview of all six Learn2Reg tasks adressing the imminent challenges of medcial image registration: multimodal scans • , few/noisy annotations • , partial visibility • , small datasets • , large deformations • , small structures • , unsupervised registration • and missing correspondences •. 2) Ranking Scheme: We ranks methods using statistically significantly different results. For each metric applied in a task, methods are compared against each other (Wilcoxon signed rank test with p<0. 05), ranked based on the number of "won" comparisons and finally mapped to a numerical metric rank score between 0. 1 and 1 (with possible score sharing). A task rank score is then obtained as the geometric mean of individual metric rank scores. All methods for which no metric is available (not submitted to the task, no Docker container submitted) share the lowest possible metric rank score of 0.