dataesr/scientific-paragraphs-categorization huggingface dataset:
EndoVis challenge 2019 Based on our data set, 12 international teams trained and evaluated their algorithms during the EndoVis challenge 2019 (EndoVis challenge, 2021) within the sub-challenge for "Surgical Workflow and Skill Analysis" hosted in conjunction with the Medical Image Computing and Computer Assisted Intervention conference (MICCAI). The aim of this sub-challenge was to investigate the current state of the art on surgical workflow analysis and skill assessment in laparoscopic cholecystectomy on one comprehensive dataset. Specifically, the aims were (1) surgical phase recognition with high precision and recall, (2) surgical action recognition with high precision and recall, (3) surgical instrument presence detection with high precision and recall as well as (4) surgical skill assessment with a low mean absolute error.
The challenge committee members were M. Wagner, S. Bodenstedt, A. Kisilenko, H. Kenngott, L. Maier-Hein, S. Speidel and B. Müller-Stich with affiliations as stated in the authors list. The award policy was to award a prize to each winner of one of the four tasks, if at least three participants entered a submission. Members of the organizing institutes were allowed to participate, but were not eligible for awards. On May 31st 2019, the first part of the training dataset consisting of 12 videos was published, followed by the second part, also consisting of 12 videos, on August 15th 2019. With the second part, the organisers' evaluation software scripts for computing metrics and rankings were provided. The evaluation and submission period of the Docker containers was between October 1st and 7th 2019.
It is unclear whether data was used for pre-training. 3. 2. 7. Team Konica Minolta (Action & Instrument) Team Konica Minolta used a 101-layer ResNeXt  (Xie et al. , 2017)  with Squeeze-and-Excitation block  (Hu et al. , 2019)  as their base network. They attached a fully-connected layer for the instrument presence detection and one for the action recognition to the network. Furthermore, a third layer was added for detecting instrument super-classes. Here, one superclass generally consisted of a combination of instruments that regularly occur together, e. g. grasper and scissors. ResNeXt was pretrained on ImageNet. 3. 2. 8. Team MEVIS (Phase) Team MEVIS used a ResNet  (Qiu et al. , 2017)  for extracting visual features from the laparoscopic video frames. A LSTM was used to incorporate information from past frames.
For each task, a fully-connected layer was added to the LSTM. During training, frames at the beginning of a video clip were given a weight of zero to allow the network a "warm-up phase". ResNet was pretrained on ImageNet and Cholec80. In addition, video-clips from 16 Cholec80 videos were added during training to increase the data set and video-clips were sampled stratified based on the phase-labels. 3. 2. 9. Team NCT (Phase, Action & Instrument) Team NCT used a multi-task approach for determining from a laparoscopic video the phase, which instruments were visible and what actions were being performed. The approach used a ResNet  (Qiu et al. , 2017)  for feature extraction, a LSTM was used to propagate information from past frames along the temporal axis and used the information for predicting current frames.
Assessment Method The properties of the dataset were analyzed with descriptive statistics mean and standard deviation (SD). To calculate results of the EndoVis challenge for the tasks of phase, action and instrument recognition, the F1-score 1 = 2 ⋅  ⋅   +  over all classes was computed. The final ranking was calculated by averaging the F1-scores per video. We selected the F1-score as it takes both false positives and negatives into account. We decided to average over all classes and all videos so that each class and video would have the same weight in the final score regardless of occurence rate and length. For skill assessment, the mean absolute error  = 1  ∑  =1 |  ̂-  | over all criteria was evaluated and ranked. We decided to use the MAE for skill assessment, as it gives a good impression on how far predictions are on average from the reference.
To analyze the ranking stability, bootstrapping methods were applied as suggested in  (Maier-Hein et al. , 2018). The rankings were recalculated over 1,000 bootstrap datasets drawn from the test set to assess the variability in ranks against small perturbations. Kendall's tau  (Kendall, 1938)  correlation coefficient was used to quantify the changes between the original and the bootstrap rankings for all three tasks. It is ranged between -1 and 1, where a value of 1 indicates an unchanged ranking and -1 a reversed ranking. Finally, pairwise Wilcoxon signed rank tests with a 5% alpha level and adjustment according to Holm for multiple testing were computed between each algorithm pair. The tests were used to check for algorithms that were significantly superior in terms of performance compared to their competitors.
The left hand of the surgeon, as well as the assistant hand performed only grasp (A0) and hold (A1). Cut (A2) and clip (A3) were performed by the surgeons right hand only. Mean total action length per procedure of hold (A1) was 1497. 8 s ± 620. 6 s (left hand of the surgeon), 68. 6 s ± 56. 6 s (right hand of the surgeon) and 1356. 3 s ± 830. 1 s (assistant hand). However, the performer (i. e. surgeon right or left hand, assistant hand) was annotated, but not published as part of the dataset.
HeiChole Benchmark Our study introduces the HeiChole benchmark by presenting a novel open data set for surgical workflow and skill analysis together with the results of the EndoVis challenge 2019, subchallenge for surgical workflow and skill analysis, and its comparative validation of machine learning algorithms for automatic phase, instrument, action and skill recognition in laparoscopic cholecystectomy. In the following sections we will discuss achievements and limitations of the data set and the performance of the algorithms for each task. Finally, we will outline potential clinical applications as well as future research directions for surgical workflow and skill analysis.
Surgical phase recognition In comparison to the previously achieved average precision of 91% for automatic phase recognition on the Cholec80 data set  (Twinanda et al. , 2017) , the best participating team in this EndoVis challenge reached a F1-score of 65%. One possible reason for the lower results is that, in contrast to the Cholec80 data set, there was no rigid phase order for the HeiChole data set, which makes the recognition of the phases more complex. Furthermore, phases such as clipping and cutting (P2) and gallbladder packaging (P4) proved to be more challenging to recognize than others, because instruments typical for these phases, such as scissors for P2 or the specimen bag for P4, were also briefly used in previous or subsequent phases. In surgical reality, there are many variations in the workflow, especially for the more difficult cases that at the same time could benefit more from the help of computer assistance.
Thus, the reflection of variations is important for the future clinical translation of the results. Another important factor in improving the results is increasing the size of the open data set to enhance the training of machine learning algorithms. For example, phase recognition by machine learning on a data set of 1243 laparoscopic cholecystectomies was published, but only hyperparameters, but not the training data were made publicly available  (Bar et al. , 2020). From a technical aspect, it is interesting to note that the top three teams (CAMMA, CUHK and HIKVision) all used a combination of CNNs with a recurrent network, here in all cases a LSTM. While it can be expected that these methods outperform approaches that do not take temporal information into consideration (e. g. Wintegral), it is of interest to note that the recurrent approaches also generally outperform methods based on 3D convolutions.
The submitted methods were all able to similarly well detect the action "Hold" (F1 scores ranging from 0. 84 to 0. 91), but could not reliably detect any other action. The average F1 scores for each action apart from "Hold" for each method was well below 0. 03. Since "Hold" was the most common action in the dataset, we conclude that the current training dataset did not adequately mirror the variance in the other actions.
Reducing bias in the challenge The introduction and validation of the HeiChole benchmark succeeded by means of the challenge design of the EndoVis challenge. This allowed the comparison of the results of several different algorithms submitted by international teams, demonstrating the usability of the HeiChole benchmark as an validation tool for phase, instrument, action and skill recognition. As biomedical challenges become increasingly important for the evaluation and validation of methods for surgical data science, quality control is of utmost relevance to ensure reproducibility and comparability of the results  (Maier-Hein et al. , 2018). For example, this was addressed in the EndoVis challenge by providing detailed information about the data collection and processing in response to the challenge design questionnaire required by MICCAI 2019 for submission.
n=15) were recorded with a laparoscopic 2D camera (Karl Storz SE & Co KG, Tuttlingen Germany) with 30° optics, a resolution of 960x540 pixels and 25 frames per second. The operation videos at Salem Hospital (n=15) and the GRN-hospital Sinsheim (n=3) were recorded with the laparoscopic 2D camera ENDOCAM Logic HD (Richard Wolf GmbH, Knittlingen, Germany) with 30° optics, a resolution of 1920x1080 pixels and for the greater part 50 frames per second. Three operations at Salem Hospital were recorded with a resolution of 720x576 pixels and 25 frames per second. Every video starts at the first insertion of the laparoscopic camera into the patient's abdomen and ends with the last removal of the laparoscopic camera. The videos were split into the training (n=24) and test (n=9) data set. In the training dataset, videos from Heidelberg University Hospital and Salem Hospital are equally represented (n=12 each).
Acknowledgements We thank  T. Nguyen  (team IGITech),  L. Zhao ,  Z. Ge ,  H. Sun ,  D. Xie  (team HIKVision) and  M. Guo ,  C. Gong ,  D. Liu  (team VIE-PKUE) for participation in the challenge. They were not listed as coauthors, because they did not respond to the requested authors confirmation, but agreed to publish their results when participating in the challenge.
Funding This work was supported by the  National Center for Tumor Diseases (NCT) Heidelberg  within the  Cancer-Therapy-Program "Surgical Oncology ", by the  German Federal Ministry of Economic Affairs and Energy within project "InnOPlan"  (grant number  BMWI 01MD15002E ), by the  German Federal Ministry of Health  within project " Surgomics " (grant number  BMG 2520DAT82D ), by the  German Research Foundation DFG  within the  of Excellence EXC 2050: "Centre for Tactile Internet  with  Human-in-the-Loop (CeTI)"  (project number  390696704 ), by the  German Academic Exchange Service (DAAD)  with a scholarship for  T. Davitashvili  for studying medicine in Germany ( Scholarship program  ID: deutsche Auslandsschulen, 2017 (57314589)), by the  Helmholtz Imaging Platform (HIP) , a platform of the Helmholtz Incubator on Information and Data Science, by  Agence Nationale pour la Recherche  ( ANR-16-CE33-0009 ,  ANR-10-IAHU-02 ) and  Banque Publique d'Investissement (BPI CONDOR) , by the  Richard und Annemarie Wolf-Stiftung , by the  Guangdong Key Area Research and Development Program  ( 2020B010165004 ), by the  Shenzhen Key Basic Science Program  ( JCYJ20180507182437217 ), by the  Hong Kong RGC   TRS  Project No.
( n=15) were recorded with a laparoscopic 2D camera (Karl Storz SE & Co KG, Tuttlingen Germany) with 30° optics, a resolution of 960x540 pixels and 25 frames per second. The operation videos at Salem Hospital (n=15) and the GRN-hospital Sinsheim (n=3) were recorded with the laparoscopic 2D camera ENDOCAM Logic HD (Richard Wolf GmbH, Knittlingen, Germany) with 30° optics, a resolution of 1920x1080 pixels and for the greater part 50 frames per second. Three operations at Salem Hospital were recorded with a resolution of 720x576 pixels and 25 frames per second. Every video starts at the first insertion of the laparoscopic camera into the patient's abdomen and ends with the last removal of the laparoscopic camera. The videos were split into the training (n=24) and test (n=9) data set. In the training dataset, videos from Heidelberg University Hospital and Salem Hospital are equally represented (n=12 each).
Terrasolid based Approach TerraScan is a powerful software solution for processing LiDAR point clouds  [8]. With the Terrasolid software, the aim was to create a classification routine by generating a smooth surface that can then be classified while disregarding residual noise. From the original point cloud, less reliable, noisy and isolated points are removed. In a subsequent step, a smoothed surface is generated and a vegetation index is calculated for the ensuing ground classification. The Terrasolid-based ground classification algorithm is limited in distinguishing between grass vegetation and higher vegetation such as shrubs and trees. Along the channel bed, this limitation has little significance. However, when the derived terrain model is used for numerical simulations, the results require a great deal of caution when analysing them.
The toolbox The toolbox, implemented in Matlab, contains a small set of routines to create and analyze spike trains. A spike train is represented as an array of floating-point numbers, each number representing time relative to a time reference locked to the beginning of the train. Spike trains are generated according to the thinning method. The input time series (instantaneous rate aka driving function) is scanned for its maximum value, this value parametrizes a homogenous Poisson process that produces a train of inter-spike intervals at that maximum rate. These are cumulatively summed (assuming a first spike at time 0) to obtain a train of spike times. This spike train is then thinned probabilistically depending on the value of the nominal instantaneous rate at the time of each spike, to simulate the timevarying Poisson process.
Finally, the spike train is again thinned probabilistically according to a recovery function. Initial spike generation and thinning are vectorized and thus fast, but the final thinning requires a loop. The toolbox includes a few additional routines to modify the spike train (jitter, cancellation filter, c. f. de Cheveigné 2021, 2023), and for routine statistics such as first-order and allorder inter-spike interval histograms, peri-stimulus time (PST) histograms, and crosscoincidence histograms). The toolbox is mainly intended as an add-on to existing auditory model toolboxes (e. g. Osses et al. 2022 ). The toolbox is archived at https://zenodo. org/badge/latestdoi/635354073.
Further work is needed to clarify the role of feedback loops, feedforward loops, and other paths from components with multiple targets to components with multiple regulators in the dynamical properties of cuttable networks to elucidate whether the structural conditions for linear cuts could then be further generalized. We have implemented the linear extension of Boolean networks in the bioLQM software  (Naldi, 2018) , enabling the use of the extended semantics in existing software tools supporting the classical asynchronous semantics. Note that efficient analysis based on trap spaces does not require this explicit extension and can be performed directly on the original Boolean networks using existing implementations of trap spaces identification in PyBoolNet  (Klarner et al, 2017)  or BioLQM.
Acknowledgments We thank  Heike Siebert  for insightful discussions. Funding AN was supported by the  Deutsche Forschungsgemeinschaft  ( GRK 1772 ) and  French National Research Agency  ( ANR-MOST  project  ANR-16-CE18-0029  " BIOPSY "). AR was supported by the  French National Research Agency  ( Young Researcher  project  ANR-18-CE40-0002-01  " FANs ").
Identification of 11 C: the TFC algorithm Due to their long lifetimes, the events from 11 C, 10 C, and 6 He backgrounds cannot be removed by a short-time veto cut following a muon event. Fortunately, the spallation reaction by the parent muon is followed by a cosmogenic decay and a neutron capture followed by the emission of a characteristic 2. 2 MeV γ-ray, which allows us to use the so-called Three-Fold-Coincidence (TFC) algorithm  [61]. By exploiting the spatial and time coincidence of those events, this method identifies space-time regions where the creation of cosmogenic backgrounds is highly probable: typically, the selected regions are a cylinder around the muon track and spheres around the point where the γ from the neutron capture is reconstructed. Based on this, the JUNO solar neutrino dataset is split into two complementary data samples: TFC -tagged and TFC -subtracted.
The fit is simultaneously performed on two complementary partitions of the available dataset: the so-called TFC-subtracted one, depleted in 11 C by means of the Three-Fold-Coincidence technique (see section 3. 3), and the complementary TFC-tagged one, more populated in 11 C. The reference PDFs for the signal and backgrounds used in the fit are obtained from complete Monte Carlo simulations of the JUNO detector. In the following, we describe in detail the fundamental steps for these sensitivity studies: the production of the reference energy distribution for each background and signal species (section 4. 1), the production of toy datasets (section 4. 2), and the fit to extract the contribution of every background and signal species (section 4. 3). Note that this analysis assumes a perfect knowledge of the detector energy response and of the theoretical shape of the energy distributions for neutrinos and background.
Production of reference energy distribution (PDFs) The energy PDFs used in the fit are obtained from Monte Carlo simulations performed with the official JUNO offline software framework based on Geant4 and customized for the experiment  [64]. This code fully describes the detector response, taking into account all the physics processes occurring in the detector: from energy deposition, light emission, propagation and detection, up to the electronics signal processing and event reconstruction algorithms. The event reconstruction is performed by the official JUNO software code. The analysis energy estimator is the total charge collected by each PMT, expressed as the number of detected photoelectrons (p. e. ), subtracted by the mean dark noise hits expected, and including an effective correction to account for the non-uniformity of the detector energy response.
JCAP10(2023)022 All the neutrino signal and background components have been simulated uniformly within a r sim < 15. 0 m sphere, while the fiducial volume employed for this sensitivity analysis is a r FV < 14. 0 m sphere. For each of the species of interest, a PDF of the reconstructed energy variable can be built directly from the corresponding Monte Carlo sample. Because of the huge statistics that will be acquired in JUNO, the number of simulated events is smaller with respect to what is expected for a real dataset. In principle this could bias the fit result, due to statistical fluctuations in the PDFs. We solve this issue by applying an optimized low-pass filter  [65]  on the generated PDFs, suppressing as much as possible the high-frequency fluctuations without distorting the spectral features.
Toy dataset generation The TFC-tagged and TFC-subtracted energy distributions for each toy dataset are obtained by randomly sampling the PDFs of each neutrino and background components. The sampling is Poissonian, assuming the expected number of events as central value. Examples of generated datasets, for six years of data in the different background scenarios, are shown in figure  3 , highlighting separately the 7 Be, pep, 13 N, and 15 O solar neutrino contributions. The TFC-subtracted and TFC-tagged datasets are shown in the left and right panels, respectively. Note that in the Very Low and Low Background scenarios, the signal due to 7 Be solar neutrinos (a characteristic Compton-like shoulder at ≈ 1000 p. e. ) can be easily seen by eye. On the other hand, in the most pessimistic High Background scenario, the 238 U and 232 Th chain decays dominate the count rate in the entire ROI.
The contribution of every individual neutrino and background species considered for the sensitivity analysis is shown with lines of different colours, for the Medium Background scenario in figure  4. The contributions of all the neutrino and background species considered for the sensitivity analysis for six years of data taking, in the Medium Background scenario. The TFC-subtracted and TFC-tagged datasets are reported in the left and right panels, respectively. The 7 Be, pep, 13 N, and 15 O solar neutrino contributions are shown as red, orange, pink, and golden solid lines, respectively. The background contributions ( 210 Bi, 210 Po, 85 Kr, 40 K, 238 U chain, 232 Th chain, 11 C, 10 C, 6 He) are displayed as dotted lines.
Multivariate fit To extract the signal and background contributions from each dataset, the generated TFCtagged and the TFC-subtracted spectra are fitted simultaneously, minimizing a Poissonian binned likelihood function. This is built as the product of two independent standard Poissonian likelihoods associated to the TFC-tagged and TFC-subtracted spectra. The fit parameters are the numbers of events for each involved signal and background components. Constraints on some of the fit parameters, coming from information which is external and independent from the spectral fit, can be applied in the form of multiplicative Gaussian pull terms to the overall likelihood. In the following, we don't apply any constraint, unless explicitly stated. The neutrinos and the internal radioactivity ( 210 Bi and 210 Po from the 210 Pb chain, 85 Kr, 40 K, 238 U chain, 232 Th chain) populate the TFC-tagged and TFC-subtracted datasets in the same relative proportions.
The long-lived 11 C isotope instead is present in both datasets, but with two different contributions due to the application of the TFC algorithm. To account for this, two independent fit parameters are included ( 11 C sub and 11 C tag ). The other two most relevant cosmogenic isotopes 10 C and 6 He are included in the TFC-tagged dataset only, given their negligible contribution to the TFC-subtracted dataset  [13]. Indeed, the estimated rate for these two isotopes in TFC-subtracted dataset rates are R( 10 C) ROI Sub < 0. 025 cpd/kton and R( 6 He) ROI Sub < 1. 27 cpd/kton, respectively. We have not included in this analysis the short-lived cosmogenic isotopes since we assume that the veto applied after each muon crossing the detector will be sufficient to effectively remove them (see section 3.
JCAP10(2023)022 5 Solar neutrino spectroscopy Thanks to the large active mass and the unprecedented energy resolution, JUNO will be very competitive in the solar neutrino spectroscopy field. Of course, the achievable precision to 7 Be, pep, and CNO fluxes is strongly related to the overall exposure and to the signal over background ratio, which in turn depends on the scintillator radiopurity levels. We have performed the sensitivity studies as a function of exposure and for the four different background scenarios described in section 3. For each exposure and background condition a large number (10 4 ) of toy-datasets is simulated and fitted in order to evaluate the capability of the multivariate fit to disentangle the signal and background components correctly. The median of the relative statistical error distribution is quoted as the detector sensitivity; the left and right errors are extracted as the distances between the median and the 34% C.