dataesr/scientific-paragraphs-categorization huggingface dataset:
Explaining through latent space analysis The PGAAE trained for abele projects the ISIC dataset into a 256-dimensional latent space with a coherent posterior distribution. Indeed, as shown in  [14] , as a side effect of abele, we can exploit the latent space to visualize the level of proximity of individual instance of the dataset and gain useful insights. In particular, we believe that such visual aid can help medical expert and data scientist to better understand different skin cancer characteristics and exploit it to further improve the classification performance, or the trust in the explainer. We adopt a Multidimensional Scaling (MDS)  [43]  as a form of dimensionality reduction to translate information about pairwise distances among latent projections into a configuration of the same cardinality mapped into a Cartesian plane.
Also, we conducted a survey involving real user experts and not experts in skin cancer and of healthcare domains. The survey supports the hypothesis that explanation methods without consistent validation are not useful. As future research directions, it will be interesting to apply abele explainer to different diseases and health domains, especially domains where the only meaningful data is the raw image or scan of a particular internal body portion. Indeed, in skin lesion cancer also the touch plays an important role in doctors' analysis, not just the image. Also, we would like to extend the user visualization module presented to a real-time explanations generator. This improvement would require significant efforts and resources as in the current implementation extracting explanations can be time-consuming depending on the image.
Acknowledgements This work is supported by the  European Community  under the  Horizon 2020 programme : G. A. 871042   SoBig-Data++ , G. A. 952026 HumanE AI  Net, G. A. 101092749 CREXDATA ,  ERC-2018-ADG G. A. 834756 XAI , G. A. 952215   TAILOR , and the  NextGenerationEU programme  under the funding schemes  PNRR-PE-AI  scheme (M4C2, investment 1. 3, line on AI)  FAIR (Future Artificial Intelligence Research) , and " SoBigData. it -Strengthening the Italian RI for Social Mining and Big Data Analytics " -Prot. IR0000013.
Creation of pangenome references We constructed pangenome references of the chicken genome using two different methods, both used by the Human Pangenome Reference Consortium  [30] : PanGenome Graph Builder (PGGB)  [30]  and minigraph-cactus  [39]. PGGB and minigraph-cactus both take multiple assemblies as input, perform whole-genome alignments on them, and derive a pangenome graph from these alignments. We made a preliminary graph using each method and five chromosome-level assemblies (Table  1 ). For minigraph-cactus, we then created a 30-assembly graph using these five chromosomelevel assemblies as well as the contig-level alternate haplotype assemblies of bGalGal4 and bGalGal5 and assemblies of both haplotypes of thirteen additional chickens from HiFi data (Supplementary Table  1 ). We did not create a 30-assembly graph with PGGB due to the computational intractability of the 5-assembly PGGB graph for downstream applications, as described below.
Sequencing and assembly of bGalGal4 and bGalGal5 20 One female Ross 308 (Aviagen) and one female Cobb 550 (Cobb-Vantress), both commercial broiler chickens, were euthanized in the framework of a research experiment at 38 days of age. Cardiac puncture was immediately employed to collect 12 aliquots of 100 ul of blood in tubes with EDTA and 1 ml of ethanol >99. 7% from each animal. Samples were frozen at -20 °C. For both assemblies (bGalGal4 and bGalGal5), we followed the VGP 2. 0 pipeline  [12]. We generated 32x Pacbio HiFi data on a Sequel IIe, and then used cutadapt  [60]  to trim off adapters that were not trimmed in the Pacbio software processing. We assembled contigs using HiFiasm v0. 14  [61] , generating a semi-haplotyped phased primary contig and alternate contig assembly. From the primary assembly, we removed false haplotype duplication and placed them in the alternate using purge_dups v1.
2. 5  [62]. We then scaffolded the contigs with Bionano Genomics optical maps (319x and 459x respectively), generated on a Saphyr instrument using DLE label, with Bionano Solve. We then further scaffolded with Arima Genomics Hi-C v2 (65x and 122x respectively), using salsa v2. 2  [63]. The primary assembly was then curated using gEVAL  [64] , structural errors corrected, and chromosomes named according to their numbers in the bGalGal1 GRC7g reference. 10X Genomics data were also generated, and used for orthogonal validation, but not scaffolding. The primary and alternate assemblies were deposited in NCBI under accession numbers GCA_027557775. 1 (bGalGal4) and GCA_027408465. 1 (bGalGal5), and all data are available in Genome Ark https://genomeark. github. io/genomeark-all/Gallus_gallus/.
Sequencing and assembly of additional chickens High molecular weight (HMW) DNA from blood of 13 juvenile male chickens (Supplementary Table  1 ), maintained and bled under ADOL IACUC-approved Animal Use Protocol #2019-15 for breeder management, was sequenced on the Pacific BioSciences Sequel IIe. HMW samples were sheared using a Diagenode Megarupter3 shearing device targeting 18-22KB fragments. Libraries were prepared with the PacBio SMRTbell Prep Kit 3. 0. Library size distribution was determined on the Agilent Femto Pulse and a Qubit fluorometer was used to measure concentration. Sequencing polymerase was bound to the SMRTbell libraries with the Binding Kit 3. 2, and run on Sequel IIe with the Sequel II Sequencing Kit 2. 0 and SMRT Cell 8M. HiFi data was collected with Instrument Control Software Version 11.
Additional sequence analysis We determined the amount of additional sequence contributed to the graph by each sample through an iterative process. First, we removed all nodes traversed by the Huxu assembly from the graph as it is the most complete assembly. Then, for each remaining bird, we summed up the length of all nodes traversed by either haplotype of this bird, found the bird with the largest sum, and removed all nodes traversed by this bird's haplotypes from the graph. We repeated this process until there were no samples remaining. The python program we wrote for this purpose is included in the repository cited in the Code Availability statement.
Genotyping We genotyped the 100 chickens based on these alignments using elprep  [74]  v5. 1. 2, a multithreaded reimplementation of GATK. Briefly, we generated an elfasta sequence reference for bGalGal1b using the command "elprep fasta-to-elfasta", created a list of sites from the minigraphcactus vcf output with SVs larger than 1000bp filtered out using the command "elprep vcf-to-elsites", and ran the "sfm" command with settings as recommended in the manual to generate a gvcf for each bird, which we then combined into a single gvcf with GATK CombineGVCFs and joint genotyped with GATK GenotypeGVCFs  [17]. The location of our scripts for genotyping, as well as all other analyses in this paper, is given in the Data Availability section.
Genotyping method comparison To compare the respective outputs of the giraffe-and minimap-based genotyping pipelines, we used bcftools v1. 17  [69]  command "isec -c some" to create four vcf files: variants only detected by the giraffe pipeline, variants only detected by the minimap pipeline, giraffe pipeline calls of variants detected by both pipelines, and minimap pipeline calls of variants detected by both pipelines. We counted variants with QUAL>=10 in all of these files, subsetting by variant type with "bcftools viewv [snp|indel]". To compare the per-sample calls made by the respective methods for variants detected by both, we used "bcftools merge --force-samples" to create a single vcf containing calls made by both methods, and then used a custom python script (included in code availability) to calculate the percent agreement for each variant.
Reference bias estimation We estimated the amount of reference bias by calculating the mean fraction of reads mapping to putative heterozygous sites containing the alternate allele, and comparing to the expected value of 0. 5. We define putative heterozygous sites as positions with coverage of at least 10x where the portion of reads containing the minor allele is at least 25%, as in  [15]. Briefly, we filtered low-quality mappings and multimapping reads with "samtools view -F2304 -q10", created pileups with "samtools mpileup -d100 -no-BAQ", and piped the results to a custom C program to find putative heterozygous sites and calculate alternate allele frequencies at these sites. All code used to perform this analysis is in the project's code repository.
Funding This work was supported by  USDA NIFA  grants  2020-67015-31574  and  2022-67015-36218  and the  European Union 's  Horizon Research and Innovation Programme  under grant agreement No. 817729  (Project HoloFood). Computation for this work was performed on the high performance computing infrastructure provided by Research Computing Support Services and in part by the  National Science Foundation  under grant number  CNS-1429294  at the  University of Missouri, Columbia MO. The authors gratefully acknowledge the  Gauss Centre for Supercomputing e. V. (www. gauss-centre. eu) for funding this project by providing computing time on the GCS Supercomputer SuperMUC-NG at  Leibniz Supercomputing Centre  (www. lrz. de).
Availability of data and materials 30 The datasets generated and/or analyzed in the current study are available in NCBI repositories under BioProject accessions PRJNA838369, PRJNA838370, and PRJNA971225. The pangenome graph, a vcf of variants present in the graph, and vcfs of the resequenced chickens genotyped using both linear and pangenome methods are available in a Zenodo repository at [embargoed until final publication]. The code used to perform the analysis in the current study is available on GitHub at https://github. com/WarrenLab/chicken-pangenome-paper.
This paper reports on the second "Throughput" phase of the Tracking Machine Learning (TrackML) challenge on the Codalab platform. As in the first "Accuracy" phase, the participants had to solve a difficult experimental problem linked to tracking accurately the trajectory of particles as e. g. created at the Large Hadron Collider (LHC): given O(10 5 ) points, the participants had to connect them into O(10 3 ) individual groups that represent the particle trajectories which are approximated helical. While in the first phase only the accuracy mattered, the goal of this second phase was a compromise between the accuracy and the speed of inference. Both were measured on the Codalab platform where the participants had to upload their software. The best three participants had solutions with good accuracy and speed an order of magnitude faster than the state of the art when the challenge was designed.
It will collect data of increasing complexity and at an increasing rate with a large upgrade the so-called High Luminosity LHC (HL-LHC)  [6]  currently planned for 2027. The analysis pipelines of the proton collisions (or events) rely as an important step on the reconstruction of the trajectories of the particles within the innermost parts of the detector. The time to reconstruct the trajectories -in a constant magnetic field these would follow a helical path -from the measurements (3D points) is expected to increase faster than the projected computing resources. New approaches to pattern recognition are thus necessary to exploit fully the discovery potential of the HL-LHC. A typical event of this challenge would have about 100. 000 points to be associated into about 10. 000 trajectories. The state of the art was about 10s per event on relatively modern hardware when the challenge was designed.
Given that 10 to 100 billion such collisions need to be processed each year, the importance of the increase of the reconstruction throughput becomes evident. A dataset consisting of an accurate simulation  [7]  of an LHC-like experiment has been created, listing for each event the measured 3D points, and the list of 3D points associated to a true track. The dataset is large enough to allow for the training of data-intensive Machine Learning methods; the order of magnitude estimates are: ten thousand events, one billion measurement points, one hundred million trajectories ("tracks") to be found. In practice, the task is to build the list of 3D points belonging to each track. As usual, the solutions proposed by the participants were evaluated on a test set stripped of the ground truth. The final step of track reconstruction, i.
e. the inference of the particle properties (track parameters) at the particle's origin, was not a goal of the challenge, given that the estimation of track parameters by applying fitting or other inference techniques is believed 1https://www. kaggle. com 2https://competitions. codalab. org to be well understood and does not significantly drive the computing requirements. For the Accuracy phase, participants had to upload a solution file (in csv format) indicating how the points are clustered (like for a typical competition on the Kaggle platform), while for the Throughput phase participants had to upload their software to the Codalab platform, on which it was executed in a controlled environment. By doing so, the resource usage was measured in a standardized way, and the Throughput score was then derived from a combination of the accuracy and the inference speed.
This paper is organised as follows. Section 2 details the setup of the competition, the changes to the dataset with regards to the Accuracy phase and accuracy score evaluation, the score and the details of the implementation on the Codalab platform. Section 3 narrates the competition as it happened. Section 4 details the performance of the algorithms submitted. The top three algorithms are then detailed each in sections 5, 6 and 7, respectively, and 8 is the Conclusion.
Dataset update The dataset for this phase was produced with the fast detector simulation that is part of the ACTS project  [7]. The detector setup, as pictured in Fig. 1 , was unchanged with respect to the Accuracy phase  [8, 9] , and mimics a typical LHC general purpose experiment. The overall detector setup is as follows: A central silicon pixel detector with the 50 µm square pixels is enclosed by a Silicon short strip detector and an outermost long strip detector and embedded in a solenoidal magnetic field with a central field strength of 2 T. Minor adaptions to the dataset have been made, predominantly to correct issues that have been identified with the Accuracy dataset (those issues, however, were checked to not have any impact on the outcome of the Accuracy phase results). These changes were: -Correction of electron scattering: due to an incorrect unit setting in the multiple scattering module, the strength of multiple Coulomb scattering had been overestimated in the Accuracy phase dataset, this affected at maximum of 0.
These secondary tracks are not originating close to the origin and are largely less interesting from the point of view of physics. Those particle tracks, however, can always be found in a second pass reconstruction after the majority of primary tracks have been found. Not considering them eases somewhat the task of the algorithms and reduce their complexity. The speed is defined to be t the average time per event (in second) as measured on the test dataset, on the allocated resources in the docker environment as detailed in Sec 2. 3. The accuracy, S, and speed, t, were measured on a test dataset of 50 events (instead of 100 for the Accuracy phase), to limit the resource usage on the platform, and also because from preliminary tests the variance of the two quantities appeared to be limited.
Codalab implementation The first Accuracy phase of the TrackML competition was hosted by Kaggle, the well-known competition platform. For the Throughput phase, the algorithm had to be run within a fully controlled environment, so that the execution speed could be measured. This requirement was not possible on the Kaggle platform at the time. The Throughput phase was implemented on Codalab, a platform popular for scientific competitions, which allows for a more customised setup. Participants had to develop their algorithms on the training dataset (including the ground truth) provided. As for the Accuracy phase, a library  [10]  was available to them in order to evaluate their score. Then they had to prepare and upload the inference part of their algorithm to the platform. The platform then runs the algorithm on the public test dataset, statistically identical to the training dataset, but without the ground truth.
The execution time and accuracy score, together with the overall score obtained with Eq. 1, are measured and reported on the public leaderboard. It should be noted that in this Throughput phase, contrary to the Accuracy phase, the public test dataset is undisclosed to the participants, it is "public" only because it is used for the public leaderboard. As commonly done in such competitions, in order to avoid that participants overtrain on the public leaderboard score, the overall score was reevaluated after the end of the competition on a private test dataset, statistically identical to the public one. The overarching goal of the competition was to foster algorithms and ideas from a broader community than the high energy physics community. In the years the competition was designed then run, the workhorse for large scale processing of LHC experimental data has been single cores CPU with 2GB RAM.
If this pre-test was successful, the measurement was made on the 50 events public test dataset. Thorough tests prior to the competition determined that the time evaluation was reproducible to within 2%, which was independent on possible other evaluations running concurrently on the same physical machine. This small variance could have, in principle, impacted the ranking of the participants. Hence, it was decided, and participants were warned about it, that the final measurement of the time of a submission would be done after the end of the competition with a repetition of 10 runs on the final private test dataset. When this was done, no unexpected discrepancies were seen, even though the diversity of code tested was larger than in the tests run prior to the competition. A standard docker environment was provided with many usual libraries.
Participants could, in addition, install on-thefly libraries from internet, which access remained open to the worker node. However, in practice, participants preferred to directly ship additional libraries with their own software. Execution logs were not made available to the participants, as they could have been used to obtain insights on the undisclosed test dataset. Otherwise, there was no thorough attempt to eliminate all possible methods of hacking. Hacking was obviously forbidden in the competition rules participants had to agree to. In addition, the prizes were conditioned to a full release of the source code which was scrutinised (for the top participants) at the end of the competition.
The number of participants to the Throughput phase has been rather low, especially considering that there were 648 teams participants to the Accuracy phase on Kaggle, which could in principle have carried on to the Throughput phase. In hindsight, this has been understood to come from a combination of factors: the lower popularity of Codalab compared to Kaggle, where people can earn points across competitions. -the complexity of the problem. the perceived necessity to write optimised C++ code when a typical Kaggle participant is more familiar with python. -the threshold of less than 600 s per event and more than 50% efficiency, it was already non-trivial to have a non zero score. despite all the efforts to document and streamline the procedure to submit a solution, it still required a larger commitment than for a typical Kaggle competition.
-the fact that we did not provide the log files made debugging rather difficult to the participants. Nevertheless, the fewer number of participants was more than compensated by the high quality of the top three participants (see Fig. 3 ), who all obtained better than 90 % accuracy with an execution time up to 0. 5 s, compared to an initial goal of better than 10 s per event. After the end of the competition, all participants made their documented software available, which was scrutinized. The score was re-evaluated on the private test dataset, which confirmed the score from the online leaderboard. Hence the final rankings confirmed the online one. The original idea was that the algorithms developed in the Accuracy phase would be optimised and adapted to the second phase, not necessarily by the same participants.
Algorithmic performances In this section, a thorough investigation of the performance of the highest ranking algorithms is discussed, as was done for the Accuracy phase in  [1]. The box plot on Fig. 4  indicates the accuracy score on the 50 event test dataset. Interestingly, the accuracy follows the general ranking, indicating that little compromise was made in optimising the algorithms. Only the first two candidates have very similar accuracy and differ in regards to the speed, as can be seen in Fig. 2. Performance assessments of HEP detectors are typically several hundred pages in length, with many histograms assessing the performances from various angles. For practicality, algorithms evaluated in the TrackML challenge are ranked based on a single score, concerning accuracy and speed. In the TrackML Accuracy paper  [1] , it was demonstrated that the Accuracy score was indeed selecting algorithms which were indeed the best after a more thorough analysis.
Outlook As the Mikado tracker performs fast hit search within predefined search windows, it has tens of thousands of internal parameters to tune (e. g. size of the search windows). This situation is not typical for traditional track finders, where the search windows are estimated individually for each track using relatively involved trajectory extrapolation with uncertainties. These internal parameters have been tuned semiautomatically on the training dataset. In this respect, the presented algorithm is more similar to Machine Learning approaches. The Mikado tracker obtains a good efficiency of 94. 4% and a high speed of 0. 56 seconds per event on two CPU cores provided by the Codalab platform.
High-level description The tracking model has been designed and implemented as a standard C++11 shared library. It may be run using the main C++ driver program, or it may be loaded into the python runtime environment using ctypes. The architecture comprises a Tracker class for data housekeeping and steering, as well as a Reconstruction class to implement the algorithms. The data are organized in the Graph class that has been designed as a STL-like header file. The neural networks are handled by the XMLP class. The Trainer class inherits from Tracker: it takes care of neural network training. While the training is based on the Neural Network Objects  [15]  and the ROOT toolkit  [16]  there is no dependency of the tracking shared library to external packages. Persistence of graphs and neural networks has been achieved by streaming of the objects to corresponding text files.
The first Accuracy phase of the TrackML challenge proper was run on Kaggle  [1]. It identified a variety of 3D algorithms, and a thorough investigation has shown that the Accuracy score was indeed selecting the best algorithms when their performance was evaluated using standard metrics. The second, Throughput, phase had significantly lower participation but it yielded a few very high quality and very fast algorithms. It is not currently possible to compare directly to in-house algorithms which would need to be adapted to this specific dataset. Also, in-house algorithms usually ignore the numerous tracks with p T less than 400 MeV (the tracks with the highest curvature) while algorithms presented here are able to reconstruct tracks down to 150 MeV. So it can be estimated that in-house algorithms are not faster than 10 s per event on one CPU core, so one order of magnitude slower than Mikado from Sergey Gorbunov (a.