dataesr/scientific-paragraphs-categorization huggingface dataset:
k. a sgorbuno), 0. 5s 3https://ctdwit2017. lal. in2p3. fr on two CPU cores. On the other hand, the dataset was significantly simplified (in particular neglecting sharing of points between tracks) so that it remains to be seen whether the new algorithms can live up to expectations when used in the full ATLAS and CMS experiment context. The community is now in the process of doing this exercise. In the end, what role can be expected for Machine Learning in tracking in the light of the TrackML challenges ? It does not appear that a clustering algorithm can find the track directly (as was done with DBScan based algorithms in the Accuracy phase, which are much too slow). Nevertheless, after extended discussions between the three winners and experts in the field, a consensus appears that there are two likely avenues for the use of Machine Learning in such problems (i) combine ML with classical discrete optimisation, for example using a classifier to select early and quickly the best candidates as done by Marcel Kunze a.
k. a cloudkitchen (ii) use ML to automatise the lengthy tuning of the internal parameters of the algorithms on the training dataset (circa 10. 000 in the case of Mikado by Sergey Gorbunov a. k. a sgorbuno). Separately, the availability of the TrackML dataset has been extremely useful to facilitate the collaboration of experts which are usually working within their own experimental team. It has been used for further studies like track seeds classification with deep learning  [17] , investigating the use of cluster shape to help seeding  [18] , investigating tracking with graph networks  [19, 20, 21]  (including with FPGA  [22]  ), investigating tracking with simulated annealing on a D-Wave quantum computer  [23, 24]  or Quantum Edge Network  [25, 26, 27]. The dataset will be released in a near future on CERN open data portal, which we hope will stimulate further study.
Acknowledgements The team would like to thank  CERN  for allowing the use of the dataset, and  Codalab  for hosting the competition. We are very grateful to our generous sponsors without which the challenges would not have been possible. Platinum sponsors: Kaggle,  Nvidia  and  Université de Genève. Gold sponsors: Chalearn and DataIA. Silver sponsors :  CERN Openlab, Paris-Saclay CDS ,  INRIA ,  ERC mPP ,  ERC RECEPT ,  Common Ground ,  Université Paris Sud/Paris Saclay, IN-QNET ,  Fermilab  and pyTorch. TG acknowledges the support of the  Swiss National Science Foundation  under the grant  200020_181984. SG acknowledges the support of the  German BMBF ministry. This project has received funding from the  European Union   Horizon 2020 research and innovation programme under grant agreement No 724777 "RE-CEPT", No 772369 "mPP" and No 654168 "AIDA-2020 ".
This work was made possible by  Institut Pascal at Université Paris-Saclay  with the support of the program " Investissements d'avenir "  ANR-11-IDEX-0003-01. In addition, the organisers would like to thank the members of the  International Advisory Committee  :  Markus Elsing  (CERN),  Frank Gaede (DESY), Alison Lowndes (Nvidia) ,  Maurizio Pierini  (CERN),  Danilo Rezende  ( Google DeepMind ),  Marc Schoenauer  ( INRIA-Saclay ) and  Svyatoslav Voloshynovskyy (U Genève ).
, 2007). However, the geometrical features of the identified planes do not adequately fit the ones observed in 2D because the fracture planes are not completely reconstructed  (Bossennec et al. , 2021). This paper presents a new software tool technology (RECO3D) derived from the Ransac algorithm that reconstructs the fracture planes on a 3D point cloud acquired on outcrops. After describing the workflow from data acquisition to processing, results from five outcrops with different outcropping qualities are presented to show the efficiency of the automated fracture recognition. The proposed workflow integrates both LiDAR 3D point cloud analysis and classical structural scan lines carried out in the field to validate the numerical plane reconstruction. After giving some guidelines on the precautions required for this method, this study aims to validate the efficiency of the RECO3D software tool and prove its capacity to quantitatively characterise fracture network properties from 3D point clouds of rock outcrops.
Materials and Methods The RECO3D tool is tested on five basement rocks outcrops scanned with a terrestrial LiDAR. These were selected to represent typical lithologies used as analogous to deep geothermal drilling targets in the West European Rift System (WERS). They all belong to Hercynian basement rocks outcropping at the shoulders of the Limagne Graben in France and in the Odenwald at the shoulders of the Upper Rhine Graben in Germany. The next section briefly resumes the general geological history and data acquisition process on each outcrop, and thereafter the workflow of the LiDAR data processing.
For both of these outcrops the LiDAR acquisition was carried out with a Laser Scanner Focus 3D X330 from Faro© apparatus and the post-acquisition processing was done with the classic workflow of SCENE© software (FARO® SCENE Software, 2021). The acquisition in Enval is composed of three LiDAR scans at 7 m from the outcrop and a 5 m distance between each. For Combronde, three large-view scans were acquired at approximately 12 m horizontal distance from the outcrop and four high-resolution scans at 4. 5 m, with a 5 m spacing between the scans. The mean resolution obtained for the final clouds is estimated at 5 mm. The localization and geologic context of the Odenwald crystalline outcrops are extensively described in  Bossennec et al. (2022). Three outcrops are integrated in this study, with varying lithology and structural features (Figure  1c ).
P10, P21 and P32 computation Fracture properties that are automatically computed are: orientation (strike, dip and dip azimuth), length with Lmax = the maximal length of the fracture plane and Lmin the minimal one, area and X, Y, Z coordinates of the fracture centre. For comparison between classical 1D scan lines data or 2D fracture maps, fracture intensity is also computed on the filtered results. The fracture P10 (fracture intensity = number of fracture/length unit) is obtained by drawing a virtual scan line on the point cloud and extracting of the fracture planes that crosscut the line. To estimate P21 (intensity = length of fracture/area unit of the outcrop surface), a very smoothed surface fit is computed from the data point cloud. The result is a flat surface decomposed in cells of the variable surface.
Quality check of the automatised recognition To quantitatively test the quality of the fracture reconstruction, the orientation of 30 fractures planes randomly picked on each outcrop is compared between Test C results and the fracture orientation picking tool of CloudCompare® software (Compass 2. 0) (Figure  7 ). This tool allows to pick points from the cloud in a given circular area and estimates the orientation of a plane fitting all points using least squares regression  (Thiele et al. , 2017). The comparison shows that most planes are in a range of -3 to +3 degree in dip azimuth and dip. The few planes with a more significant difference are small planes where few points could be picked with the circular tool, or with a small point density linked to the LiDAR acquisition process, or at the contrary large planes with a certain curvature.
For the latter, the orientation given by the pinking tool clearly depends on the picked position while the RECO3D tool calculates the mean orientation of the whole fracture. The observed differences are thus globally consistent given the different calculation methodologies between the two tools. Moreover, for a natural fracture, the variation in dip and dip azimuth linked to fracture roughness and curvature is also commonly in a 10°range when measured in the field, with a compass for instance. Thus, the fracture orientation calculated after the fracture reconstruction with our RECO3D fits well with data acquired in the field.
Tromm The first step of filtering on the Tromm digital outcrop includes the planes reconstructed with lengths between 0. 06 and 8 m, which are larger than Combronde and Enval outcrops, with a total of 3 197 planes (Table  2 ). Indeed, the Tromm outcrop exposes mostly large fracture planes after visual inspection. Such results are thus expected (Appendix C). Orientation distributions show the main fracture set N150°E, with a cluster of dip around 45°to the SW, and two secondary fracture sets around N070°E and N120°E (Figure  10 ). Filter II decreases the validated fracture planes to 2 059. The cleaned areas are mostly on the horizontal surfaces of the outcrop, which are eroded and poorly constrained by the terrestrial LiDAR data (Appendix C). On this filter, the main fracture sets remain identical to filter I, as the length distribution with a power law with an exponent around -1.
8 m, the sampled number is quite similar (167 on the field and 179 for RECO3D). As observed on the whole outcrop with the RECO3D detection (Figure  7 ), the fracture network is scattered even for the field data with at least 6 fracture sets (N030°E, N045°E, N075°E, N120°E, N150°E and N170°E) (Figure  13 ) Even if the proportion of each set varies between the outcrop and the virtual SL, both dataset show the same kind of result. The fractures extracted from the virtual SL for Combronde are far fewer than in the field, respectively 69 vs 256 (Figure  13 ). As for Enval, more longer fractures are sampled in the field than on the virtual SL. On both, the main fracture set is N000°E oriented as on the whole outcrop analysed by RECO3D (Figure  9 ). There is also a minor fracture set around N045°E direction, but the N100°E set on the outcrop is not represented on the virtual SL.
Thus, if scan line data will show an exponent a ≈ 1, fracture maps on the same geological object should show a length distribution with an exponent a ≈ 2, and the fracture network modelled in 3D and size distribution with a ≈ 3. Concerning the five outcrops studied, the fracture network of Combronde shows a length distribution with exponent of 2. 1-2. 2 depending on the filters, Tromm between 1. 7 and 1. 9, MULO between 1. 9 and 2. 0 and SC2 of 1. 9. The value of Enval is partly due to the two-step power law. Interestingly, the length of the planes reconstructed by the RECO3D tool shows distribution similar to the classic fracture traces maps, with an exponent of around 2. It means that the length distribution from the RECO3D tool may be used in the same way as the classic 2D methodology and extrapolate in the same way for input in numerical fracture network models such as DFNs.
The P32 depending on the filter varies from 1. 5 -1. 9 m -1 for Tromm to 10. 6 -13. 5 m -1 for Combronde. The comparison to published data is not simple because intensity values are linked to the lowest fracture length set, which is different for each study. Also, as mentioned before, most published P32 values are derived from P10 or P21 conversion and not direct data. For example, Miernik (2020) computed P32 from P10 on an outcrop in sandstones with values between 1. 7 and 2. 5 m -1 in damaged zones and 4. 4 to 10. 3 m -1 in fault cores;  Bonnet et al. (2001)  computed P32 in carbonates between 1 and 3. 5 m -1 with the same methodology on P21, as  Seers and Hodgetts (2016)  on shear bands show P32 between 3. 76 and 35. 56 m -1. At least, for these studies of fracture networks at almost equivalent scales, it shows that the computed fractures areas and volume by the RECO3D tool fits with the ranges obtained by P10 and P21 to P32 conversion.
Acknowledgements. We would like to thank the  HLNUG  who provided digital elevation data,  Sebastian Schröder  and the municipalities of  Wald-Michelbach  and  Rimbach  for giving us access to the quarries in the Tromm Granite. We also would like to thank the  Mitteldeutsche Hartstein-Industrie AG  for granting access to the  Mainzer Berg and Mühltal quarry. RÖHRIGgranit GmbH  is thanked for granting us access to their quarries in Heppenheim. We also would like to thank  BPI France  for their support of the tool development.
Furthermore, thanks to its neural network, SpliceAI is able to make predictions about the global splicing outcome (e. g. , exon skipping, splicing rescue by cryptic site activation, pseudo-exon creation, etc. ). This ability to focus not only on the nearby site (destruction or creation) but at the whole transcript level is a unique feature of these deep-learning-based next-generation splicing predictors, such as SpliceAI or Pangolin  [11]. In a recent improvement, the SpliceAI neural network has been retrained with a curated and manually validated isoforms dataset  [12]. Still, the standard version of SpliceAI (currently v1. 3. 1) has some limitations. First, predictions and relative positions of the altered splice sites are displayed as numerical values, which can be confusing when estimating which exact sites are altered, or when dealing with long-distance effects.
Methods In this study, we refer to "raw" scores (RS) for the absolute prediction of SpliceAI, in opposition to the "delta" scores (DS). We wish to dismiss any confusion concerning the "raw" scores found in the SpliceAI terminology, referring there to "raw" delta scores, in opposition to "masked" delta scores (see https:// github. com/ Illum ina/ Splic eAI for more details).
SpliceAI-visual For SpliceAI-visual, the SpliceAI model (https:// github. com/ Illum ina/ Splic eAI, custom sequence function) is run independently on two sequences (reference allele; variant allele), generating for each nucleotide its likelihood (RS) to be used as an acceptor or a donor site in a biological context. Results are then used to generate 2 bedGraph files (http:// genome. ucsc. edu/ golde nPath/ help/ bedgr aph. html). In the Colab notebook, scores are computed for both the entire reference and variant transcripts in real time. The reference and variant bedGraph files can be loaded in a genome browser. To integrate SpliceAI-visual in MobiDetails, we have used SpliceAI v1. 3. 1 to pre-compute the RS for 57,271 transcripts including 19,120 Matched Annotation by NCBI and EMBL-EBI (MANE) transcripts available on the Web site  [16] , using Illumina ® models available for non-commercial usage (see https:// github.
com/ Illum ina/ Splic eAI for more details). Then, RS predictions for the wild-type sequences for these full transcripts are stored as bedGraph files and are directly available for comparison with the variant RS predictions. RS predictions for the variant have to be computed in real time (the software architecture is described in Additional file 1: Fig. S1 ). The variant allele consists of 10 kb of genic sequence surrounding the variation (truncated if the variation is located less than 5 kb from the 3' or 5' end of the transcript). Indeed, the authors of SpliceAI have shown that their algorithm was the most accurate using 5 kb of DNA sequence surrounding the variant position  [2]. We added an additional 5 kb on each side of the variant to display a larger picture of the splicing pattern of the region.
A dedicated Flask API (https:// palle tspro jects. com/p/ flask/) available in a private server (source code available at https:// github. com/ mobid ic/ splic eai) is asynchronously called by the public MobiDetails server to compute the variant allele RS (in about 30 s) (see Additional file 1: Fig. S1 ). Computation requests on the private server are handled by the Apache Web server (https:// apache. org/) and queued with the SLURM workload manager (https:// slurm. sched md. com/). SpliceAI is run in CPU-only mode. The API returns JSON objects including the DNA sequence and the associated SpliceAI RS, which are converted into BedGraphs by the MobiDetails public server. BedGraphs are then displayed on the Web page within an igv. js genome browser (https:// github. com/ igvte am/ igv. js/) as two separate tracks (reference and variant BedGraphs).
Examples from the literature Identifying pseudo-exon inclusion SCN1A The deep intronic substitution NM_001165963. 4(SCN1A):c. 4002 + 2461 T > C (Table  2 , Fig. 1 ) has been demonstrated by minigene assays to induce the exonization of an out-of-frame 64-bp intronic sequence  [17]. This 64-bp exonization mechanism has not been elucidated, but was correctly identified by SpliceAI with low DS (AG: 0. 18; DG: 0. 15). Using SpliceAI-visual, we show that while the DS are below the recommended threshold, the RS for the wild-type sequence are already significant (acceptor site: 0. 64; donor site: 0. 73). This results in high RS for the variant sequence T > C (acceptor site: 0. 82; donor site: 0. 87) and finally in the inclusion of the intronic sequence in the transcript. The mRNA proportion aberrant/normal transcript was not estimated.
Solo-exome sequencing showed a hemizygous substitution in a canonical donor site of the gene CASK, NM_003688. 3(CASK):c. 172 + 1G > A, absent from control databases (gnomAD, deCAF)  [24, 25]. No other pathogenic or likely pathogenic variant was retained. This donor site disruption affects the MANE transcript of CASK. This hemizygous variant of patient 3 is predicted by SpliceAI to result in a DL, along with a + 0. 71 DG. With SpliceAI-visual, this DG was predicted to lead to in-frame retention of 18 bp (6 amino acids, no stop codon, Fig. 5 ). Furthermore, this donor's DS of + 0. 71 adds to a probability of 0. 28 on the reference allele, resulting in an RS of 0. 99 on this donor site (Fig. 5 ). In accordance with SpliceAI-visual predictions, RT-PCR on peripheral blood of patient 3 identified the 18-bp retention on 100% of transcripts (Fig.
20 used for DS has been qualified as "relatively permissive" and as a "high recall" threshold by the original authors of SpliceAI (https:// github. com/ Illum ina/ Splic eAI). 2 However, the three deep intronic pathogenic or likely pathogenic splicing variants of SCN1A, MFGE8, and SETD5 would have been filtered out with this threshold. SpliceAI-visual represents a convenient manner to predict the splicing outcomes of these variants. Interestingly, the authors of SpliceAI observed a decreased sensitivity of SpliceAI to predict the splice alterations of deep intronic variants, compared to variants located near exons. This was also recently reported for Pangolin  [11]. They hypothesized this phenomenon to be caused by a putative intronic deprivation of specific markers, which are usually enriched near exons by selection.
Monoallelic alterations of the SETD5 gene are implicated in intellectual disability, combining delayed psychomotor development and poor language development (OMIM #615761). The duplication of a natural splice site in SETD5 identified in patient 5 in the heterozygous state, absent from the gnomAD database, and annotated as frameshift would have been consistent with the previous descriptions, where the intellectual disability is often mild. This variant was inherited from the asymptomatic mother, but this has been previously described for other pathogenic SETD5 variants  [39]. Thanks to SpliceAI-visual, the benign splicing outcome of this presumed frameshift duplication could be suspected and was further confirmed by RNAseq. The variant was then assumed to be probably benign. SpliceAI-visual has also been useful to guide functional exploration in the GRN case, as it enabled the correct design of RT-PCR primers specific to the intronic retention.
As previously described, the amount of transcript has been found to be similar in the presence or in the absence of nonsense-mediated decay (NMD) inhibitor, suggesting a limited NMD effect  [19]. Interestingly, the retention included two AUG codons with moderate potential to initiate translation, as their Kozak consensus sequence strength was similar to that of the natural AUG of GRN. As small upstream open reading frames (uORF) can reduce the translation efficiency of a transcript, we hypothesize that these uORFs caused a nearly complete extinction of translation in the transcript including the retention  [40]. SpliceAI-visual is also useful to assess the splicing outcomes of complex variants such as deletions/ insertions, as, apart from running a private instance of SpliceAI, this is currently the only tool that computes such SpliceAI predictions.
Availability of data and materials SpliceAI-visual is freely available on MobiDetails at https:// mobid etails. iurc. montp. inserm. fr/ MD/, or in Google Colaboratory at https:// tinyu rl. com/ splic eai-visual. It can be freely copied for local usage, or used online in Google Colaboratory with the requirement of a Google account. All variants described in this manuscript are available in MobiDetails (https:// mobid etails. iurc. montp. inserm. fr/ MD/ auth/ varia nt_ list/ splic eAI_ visual_ 2022 or https:// tinyu rl. com/ bpyz9 x6j). Variants included in the Additional file 1: Table  S1  are also available in MobiDetails (https:// mobid etails. iurc. montp. inserm. fr/ MD/ auth/ varia nt_ list/ splic eAI_ visual_ compl ex_ 2022or https:// tinyu rl. com/ 49nuj ud4). MobiDetails code is available at https:// github.
In its early definition, VQA aim to answer simple questions given an input image, which allows assessing the reasoning abilities of models on visual and language understanding. More recent work  [26, 37, 47, 48]  proposed a Knowledge-based VQA task formulation that requires reasoning abilities that go beyond counting objects and color recognition. In contrast, Knowledge-based VQA datasets include questions that require external commonsense knowledge to be correctly answered, as the image content does embed only partial information. To further expand the scope of possible improvements, recent work  [18, 38]  pointed out the limitations of the knowledge-based VQA datasets, which restrict the reasoning to commonsense knowledge, arguing with the necessity to answer questions that require knowledge about named entities.
Following  [38] , who had previously introduced questions focusing on named entities with text and image,  [18]  more specifically proposed the Knowledge-based Visual Question Answering about named Entities (KVQAE) task, a VQA problem formulation where answering questions requires knowledge about named entities defined within a knowledge base (KB). They proposed the ViQuAE dataset, which covers hundreds of entity types whereas the KVQA dataset  [38]  is limited to person entity types. In this context, we classically tackle the KVQAE task as a twostep process: a first Information Retrieval (IR) step followed by a Reading Comprehension (RC) step. As illustrated by Figure  1 , the IR step starts from a (question, image) pair and aims to retrieve a restricted set of (passage, image) pairs from the reference KB.
RELATED WORK Visual Question Answering VQA focuses on answering questions conditioned on visual input. It has been a longstanding benchmark for vision and language integration/reasoning. Early research effort  [3, 24]  has been spent on developing many datasets that include questions about object names, colors, and attributes. With the rapid improvement of image understanding techniques, answering such questions boils down to a visual recognition task since images contain all the necessary knowledge to answer questions. Similar to Knowledge-based Textual Question Answering  [23, 40, [54] [55] [56] , Knowledge-based VQA received attention  [29, 30, 47, 48, 51]  to further explore commonsense/visual reasoning abilities. Several datasets have been developed, such as FVQA  [47]  and KB-VQA  [48] , where external knowledge is leveraged to answer questions whose image does not carry all the required knowledge.