dataesr/scientific-paragraphs-categorization huggingface dataset:
However, such small-scale datasets comprised only trivial questions that required knowledge about common nouns, involving simple reasoning schema such as KB retrieval. Alternatively,  [26]  proposed OK-VQA, a dataset for visual reasoning with open knowledge, i. e. , the external knowledge is not restricted to a predefined (closed) Knowledge Graph (KG). Moreover, the required commonsense knowledge is not involved in the dataset building process, resulting in unbiased questions. Likewise,  [37]  proposed A-OKVQA, an open-domain KB-based VQA dataset that improves the previous one with more qualitative questions and varied commonsense knowledge that alleviate the single retrieval problem. Thus, questions require multiple steps of reasoning to be correctly answered. Different from knowledge-based VQA, KVQAE questions rely on knowledge about named entities defined in a KB rather than commonsense knowledge.
Vision-Language Pretrained Models Joint representations learned from text/image pairs are used in various multimodal downstream tasks such as VQA, image captioning, and cross-modal retrieval  [3, 43, 46]. Recently, Vision-Language Pretrained Models demonstrated significant performance gains, which led to a proliferation of architectures and pretraining objectives. Taking inspiration from the advances made in studies about attention, these models mainly rely on transformers  [45]. Inspired by BERT  [6] , they are pretrained on a large amount of aligned text/image pairs to address text+image matching and Visual-Language Mask-based modeling objectives. Specifically, LXMERT  [42]  and ViLBERT  [22]  employ two separate encoders pretrained on Masked multimodal learning and multi-modal alignment.
PROBLEM FORMULATION AND DATASET 3. 1 Task definition Our work addresses the KVQAE task originally defined in  [38]  for person entities but adopts its extension by  [18]  for many more types of entities. Hence, we consider 980 types of entities rather than one (person) only and we apply our approach to the ViQuAE dataset  [18]. The target task can be seen as a multimodal information retrieval (IR) problem where the objective is to retrieve relevant textual passages given input questions and their visual content. Specifically, text passages are part of unstructured texts from Wikipedia articles; thus each passage can be mapped to a Wikidata entity. Formally, given a text-image pair (𝑄 𝑇 , 𝑄 𝐼 ) representing a natural language question associated with visual content and a knowledge base KB = {(𝑃 𝑇 , 𝑃 𝐼 )} of text-image passage pairs, the goal is to retrieve the k most relevant passage pairs {(𝑃 𝑇 , 𝑃 𝐼 ) 1 • • • (𝑃 𝑇 , 𝑃 𝐼 ) 𝑘 } with respect to the query question pair.
Knowledge Base Reduction We identified 2. 4k unique entities in the questions and corresponding answers. Many of the answers are directly present in the entity Wikipedia article. Therefore, we matched the entities with their Wikipedia articles thanks to the key Wikidata ID. However, all the entities do not have their paired article in the KB, which means that the answers are present in other articles. Indeed, answers can be found in the entity article and/or in other linked articles (e. g. some answers about the entity James Bond can be found in the articles dedicated to the movies about James Bond). Such cases were removed, which brings the number of entities in the dataset to 2,337 (against 2,397 in ViQuAE) and 3,618 questions (against 3,697 in ViQuAE). After matching the articles, we have 2.
4k articles to build the KB. Then we add 2,663 randomly chosen non-relevant articles to add some noise while maintaining the proportion of entity type with regard to the original KB (human and non-human). After dividing the retained articles into passages, the new KB size is 170k passages, which is 1. 4% of the original size (12𝑀 passages). The computation time is reduced accordingly, as verified in Section 4. 3. We keep the same train/val/test split as in ViQuAE. To assert whether the resulting miniViQuAE benchmark is a good proxy of the larger one, we reproduce all the IR experiments of the ViQuAE paper  1. From the DPR model pretrained on TriviaQA  [13]  (filtered from the same questions as ViQuAE), we fine-tuned the model on the reduced dataset and passages and computed the embeddings of the dataset and passages with the best fine-tuned model.
We also extract features with BM25 from this textual data. For visual features, we extracted embeddings with CLIP, a Resnet-50 pretrained on ImageNet and ArcFace  [5]. We compute the retrieval score 𝑠 𝑚𝑒𝑡ℎ𝑜𝑑 for each feature, then apply a late fusion scheme  [28]  similar to  [18] : 𝑃 = 𝑟 𝑚 𝑠 𝑏𝑚25 + (1 -𝑟 𝑚 )𝑠 𝑑𝑝𝑟 +𝐹𝛼 1 𝑠 𝑎𝑟𝑐 𝑓 𝑎𝑐𝑒 + (1 -𝐹 )𝛼 2 𝑠 𝑐𝑛𝑛 + 𝛼 3 𝑠 𝑐𝑙𝑖𝑝 (1) where 𝑟 𝑚 is a binary variable to choose whether to use BM25 or DPR for the textual modality, 𝐹 takes a binary value at inference depending on face detection, and 𝑖 𝛼 𝑖 = 1. All hyperparameters are determined on the validation set. The results of the evaluation are reported in Table  2  for textonly and multimodal settings and can be compared to those of  [18]  also reported in Table  3. Although the performances are higher on miniViQuAE, which is easily understood since the dataset is smaller and less noisy, one finds the same relative order of the methods, showing the relevance of our reduced benchmark as a proxy.
This allows relating entity-level information of each passage to external knowledge resources. For this last, we leverage Wikidata5M (W5M)  [49] , a large-scale knowledge graph (KG) with nearly 5 million nodes and 20 million edges (subject-relation-object triples). It was built upon the July 2019 dump of Wikidata and Wikipedia where each entity in Wikidata is aligned to its Wikipedia page while entities with no pages or with descriptions of fewer than 5 words are discarded. We first map each linked entity 𝑒 𝑖 ∈ 𝐸 𝑖 to a node 𝑛 𝑖 in the W5M KG, resulting with the corresponding node set 𝑁 𝑖. Ideally, in order to leverage KG structural information, we would have built for each passage 𝑃 𝑖 the induced W5M subgraph that connects either directly the corresponding nodes in 𝑁 𝑖 or through all the intermediate nodes in the KG.
Late Fusion and Indexing Following  [18] , we adopt a late fusion information retrieval approach since our work focuses on enhancing text retrieval encoders with the ability to integrate external knowledge. Indeed, late fusion involves a single modality search at a time before fusing the resulting scores. We thus encode question and passage texts using the proposed knowledge-enhanced text retrieval encoder E TK and then, using a visual encoder E I , we compute a representation for all questions and passage associated images. Once texts and images for each question and passages are mapped to a dense vector representation, the most relevant passages are retrieved according to a similarity measure such that their vector representations are the closest to question ones in the embedding space. Given the multimodal representations of all questions in the dataset and all passages in the KB, we index them using the Faiss  [12]  library and perform a dense similarity search to retrieve the top-100 closest passage vector representations to the question ones using the maximum inner product.
EXPERIMENTS We fine-tuned the proposed knowledge-enhanced dual encoder on the ViQuAE dataset following the official split: train  (1, 190) , validation (1,250), and test  (1, 257). The ViQuAE KB comprises 12M passages whose maximum length equals 100 words. The associated Wikipedia article title is added to each passage header. Note that a single visual content is associated with the passages extracted from the same Wikipedia article while the ViQuAE KB comprises on average 8 passages per article. Hence, the visual content is not helpful in discriminating between passages originating from the same article.
In the text-only retrieval setting, our approach performs better than DPR on all metrics, showing that our knowledgeenhanced encoders provide additional discriminative power with richer question and passage representations. It is worth mentioning that our experiments were carried out using a batch size per GPU of 16 compared to 32 used in  [18]  due to GPU memory limitations. This suggests that some implementation optimizations could potentially lead to greater performance gains, as contrastive learning benefits from more in-batch hard negatives during training. We also observe that our approach performs best at retrieving (+1. 3pt P@1) the top-1 relevant passages, which confirms the benefit of incorporating external knowledge with contextual information. In the multimodal setting, as pointed out in  [18] , visual content provides substantial improvement for both BM25 and DPR due to the entity type bias in the dataset.
Indeed, their fusion analysis showed that image encoders greatly help to retrieve relevant passages for questions about person entities compared to non-person entity types. In particular, a subset of the ViQuAE KB images overlaps with the MS-Celeb dataset used to pretrain the ArcFace encoder, which allows a straightforward alignment between question and passage image representations. Thus, our approach naturally benefits from this bias, but also further achieves improvement over the DPR-based fusion baseline. This suggests that our knowledgeenhanced approach effectively captures additional information and enriches contextualized representations with structural information. The evaluation results on the reduced KB reported in Table  2  follow the same trend as for the full KB, confirming that the reduced KB a good proxy.
Limitations In our approach, we fine-tune the proposed knowledge-enhanced dual encoder on a relatively small dataset, which can rapidly lead to overfitting. A pre-training stage on large QA datasets and knowledge graphs would potentially help to produce richer representations. Moreover, our approach is agnostic to the type of entity a passage is related to. Some work  [7]  on comparable tasks suggests that the explicit injection of such knowledge may improve IR performance. Another limitation of our work is the random-based neighbor graph sampling, which is likely to inject noisy information into representations. Future work includes investigating more deterministic neighbor graph-building strategies beyond random sampling, for example by exploring more relevant paths in the knowledge graph that better integrate meaningful information depending on the target passage.
CONCLUSION AND PERSPECTIVES We presented an explicit knowledge integration approach for information retrieval on the KVQAE task. We proposed to leverage external resources such as knowledge graphs in order to enhance dense contextual vector representations. In this work, questions being mainly visual with poor entity-level information, we proposed to enhance their representation using syntactic information in the form of dependency parsing trees. On the other hand, entity linking is performed on passages, which allows the building of local subgraphs using an external knowledge graph. Altogether, syntactic dependencies and local knowledge subgraphs are encoded using graph convolutional networks. By combining contextual and graphbased representations, we demonstrated through experiments the benefit of such integration while improving the state-of-the-art IR on the ViQuAE dataset.
Bioinformatics is a field known for the numerous standards and formats that have been developed over the years. This plethora of formats, sometimes complementary, and often redundant, poses many challenges to bioinformatics data analysts. They constantly need to find the best tool to convert their data into the suitable format, which is often a complex, technical and time consuming task. Moreover, these small yet important tasks are often difficult to make reproducible. To overcome these difficulties, we initiated BioConvert, a collaborative project to facilitate the conversion of life science data from one format to another. BioConvert aggregates existing software within a single framework and complemented them with original code when needed. It provides a common interface to make the user experience more streamlined instead of having to learn tens of them.
To address these challenges, the open source community has made significant progress in different areas. First of all, in the software implementation and installation perspective, great efforts have been made to provide reusable and documented software. For example, the Bioconda  [7]  project offers a large selection of bioinformatics packages (over 7,000) that are pre-compiled for various platforms. Other resources, such as Python's Pypi, R's Bioconductor, and Perl's CRAN, also provide a wealth of packages for common programming languages in life sciences. Additionnaly, large efforts have been made in the inventory, classification and description of bioinformatics tools. For example, EDAM ontology  [8]  has been developed to classify and describe bioinformatics tools and data formats in a machine understandable way.
Another challenge for users is that new data formats are frequently introduced as new technologies emerge (for example, the FAST5 format from Nanopore  [12] , the consensus file from PacBio  [13] ). A community centered around a common software tool for converting life sciences data would enable the integration of new technologies and software at a faster pace. There are already many conversion tools currently available, such as ReadSeq  [14] , SBFC  [15] , Seqret, Gotree/Goalign  [16] , etc. However, they are dispersed, each of them only support a limited number of formats, they may be difficult to install, or may not be optimized for efficient use (e. g. , to be applied on huge datasets). To overcome these limitations, we initiated BioConvert, a collaborative project that aims at providing a common interface for converting life science data from one format to another.
Materials and Methods BioConvert is a software library written in Python that utilizes an object-oriented approach, where all conversions inherit from a common class. This design ensures that developers follow a consistent set of guidelines and makes it easier to extend the library over the long term. Our primary goal while designing BioConvert was to provide a single command line interface for end-users, as demonstrated in the Results section. BioConvert provides two main functionalities that are described below: i) a format conversion framework designed to convert data between formats, and ii) a benchmark framework designed to test the efficiency of all integrated tools and help selecting the most efficient tools for each conversion.
Conversion framework Regarding the conversion framework, we provide two kinds of implementations. The first ones consists of original implementations developed in Python within BioConvert, and are well adapted for simple conversions such as FastQ to FastA sequencing data format. These native implementations have the benefit of reducing the number of external dependencies. The second kind of implementations transparently integrates external tools in BioConvert's own structure and call them by interacting with the host system. This approach is well adapted to more advanced conversions involving more complex formats. In this case, it is often more efficient to use external tools, especially when they are well-established and known to be effective. For example, BioConvert uses SAMtools  [3]  or BAMtools  [17]  software to perform conversions of read alignment formats (SAM and BAM).
BioConvert was designed for easy integration of new conversions. Technically, each conversion derives from a single class that defines common methods and attributes. As shown in Listing 1, a conversion from format A to B is named A2B (line 3) and must have at least one valid method performing the conversion. However, multiple methods can be defined, typically for providing several alternative tools (see lines 7 and 11). Methods must be named with the _method prefix followed by a user-defined identifier, and they can either use external libraries or be implemented in pure Python. As shown in listing 1, it is possible to write a new conversion in just a few lines of code. In BioConvert, we have implemented a framework that automatically analyses code to detect new conversions, making it easy for developers to add their own methods.
When integrated in the BioConvert library, the code is automatically parsed to extract the necessary information about the converter: i) the name of the conversion (in this case, A2B), ii) its input and output formats (A and B), and iii) the default conversion method. This information is then made available for users of the bioconvert standalone (e. g. , in the command help), with no additional work required by the developers. In the example given in Listing 1, we present a conversion from format A to B, and show how to implement two methods called "python" and "EXE". It is worth noting that BioConvert allows the integration of an unlimited number of methods for a given conversion. That is why indicating a default method is important for typical end-users (in Listing 1, the "python" method is defined as the default on line 4).
To facilitate better comparisons, we also provide some larger input files for benchmarking purposes. Output files are not required, as the focus of this specific test is not on measuring the exactitude of the conversion, but rather on measuring the computational time required to process input files. Since input files may be large and multiple conversions may be performed, it is not practical to store these files within the BioConvert library. To easily access these files, we have created a Zenodo community https://zenodo. org/communities/bioconvert/ to publicly store benchmarking files. These files are only provided when more than one method has been implemented for a given conversion.
Benchmarks In BioConvert, most converters have only one method available. However, around 30% of the conversions have 2 or 3 methods, as shown in Fig. 3. One conversion, fastq2fasta, has 8 methods, which was used as a practical example during the initial setup of the BioConvert framework. When a conversion provides multiple methods, a default method needs to be specified. This is usually based on expert knowledge or the best-performing method. While many benchmarks have been published for specific software tools  [19, 20, 21] , local computing resources, the input file's nature, and hard disk performance can all impact benchmarking results. To address this, we have implemented a benchmark framework within BioConvert that can be run locally on any input file. This is known as single-mode benchmarking.
The only difference between top and bottom panels is related to the version of samtools used for benchmark (1. 7 and 1. 15 respectively). In the top panel, the performance of samtools and sambamba were similar, while in the bottom panel, samtools was 2-3 times faster. This difference can be attributed to the updated version of samtools resulting in a significantly increased performance. The benchmark framework in BioConvert is useful not only for comparing the performance of different conversion methods but also for optimizing the local installation of BioConvert. Developers can use the framework to verify previous benchmark results or to evaluate the impact of software updates. For instance, in Fig. 6 , the conversion from BAM to SAM format was benchmarked using a local installation of samtools version 1.
A website deployment BioConvert is a Python-based tool that can be installed on any modern platform with Python 3. It requires several external tools, which can be installed using Conda, for example. It may be difficult for some users to install all dependencies. Therefore, in addition to a local installation, a web-based instance of BioConvert is also available, hosted on the Institut Pasteur website at https://bioconvert. pasteur. cloud. The backend of the website is powered by the flask microframework to handle the upload of user's files or dynamically proposed conversion based on the input file extension. The website is scalable and managed by Kubernetes, a container orchestration platform that automates deployment, scaling, and management of containerized applications. Nonetheless, the website uses the default conversion method only, and does not include conversions that require extra arguments.
Data and Software availability The source code for BioConvert is available on GitHub under the github. com/bioconvert/bioconvert repository, with package releases posted on pypi. org website. Additionally, pre-compiled binaries are built in the Bioconda  [7]  project to ensure reproducibility, and Bioconda releases also provide biocontainers available on quay. io/repository/biocontainers/bioconvert website. The source code for the BioConvert website can be found on https://gitlab. pasteur. fr/salsa/bioconvert. The BioConvert line can easily be parallellised and executed on HPC infrastructure using Snakemake  [18]  or Nextflow  [22] , as demonstrated in the Supplementary section. Appropriate Apptainer containers, containing all third-party tools required by BioConvert, can be found on Zenodo as part of the Damona project.
The BioConvert container DOI is https://zenodo. org/record/7704649. Additionally, the Sequana  [6]  project also provide a parallelized version called sequana_bioconvert, which is also available on pypi. org, and provides a simple user interface to BioConvert. Indeed sequana_bioconvert downloads a ready-to-use Apptainer container from the Damona project mentionned above. See Supplementary for installation and example.
The device downscaling of electronic components has given rise to the need to consider specific failures in onboard airplane electronics. Single Event Effects (SEE) are a kind of failures that occur due to radiation in the atmosphere. For the purpose of ensuring onboard electronic reliability, there is a clear need for new tools to predict the SEE rate, at both avionic altitudes and at ground level. In this work, we develop a new tool: RAMSEES (Radiation Atmospheric Model for SEE Simulation), which simulates the atmospheric radiative environment induced by cosmic rays. This multiscale and multi-physics phenomenon is simulated using the Geant4 toolkit, allowing the creation of a database to characterize the radiation environment in the atmosphere as a function of altitude. We show the need to simulate very high-energy particles such as 100 TeV space protons, because they are the main contributor of radiation at avionic altitudes as well as at ground level.
Approach in Modelling the EAS Phenomenon To give the user a tool able to calculate SEE, we divided the work into four steps, presented in Figure  2. The first one is the collection of the information needed to simulate the EAS phenomenon, which are the primary particles-in our case, the protons coming from space-and the atmosphere definition. The second one is the choice of the physics used to simulate the EAS; in our case, the Monte Carlo simulation by Geant4 using the QGSP_BERT physics list. The third one is the EAS modelling itself, representing the geometry and simulating enough time to gather sufficient statistics. We simulated the EAS phenomenon in the CNES (Centre National d'Études Spatiales) Cluster during 100,000 h equivalent on one CPU using our C++ code with the Geant4 toolkit. The fourth one is the compilation of the simulation.
We combined all the calculations, i. e. , 350,000 simulations, for more than 1. 2 TB of data by using Python and the Django framework. Finally, we developed our tool, which includes a database of the EAS interaction in the atmosphere, allowing particle spectra to be estimated for any place on Earth. phenomenon in the CNES (Centre National d'Études Spatiales) Cluster during 100,000 h equivalent on one CPU using our C++ code with the Geant4 toolkit. The fourth one is the compilation of the simulation. We combined all the calculations, i. e. , 350,000 simulations, for more than 1. 2 TB of data by using Python and the Django framework. Finally, we developed our tool, which includes a database of the EAS interaction in the atmosphere, allowing particle spectra to be estimated for any place on Earth.