dataesr/scientific-paragraphs-categorization huggingface dataset:
Although preliminary research data suggest that depressed participants engage more slowly with positive stimuli than non-depressed participants  (Sanchez et al. , 2017) , the nature of the attentional components involved in bias related to positive cues remains poorly understood. Then, we computed CV scores to measure attentional bias to positive cues to include both attentional components. With longer stimulus-onset asynchronies, as were used in this study, a positive cue validity effect suggests that attention is maintained on the cue. The split-half reliability indices were computed separately for each task via Spearman-Brown correlations with the first and second half of trials in each experimental condition. Spearman-Brown correlations ranged from 0. 43 (Happy valid) to 0. 48 (Happy invalid).
Statistical Analysis The Shapiro-Wilk test suggested that all variables were non-normally distributed (all p s < 0. 002). First, we computed non-parametric Spearman correlations between all variables to describe our group characteristics using JASP Version 0. 13. 1 (JASP Team, 2020). Thereafter, path analysis with a maximum-likelihood estimation method was computed with the Lavaan package in R, version 0. 6-8  (Rosseel, 2012). According to Rosseel, the parameters estimated by this method are consistent with non-normal data. 3 The goodness of fit is indicated by a non-significant χ 2. If the chi-square is significant, a χ 2 /degrees of freedom ratio of less than 2 indicates a good fit, while a result of less than 3 is acceptable  (Cangur and Ercan, 2015). We also computed several other fit statistics, including the Root Mean Square Error of Approximation (RMSEA), the Standardized Root Mean square Residual (SRMR), the Tucker-Lewis Index (TLI), and the Comparative Fit Index (CFI;  Bentler, 1990; Hu and Bentler, 1999; Cangur and Ercan, 2015).
42 participants fell into this category, and the following analysis was conducted on their responses. The data was subsetted to items in DEstrong-UEstrong and the better rated condition between DEstrong-UEweak and DEweak-UEstrong (as determined for each participant separately). A linear mixed model was fitted on this data set with condition (DEstrong-UEstrong vs. Other) as a fixed effect and random by-participant intercepts and slopes. A comparison of this model with a reduced model without condition as a fixed effect revealed no significant effect of condition (χ(1) = 0. 01, p = 1). For reference, the mean rating of the better rated conditions between DEstrong-UEweak and DEweak-UEstrong (as determined for each participant separately) was 88. 6 (SE = 2. 8), while their mean rating in DEstrong-UEstrong was 88.
Acknowledgements We wish to thank  Emmanuel Chemla  for many important discussions and suggestions. We also thank  Lucas Champollion ,  Keny Chatain ,  Gennaro Chierchia ,  Floris Roelofson ,  Benjamin Spector  and the audiences of LINGUAE seminar at ENS, XPRAG 2019 and Amsterdam Colloquium 2019 for helpful feedback. The research leading to these results has received funding from the  European Research Council  under the  European Union 's  Seventh Framework Programme  ( FP/2007-2013 ) / ERC Grant Agreement n. 313610  and n. STG 716230 CoSaQ , and was supported by  ANR-17-EURE-0017.
Introduction The recent years have been characterized by a widening diffusion of artificial neural networks (ANNs) for solving varied tasks in many different domains. With this growing popularity, and in view of neural architectures becoming more sophisticated, the research area known as explainable artificial intelligence (XAI) has gained relevance, especially when attempting to define approaches for investigating how machine learning (ML) models make their predictions. In this work, we move in this direction by studying the internal behavior of a deep neural source code analyzer (namely, the CuBERT transformer  [1] ) trained in the detection of software vulnerabilities. Our goals are conceived along two lines: that of XAI, in which we investigate if (and, possibly, where) human-understandable concepts emerge in the internal activations of a complex neural network, and that of cybersecurity, in which we probe the classification of the network on the basis of emerging concepts related to the vulnerabilities over which the network is trained on so as to identify possible misconceptions or blind spots in what has been learned.
ML for Static Analysis In recent years, besides the traditional techniques and tools for static analysis  [4] , also approaches based on machine learning  [5] , and especially on deep learning  [6] , are becoming popular to deal with source code. Many of these techniques are derived from NLP approaches. In particular, architectures based on the transformer model  [7] , besides the native applicability in NLP, have proven to be effective also in the context of source code analysis: for instance, we can mention PLBART  [8] , a sequence-to-sequence model used for tasks such as code generation and code summarization, and CuBERT  [1] , a derivation of the popular BERT model  [9]  adjusted for processing source code instead of natural language. Among the reasons for the popularity of these kinds of neural architectures, we can mention the fact that data need only a simple preprocessing since the source code is tokenized and given as input as a vector of integers, where each integer represents a token and, most importantly, the training is performed in two steps: a first unsupervised (and computationally heavy) step in which the transformer is trained on large corpora of data, and a second supervised step, to specialize the model in a given task.
Proposed Approach The proposed approach is devised for studying the decision process of a neural classifier. Our experimental workflow, which is outlined in Figure  1 , basically consists in treating the input instances of a neural network as points of the geometric space defined in the internal layers of the network, as detailed in Section 3. 2. Together, within the input dataset, we select samples of instances that represent a given concept. In general, whatever is expressible in natural language can be considered as a concept. Here, in the specific domain of source code processing, and being that the considered dataset is composed by Java methods, we will consider both lexical concepts (e. g. , the presence of targeted patterns or constructs), and syntactical ones, such as the complexity or the relation between the types of arguments and the returned objects.
The CuBERT Transformer In our experiments, the model we chose to analyze was the CuBERT  [1]  transformer, which is derived from the popular BERT  [9]  transformer for NLP, designed to effectively deal with source code. Transformers are becoming increasingly popular and powerful, for more and more applications different from the original NLP tasks they have been designed for (e. g. , image  [28]  and source code  [8]  processing), and when using such a kind of deep architecture, the benefit is that they can be trained once on large corpora of data associated to generic and self-supervised tasks, and later they can be fine-tuned on more specialized supervised tasks. This allows to use them with high flexibility. In this work, we appointed the cybersecurity domain and fine-tuning to be on the detection of software vulnerabilities by using a CuBERT model pre-trained on a Java corpus (Available: https://github.
com/google-research/google-research/tree/master/cubert, version of 11 July 2021). To this end, we used the Juliet Test Suite v1. 3 for Java  [29, 30] , a dataset developed by the NSA that contains a corpus of programs labeled according to the common weakness enumeration (CWE) (https://cwe. mitre. org/index. html, accessed on 22 September 2022) list. For each CWE, the dataset consists of a set of Java files defining both flawed methods and, correspondingly, a number of non-flawed constructs. To build our training sets, we preprocessed the files labeled CWE-369 and CWE-789 representing the "divide by zero" and "memory allocation with excessive size value" vulnerabilities, respectively. The set of positive instances was populated with the methods that self-contain the flaw; in fact, we removed from the dataset all the methods that, to exhibit the vulnerability, need to call other methods or to use support classes.
I/O relationship This sub-concept considers the semantic of a method in terms of the relation between what is passed as argument (i. e. , the input) and the returned object (i. e. , the output). We only consider a subset of all the possible I/O relations, namely, the presence or the absence of an array among the input arguments and whether the returned object is an array or a single element. In particular, in our experiments, we consider as a sub-concept the many-to-many relation, namely the methods that contain (at least) an array among their arguments, and that return an array. Random This concept is defined by simply assigning random labels to the methods in the dataset. The obtained partition is obviously meaningless, and the experiments on this concept are used as a baseline to assess the validity of the other results.
SVCs and Activations Spaces As outlined in Figure  4 , we considered the feedforward layers of the encoder blocks, namely the dense layers that feed the succeeding encoder block. In the CuBERT transformer, there are 24 encoding blocks, each one composed by 1024 neurons with a 512-dimensional output. For each considered sub-concept, we started by collecting the activations obtained by flattening the output of the feedforward layers (see Section 3. 2) for a balanced random sample, where 100 program instances were associated to the concept, and 100 did not represent it. Then, for both the fine-tuned models (i. e. , the binary classifiers trained in the detection of the divide by zero and uncontrolled memory allocation vulnerabilities, respectively), and for each concept, we gathered the activations of the flattened layers as points of a 512 * 1024-dimensional vector space, and for each of the 24 layers, we fit a linear support vector classifier (SVC) to correctly separate the points representing programs related to the chosen concept, and the points representing programs with no presence of the same concept.
Each evolutionary run is defined by the following: • The considered concept, among the five we described in Section 4. 2; • The considered CWE, between CWE-369 and CWE-789; • Whether we define a grammar which forces the corresponding vulnerability or not; • Whether we maximize or minimize the fitness value. For a set of combinations of these choices, DSGE is run for 12 generations, with a population of 200 individuals and the elitism of 4 individuals, with probability equal to 0. 1 for mutation and 0. 9 for crossover. In this way, we search for Java methods which maximize or minimize the presence of a concept in terms of the signal inside the network. These instances usually do not belong to the datasets used during training, and we will later check how the networks classify them, taking into account the possible presence of a vulnerability forced by our grammar.
Measuring Sensitivity to Sub-Concepts We finally studied how much the prediction of the model is influenced by the concepts by performing two analyses. The first one, which is derived from the TCAV approach  [2] , basically measures the percentage of instances that represent a concept among a set of programs that are positive with respect to a classification task. Formally, for each concept c, we considered a sample X v of 100 methods from the dataset that are both labeled and predicted to be vulnerable by our model. Notice that we considered the vulnerable class, but the same considerations can be made on the other class of safe programs. Then, we measured the sensitivity S c,v by computing, ∀x ∈ X v , the distance d c,l ( f * l (x)), where l is the layer referred to the best support vector V c,l trained on the concept c, and by computing the ratio between the number of times that d c,l ( f * l (x)) has a positive value, and the cardinality of sample X v : S c,v = |{x ∈ X v : d c,l ( f * l (x)) > 0}| |X v | (1) A second analysis was performed on the instances generated via evolutionary search.
Discussion The previous sections introduced the goals, the approach, and the raw outcomes of the experiments. Quantitative results allow analysis of the behavior of the neural classifiers. We can discuss the outcomes of evolutionary exploration of classifiers' behavior, whose outcomes are presented in Table  2. We aim at discovering mistakes of the neural networks, and possible reasons for them, after the training ended with high accuracies over the training/test datasets. According to our approach, the experiments first required that we identified understandable concepts that could affect the classification, and then those concepts allowed to train and select a corresponding SVC on a whole neural layer. Finally, the best SVC provided a fitness function to control the evolutionary search of input instances, either strongly or weakly related to the concept.
Data Availability Statement: The pre-trained CuBERT model is publicly available in the Google Research GitHub repository: https://github. com/google-research/google-research/tree/master/ cubert, access on 22 September 2022. Juliet Test Suite for Java used for fine-tuning CuBERT and for the sub-concepts investigations is developed by the NSA Center for Assured Software and can be downloaded from https://samate. nist. gov/SARD/test-suites/111, access on 22 September 2022. The DSGE library used for the evolutionary experiments is available on GitHub: https: //github. com/nunolourenco/sge3, access on 22 September 2022.
In the field of medical imaging, the division of an image into meaningful structures using image segmentation is an essential step for pre-processing analysis. Many studies have been carried out to solve the general problem of the evaluation of image segmentation results. One of the main focuses in the computer vision field is based on artificial intelligence algorithms for segmentation and classification, including machine learning and deep learning approaches. The main drawback of supervised segmentation approaches is that a large dataset of ground truth validated by medical experts is required. In this sense, many research groups have developed their segmentation approaches according to their specific needs. However, a generalised application aimed at visualizing, assessing and comparing the results of different methods facilitating the generation of a ground-truth repository is not found in recent literature.
According to different image modality, the main objective is to develop applications for different aspects, such as edge detection, location of organs, segmentation of anatomical regions, or lesion detection. The challenge is how to extract global features of the image to distinguish ROI from the background. The main limitation is that there is no algorithm that is suitable for all the cases. Moreover, it could be a tedious and inefficient procedure applied for large image database. In this sense, there have been various approaches proposed to the task of segmenting ROIs in medical images using traditional segmentation algorithms. A comprehensive review of the top methods in 10 3D medical image segmentation challenges during 2020 was studied, covering a variety of tasks and datasets  [4]. Challenges provide an open and fair platform for various research groups to test and validate their segmentation methods on common datasets acquired from the clinical environment.
The advent of DL may open a new paradigm to do something with traditional techniques to overcome the many challenges DL brings (e. g. , computing power, time, accuracy, characteristics, and quantity of inputs, among others). Certainly, they need a lot of domain expertise, human intervention only capable of what they are designed for, and comparison with the traditional segmentation algorithms, such as thresholding, edge detection, clustering, region growing, or active contour models, provide more promising solutions  [3]. However, in both cases, the solving of all CV problems and the usability of traditional algorithms still have a relevant importance. In addition, the development of supervised AI algorithms requires a large amount of ground truth databases validated by radiologists and medical experts to ensure an efficient solution.
For this reason, a new application (MedicalSeg) based on different segmentation approaches is presented in this work using directly Digital Imaging and Communications in Medicine (DICOM) images  [19] , and also standard image formats (jpeg, png, etc. ). Moreover, MedicalSeg includes a set of pre-processing tools and segmentation algorithms that can be easily and quickly applied to different types of medical images. Furthermore, an automatic system saving all the results in a defined structure is also implemented to facilitate the users and generate a dataset for post-processing purposes. Finally, facilities to execute this application using different modalities are available. In this paper, a related works, methodology, a case of use of this application and a discussion section with conclusions are presented.
Materials and Digital Repository In this section, materials and experimental data used for demonstration of the usability of MedicalSeg tool is described (Section 2. 1). Five medical databases (Figure  1 ) have been used: full-digital mammography database (Section 2. 1. 1); MRI prostate database (Section 2. 1. 3); Magnetic Resonance Angiography (MRA) vascular brain database (Section 2. 1. 2; ultrasound database (Section 2. 1. 5); and digital retinal database (Section 2. 1. 4). On the other hand, the design of the GUI interface (Section 3. 1), the segmentation algorithms implemented in Medi-calSeg (Section 3. 2) and the image processing tools are described in Section 3. 3.
Full-Digital Mammography Database The Mammographic Image Analysis Society Digital Mammogram Database (MIAS) used in previous works  [20]  is composed by a set of 322 Mediolateral Oblique (MLO) digitized mammograms corresponding to left and right breast of 161 women (Figure  1a ). The films were extracted from the UK National Breast Screening Programme, and digitised to 50 micron pixel edge with Joyce-Loebl scanner, a device with linear response in the optical density range 0-3. 2. Each pixel was described as a 8-bit word.
Magnetic Resonance Angiography (MRA) Vascular Brain Database Another database used to test this application is carried out with the Magnetic resonance angiography (MRA) sequences of 18 paediatric patients obtained at the diagnostic imaging service of the Sant Joan de Déu Hospital (HSJD, Barcelona)  [21]. A procedure called Triggered Angiography Non-Contrast Enhanced (TRANCE) is a cardiac triggered 3D TSE technique that dynamically subtracts images from different phases in the cardiac cycle to obtain high resolution arterial images with bright vessels and dark background. This database is provided by the FIS Acción Estratégica en Salud (AES) 2020 (project PI20/00296) funded by Instituto de Salud Carlos III of Spain. The images will be analysed by HSJD radiologist Dr. Josep Munuera providing clinical insight.
MRI Prostate Database A database with prostate MRI based on clinical data with tumour and healthy cases was used  [22]. Then, among the tested patients not all the prostate studies present tumours. The examinations used in our study contained three-dimensional T2-weighted fast spinecho (TR/TE/ETL: 3600 ms/143 ms/109, slice thickness: 1. 25 mm) images acquired with sub-millimetric pixel resolution in an oblique axial plane. All the dataset and ground truth data are provided from the medical imaging department of the University Hospital of Dijon (France)  [23]. The institutional committee on human research approved the study, with a waiver for the requirement for written consent, because MRI was included in the workup procedure for all patients referred for brachytherapy or radiotherapy.
Digital Retinal Database The photographs for the DRIVE database were obtained from a diabetic retinopathy screening program in The Netherlands  [24]. The screening population consisted of 400 diabetic subjects between 25 and 90 years of age. The DRIVE database has been established to enable comparative studies on segmentation of blood vessels in retinal images (Figure  1d ). The purpose is to use this database for retinal vessel segmentation and delineation of morphological attributes of retinal blood vessels, such as length, width, tortuosity, branching patterns, and angles which are utilized for the diagnosis, screening, treatment, and evaluation of various cardiovascular and ophthalmologic diseases, such as diabetes, hypertension, arteriosclerosis, and chorodial neovascularization.
Ultrasound Database For the completion of this work, a 2D fetal ultrasound (Sonogram) database is used. Sonogram is an imaging technique that uses sound waves to produce images of a fetus in the uterus. Fetal ultrasound images are used during pregnancy to check the baby's development, the presence of a multiple pregnancy and to help pick up any abnormalities. In some cases, fetal ultrasound is used to evaluate possible problems or help confirm a diagnosis. The different images of each baby consisted of ultrasounds performed at different planes, both coronal and sagital, and for several weeks (Figure  1e ). The size of each 2D ultrasound image is 800 by 540 pixels with a pixel size ranging from 0. 052 to 0. 326 mm obtained from Zenodo  [25].
4. Respect and to be responsible by giving credit and acknowledging of this work. As a code of ethics, users must be appreciate our effort and reference this work on their future scientific purposes. Licensing clarifies the rights of author according to the Berkeley Software Distribution (BSD) available on the file exchange licensing. Three different redistribution options are available for users. All of them can download in a GitHub repository  [26]. Firstly, it is available to download the full MedicalSeg source for Matlab developers that want to use the code according to their necessities. Secondly, an APP package was created to be installed in the main App panel of Matlab workstation. It facilitates the use of this application creating a shortcut icon of this App. Furthermore, a standalone installer file has been created, if users want to run MedicalSeg on target machines (Mac, Linux, and Windows) that do not have Matlab installed.
As can be seen in an example (Figure  3 ), a folder is created for each segmentation technique containing all the segmented results according to the criteria chosen by users. The format of the images is saved in jpeg format and in a "mat file". This option is important if we want to later continue with further post-processing tasks with the obtained results. MedicalSeg offers some options to manage the configuration of the actual status (highlighted on pink box 8 in Figure  2 ). Users can save the current work as a project Matlab file pressing the "Save project" button; it is a useful feature as often a user might want to make different tests during their work, and, in this manner, it can retrieve previous results easily using this as a checkpoint. In the same way, one of the advantages of this application is to rapidly loading a previous checkpoint of the work by selecting a project Matlab file.
For the implementation, Matlab toolbox provides some functions for manipulating planar, closed splines to implement image or video segmentation by means of deformable (or active) contours. A previous version using a Python implementation was used  [22]. In this case, in a Matlab version only a sigma value and a number of iterations are used as an input parameters. Contour topology is managed in a way that should allow changes similar to what can be observed with level sets (merging and splitting but no whole creation).