Reviewerly/exHarmony huggingface dataset:
ModDrop++: A Dynamic Filter Network with Intra-subject Co-training for Multiple Sclerosis Lesion Segmentation with Missing Modalities

Multiple Sclerosis (MS) is a chronic neuroinflammatory disease and multi-modality MRIs are routinely used to monitor MS lesions. Many automatic MS lesion segmentation models have been developed and have reached human-level performance. However, most established methods assume the MRI modalities used during training are also available during testing, which is not guaranteed in clinical practice. Previously, a training strategy termed Modality Dropout (ModDrop) has been applied to MS lesion segmentation to achieve the state-of-the-art performance with missing modality. In this paper, we present a novel method dubbed ModDrop++ to train a unified network adaptive to an arbitrary number of input MRI sequences. ModDrop++ upgrades the main idea of ModDrop in two key ways. First, we devise a plug-and-play dynamic head and adopt a filter scaling strategy to improve the expressiveness of the network. Second, we design a co-training strategy to leverage the intra-subject relation between full modality and missing modality. Specifically, the intra-subject co-training strategy aims to guide the dynamic head to generate similar feature representations between the full- and missing-modality data from the same subject. We use two public MS datasets to show the superiority of ModDrop++. Source code and trained models are available at https://github.com/han-liu/ModDropPlusPlus .
Layer Ensembles: A Single-Pass Uncertainty Estimation in Deep Learning for Segmentation

Uncertainty estimation in deep learning has become a leading research field in medical image analysis due to the need for safe utilisation of AI algorithms in clinical practice. Most approaches for uncertainty estimation require sampling the network weights multiple times during testing or training multiple networks. This leads to higher training and testing costs in terms of time and computational resources. In this paper, we propose Layer Ensembles, a novel uncertainty estimation method that uses a single network and requires only a single pass to estimate epistemic uncertainty of a network. Moreover, we introduce an image-level uncertainty metric, which is more beneficial for segmentation tasks compared to the commonly used pixel-wise metrics such as entropy and variance. We evaluate our approach on 2D and 3D, binary and multi-class medical image segmentation tasks. Our method shows competitive results with state-of-the-art Deep Ensembles, requiring only a single network and a single pass.
Massively scalable wavelength diverse integrated photonic linear neuron

Abstract As computing resource demands continue to escalate in the face of big data, cloud-connectivity and the internet of things, it has become imperative to develop new low-power, scalable architectures. Neuromorphic photonics, or photonic neural networks, have become a feasible solution for the physical implementation of efficient algorithms directly on-chip. This application is primarily due to the linear nature of light and the scalability of silicon photonics, specifically leveraging the wide-scale complementary metal-oxide-semiconductor manufacturing infrastructure used to fabricate microelectronics chips. Current neuromorphic photonic implementations stem from two paradigms: wavelength coherent and incoherent. Here, we introduce a novel architecture that supports coherent and incoherent operation to increase the capability and capacity of photonic neural networks with a dramatic reduction in footprint compared to previous demonstrations. As a proof-of-principle, we experimentally demonstrate simple addition and subtraction operations on a foundry-fabricated silicon photonic chip. Additionally, we experimentally validate an on-chip network to predict the logical 2 bit gates AND, OR, and XOR to accuracies of 96.8%, 99%, and 98.5%, respectively. This architecture is compatible with highly wavelength parallel sources, enabling massively scalable photonic neural networks.
SCGRU: A general approach for identifying multiple classes of self-admitted technical debt with text generation oversampling

Identifying self-admitted technical debt (SATD) plays an important role in maintaining software stability and improving software quality. Although existing methods can detect SATD and researchers have identified design debt and requirement debt, an approach to realize multiple classification of SATD, including defect, test, and documentation, is still lacking. In this paper, we combine text generation oversampling and the Convolutional Neural Networks-Gated Recurrent Unit (CNNGRU) model, and propose an approach called SCGRU to classify multiple debt, including defect, test, documentation, design, and requirement. First, SeqGAN-based text generation is employed to generate new samples by learning the original SATD data, thereby increasing the number of SATD samples such as defect debt and reducing data imbalance. Then, we apply the CNNGRU model to refine SATD into multiple classes. An experiment with cross-project identification of 10 projects shows that our approach is more effective than existing methods such as CNN and text mining. The proposed SCGRU approach has strong advantages especially in cases of flawed debt with very unbalanced data such as test debt and documention debt.
Intent-Aware Data Visualization Recommendation

Abstract This paper proposes a visualization recommender system for tabular data given visualization intents (e.g., “population trends in Italy” and “smartphone market share”). The proposed method predicts the most suitable visualization type (e.g., line, pie, or bar chart) and visualized columns (columns used for visualization) based on statistical features extracted from the tabular data as well as semantic features derived from the visualization intent. To predict the appropriate visualization type, we propose a bi-directional attention (BiDA) model that identifies important table columns using the visualization intent and important parts of the intent using the table headers. To determine the visualized columns, we employ a pre-trained neural language model to encode both visualization intents and table columns and predict which columns are the most likely to be used for visualization. Since there was no available dataset for this task, we created a new dataset consisting of over 100 K tables and their appropriate visualization. Experiments revealed that our proposed methods accurately predicted suitable visualization types and visualized columns.
Design of the Student Attendance Management System Based on the Internet of Things Technology

The IoT refers to the linking of items to a network through message-generating devices; in the connection process, communication and information circulation are carried out through the information dissemination medium to realize intelligent identification, tracking, supervision, and other functions. Attendance management is a system for assessing employee attendance, including being late, leaving early, and asking for leave. Student attendance refers to the attendance rate of students in class. This article aimed to study the design and research of the student attendance management system based on the Internet of things technology, and use this system to accurately record students’ attendance and develop good habits for students. An effective attendance management company has been developed to enhance the collection of information for student management in universities, to keep track of teachers’ and students’ teaching attendance in real time, and to connect teachers and students in real time using computer technology. Access to the Android version of the software speeds up the collection and processing of information and can play an important role in improving the efficiency of personnel management. On the basis of analysing the operation logic of the attendance system, matching programs and the attendance system software are designed. The experimental results of this article show that the attendance rate of students with utilitarian personality is the highest, as high as 70%; the attendance rate of students with helpless personality is 13%; the attendance rate of students who follow the current personality accounted for 10%; and the attendance rate of nonmotivated students accounted for 7%. It can be seen from this that personality is an important factor affecting students’ attendance rate. The attendance rate of positive psychology with goals is the highest, and the attendance rate of students without clear goals is the lowest. The student attendance management system of the Internet of things technology has been applied in practical work, but with the continuous construction and development of the school, the continuous improvement of new software and hardware, and the emergence of new needs in practical work, the student attendance management system is easy to crash when processing a large amount of data at the same time.
Gaze-assisted automatic captioning of fetal ultrasound videos using three-way multi-modal deep neural networks

In this work, we present a novel gaze-assisted natural language processing (NLP)-based video captioning model to describe routine second-trimester fetal ultrasound scan videos in a vocabulary of spoken sonography. The primary novelty of our multi-modal approach is that the learned video captioning model is built using a combination of ultrasound video, tracked gaze and textual transcriptions from speech recordings. The textual captions that describe the spatio-temporal scan video content are learnt from sonographer speech recordings. The generation of captions is assisted by sonographer gaze-tracking information reflecting their visual attention while performing live-imaging and interpreting a frozen image. To evaluate the effect of adding, or withholding, different forms of gaze on the video model, we compare spatio-temporal deep networks trained using three multi-modal configurations, namely: (1) a gaze-less neural network with only text and video as input, (2) a neural network additionally using real sonographer gaze in the form of attention maps, and (3) a neural network using automatically-predicted gaze in the form of saliency maps instead. We assess algorithm performance through established general text-based metrics (BLEU, ROUGE-L, F1 score), a domain-specific metric (ARS), and metrics that consider the richness and efficiency of the generated captions with respect to the scan video. Results show that the proposed gaze-assisted models can generate richer and more diverse captions for clinical fetal ultrasound scan videos than those without gaze at the expense of the perceived sentence structure. The results also show that the generated captions are similar to sonographer speech in terms of discussing the visual content and the scanning actions performed.
Towards counterfactual and contrastive explainability and transparency of DCNN image classifiers

Explainability of deep convolutional neural networks (DCNNs) is an important research topic that tries to uncover the reasons behind a DCNN model’s decisions and improve their understanding and reliability in high-risk environments. In this regard, we propose a novel method for generating interpretable counterfactual and contrastive explanations for DCNN models. The proposed method is model intrusive that probes the internal workings of a DCNN instead of altering the input image to generate explanations. Given an input image, we provide contrastive explanations by identifying the most important filters in the DCNN representing features and concepts that separate the model’s decision between classifying the image to the original inferred class or some other specified alter class. On the other hand, we provide counterfactual explanations by specifying the minimal changes necessary in such filters so that a contrastive output is obtained. Using these identified filters and concepts, our method can provide contrastive and counterfactual reasons behind a model’s decisions and makes the model more transparent. One of the interesting applications of this method is misclassification analysis, where we compare the identified concepts from a particular input image and compare them with class-specific concepts to establish the validity of the model’s decisions. The proposed method is compared with state-of-the-art and evaluated on the Caltech-UCSD Birds (CUB) 2011 dataset to show the usefulness of the explanations provided. • We extract knowledge learned by DCNNs in the form of filters that represent features and concepts important for the decision-making process. • Using the identified filters and concepts, we provide contrastive and counterfactual reasons behind a model’s decisions, making the model more transparent. • Qualitative and quantitative analysis with state-of-the-art has shown the efficacy of the proposed explanation method.
Improvement of renal image recognition through resolution enhancement

Image resizing is frequently used as a preprocessing step in many computer vision tasks, especially in medical applications. While tuning of the resizing method is usually omitted in the studies, there are many problems in which the exact influence of resampling on image textures and gradients is significant. The paper presents an in-depth analysis of image reconstruction’s impact on two inherent tasks in medical image analysis: segmentation and classification. The proposed study is conducted on the renal diagnosis dataset in which the kidney is segmented, and three renal tumours are classified. A novel image reconstruction method is introduced, namely Sampling Kantorovich Algorithm (SKA). It is compared to six other popular techniques widely used in image processing. Based on the qualitative and quantitative analyses, we proved that choice of image reconstruction method impacts the system’s overall performance. SKA turns out to be the best performing method in the classification setup. It boosts performance to 75% of the weighted F1-score by approximately 3 percentage points (pp) compared to the best baseline solution. In kidney segmentation, the SKA improves efficiency by over 2pp compared to other resizing methods. The results presented in this paper may apply to a wide range of medical image processing problems.
Gender and ethnicity recognition based on visual attention-driven deep architectures

Most of the time, when people observe, interact or speak to each other, they focus the attention on the ocular parts of the face. This daily life experience has a strong impact on the analysis of periocular facial regions. These facial regions may be exploited in order to identify individuals for several applications, including access control and services such as telebanking and electronic transactions. In this paper we suggest studying the efficiency of the periocular regions on gender and race prediction. Most researchers propose a local texture description based on LBP (Local Binary Pattern) and HoG (Histogram of Oriented Gradients) for the purpose of predicting gender. On the other hand, Deep learning techniques were proposed to predict the gender. However, this requires a huge labeled periocular data for gender which is not available. Also, the expressivity of gender and race can be decreased on the final representation of the Deep architectures comparing to the earlier stages. To overcome these points and for the aim of predicting gender and race, considering also the high impact of DCNNs (Deep Convolutional Neural Networks) techniques to solve several aspects in biometrics, we suggest a Deep architecture based on visual attention on the periocular part. The visual saliency extraction is based on primary layers’ activation by analyzing the feature-maps. We study how the visual attention-based features coupled to Deep Neural Networks can be used to discriminate between gender and race, hence extract a significant feature from periocular regions. Different pretrained architectures such as Alexnet and ResNet-50 were considered to extract visual saliency points or interest points. Several experiments were performed on periocular regions and a comparative study was conducted. The present results not only demonstrate the feasibility but also the robustness of the extracted interest points. • Efficiency of periocular region for gender and race prediction. • Deep architectures based on visual attention on periocular regions. • Visual attention-based features coupled to Deep Neural Networks. • Visual saliency extraction based on primary layers’ activation.
A simple noniterative method to accurately calculate the centroid of an interval type‐2 fuzzy set

An interval type-2 fuzzy set (IT2FS) contains the infinite number of embedded membership functions (MFs) defined as type-1 fuzzy sets. The center of gravity (COG) of an IT2FS is obtained by integrating the centroids of all these MFs. However, obtaining the COG of an IT2FS in a continuous domain is impossible because the number of embedded MFs is infinite and, in a discrete domain, comes with exponential time complexity. Therefore, different algorithms were proposed to find the center of centroids (COCs) instead of the COG. These algorithms do not necessarily obtain the exact value of COC; they compete to find a more accurate result in less time. In this study, we propose a simple noniterative method to obtain the COG of IT2FSs. This method only considers the MFs which form the boundary values, that is, the lower MF (LMF) and upper MF (UMF), and obtains their centroids separately. We show that the COG of an IT2FS is equal to the weighted sum of the centroids of its LMF and UMF if the relative areas of LMF and UMF are considered their corresponding weights. We call this method the Indirect COG (ICOG) method because it indirectly obtains the COG of an IT2FS. This method finds the accurate COG of an IT2FS by considering only two MFs instead of infinite MFs. It can also obtain the accurate COG of an uncommon polygonal IT2FS schematically. We use numerical examples to illustrate that the ICOG obtains the accurate COG of an IT2FS very fast in a nonexponential time. We also develop a new triangular interval type-2 fuzzy analytical hierarchy process that uses the ICOG method to accurately defuzzify the IT2 fuzzy weights.
A Light-Weight Interpretable Model for Nuclei Detection and Weakly-Supervised Segmentation

The field of computational pathology has witnessed great advancements since deep neural networks have been widely applied. These networks usually require large numbers of annotated data to train vast parameters. However, it takes significant effort to annotate a large histo-pathology dataset. We introduce a light-weight and interpretable model for nuclei detection and weakly-supervised segmentation. It only requires annotations on isolated nucleus, rather than on all nuclei in the dataset. Besides, it is a generative compositional model that first locates parts of nucleus, then learns the spatial correlation of the parts to further locate the nucleus. This process brings interpretability in its prediction. Empirical results on an in-house dataset show that in detection, the proposed method achieved comparable or better performance than its deep network counterparts, especially when the annotated data is limited. It also outperforms popular weakly-supervised segmentation methods. The proposed method could be an alternative solution for the data-hungry problem of deep learning methods.
Unsupervised Cross-disease Domain Adaptation by Lesion Scale Matching

Breast and thyroid lesions share many similarities in the feature representations of ultrasound images. However, there is a huge lesion scale gap between the two diseases, making it difficult to transfer knowledge between them through current methods for unsupervised domain adaptation. To address this problem, we propose a lesion scale matching approach where we employ a framework of latent space search for bounding box size to re-scale the source domain images, and then the Monte Carlo Expectation Maximization algorithm is used for optimization to match the lesion scales between the two disease domains. Extensive experimental results demonstrate the feasibility of cross-disease knowledge transfer, and our proposed method substantially improves the performance of unsupervised cross-disease domain adaptation models, with the Accuracy, Recall, Precision, and F1-score improved by 8.29%, 6.41%, 11.25%, and 9.14% on average in the three sets of ablation experiments.
Point process statistics improves particle size analysis

Abstract This paper re-considers the foundations of particle size statistics. While traditional particle size statistics consider their data as samples of random variables and use methods of classical mathematical statistics, here a particle sample is treated as a point process sample, and a suitable form of statistics is recommended. The whole sequence of ordered particle sizes is considered as a random variable in a suitable sample space. Instead of distribution functions, point process intensity functions are used. The application of point process data analysis is demonstrated for samples of fragments from single-particle crushing of glass balls. Three cases of data handling with point processes are presented: statistics for oversize particles, pooling of independent particle samples and pooling of piecewise particle data. Finally, the problem of goodness-of-fit testing for particle samples is briefly discussed. The point process approach turns out to be an extension of the classical approach, is simpler and more elegant, but retains all valuable traditional ideas. It is particularly strong in the analysis of oversize particles. Graphical abstract
Hermitian operator of a Mach–Zehnder interferometer combined with parity detection and its applications

In quantum metrology, parity detection is often used to investigate the phase estimation based on optical interferometer. Different from previous work, we consider a Mach–Zehnder interferometer (MZI) and parity detection as a whole operation in Heisenberg picture, and obtain the corresponding Hermitian operator. By this alternative operator method, one can derive the parity signal directly in terms of input states. Our results clearly show that the phase sensitivity obtained by the parity detection is independent of both configurations of the estimated phases and the output ports of the MZI. As applications, we analytically demonstrate that the no-go theorem indeed holds for both parity detection and single-mode intensity detection. We also investigate the phase estimation of the MZI seeded by the two-mode squeezed coherent state, and obtain the more concise and illuminating results.
Vector Set Classification by Signal Subspace Matching

We present a powerful solution to the problem of vector set classification, based on a novel goodness-of-fit metric, referred to as signal subspace matching (SSM). Unlike the existing solutions based on principal component analysis (PCA), this solution is eigendecomposition-free and dimension-selection-free, i.e., it does not require PCA nor the election of the subspace dimension, which is done implicitly. More importantly, it copes effectively with the challenging cases wherein the subspaces characterizing the classes are partially or fully overlapping. The SSM metric matches the subspaces characterizing the vector sets of the test and the classes by minimizing the distance between respective soft-projection matrices constructed from the vector sets. We prove the consistency of the solution for the high signal-to-noise-ratio limit, and also for the large-sample limit, conditioned on the noise being white. Experimental results, demonstrating the superiority of the SSM solution over the existing PCA-based solutions, especially in the challenging cases of overlapping subspaces, are included.