Reviewerly/exHarmony huggingface dataset:
A methodological approach to enable natural language interaction in an Intelligent Tutoring System

In this paper, we present and evaluate the recent incorporation of a conversational agent into an Intelligent Tutoring System (ITS), using the open-source machine learning framework Rasa. Once it has been appropriately trained, this tool is capable of identifying the intention of a given text input and extracting the relevant entities related to the message content. We describe both the generation of a realistic training set in Spanish language that enables the creation of the required Natural Language Understanding (NLU) models and the evaluation of the resulting system. For the generation of the training set, we have followed a methodology that can be easily exported to other ITS. The model evaluation shows that the conversational agent can correctly identify the majority of the user intents, reporting an f1-score above 95%, and cooperate with the ITS to produce a consistent dialogue flow that makes interaction more natural.
Deep metric learning for robust radar signal recognition

Signal recognition technology is a currently active area in both civilian and military applications. Recently, deep learning has aroused extensive attempts in radar signal recognition due to its remarkable capability of automatic feature extraction. However, existing radar signal recognition networks overly depend on the probability-based decision model, resulting in poor robustness. This paper develops a novel deep metric learning frame to enhance the robustness of the recognition system. First, a multiscale atrous pyramid network (MAPNet) is proposed to efficiently learn high-resolution and distinct feature representation. Second, a variance loss is designed to constrain the intra-class feature distribution in metric space. Third, according to the distribution of training signals in metric space, recognition results are recalibrated to provide explicit rejection probabilities for unknowns. Extensive experiments and evaluations demonstrate that the proposed model can accurately classify known signals while robustly identifying unknown signals. The signal database and model can be freely accessed at https://github.com/bryantky/MAPNet.
Predicting Response of Triple-Negative Breast Cancer to Neoadjuvant Chemotherapy Using a Deep Convolutional Neural Network–Based Artificial Intelligence Tool

Achieving a pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) is associated with improved patient outcomes in triple-negative breast cancer (TNBC). Currently, there are no validated predictive biomarkers for the response to NAC in TNBC. We developed and validated a deep convolutional neural network-based artificial intelligence (AI) model to predict the response of TNBC to NAC.Whole-slide images (WSIs) of hematoxylin and eosin-stained core biopsies from 165 (pCR in 60 and non-pCR in 105) and 78 (pCR in 31 and non-pCR in 47) patients with TNBC were used to train and validate the model. The model extracts morphometric features from WSIs in an unsupervised manner, thereby generating clusters of morphologically similar patterns. Downstream ranking of clusters provided regions of interest and morphometric scores; a low score close to zero and a high score close to one represented a high or low probability of response to NAC.The predictive ability of AI score for the entire cohort of 78 patients with TNBC ascertained by receiver operating characteristic analysis demonstrated an area under the curve (AUC) of 0.75. The AUC for stages I, II, and III disease were 0.88, 0.73, and 0.74, respectively. Using a cutoff value of 0.35, the positive predictive value of the AI score for pCR was 73.7%, and the negative predictive value was 76.2% for non-pCR patients.To our knowledge, this study is the first to demonstrate the use of an AI tool on digitized hematoxylin and eosin-stained tissue images to predict the response to NAC in patients with TNBC with high accuracy. If validated in subsequent studies, these results may serve as an ancillary aid for individualized therapeutic decisions in patients with TNBC.
PhotoFourier: A Photonic Joint Transform Correlator-Based Neural Network Accelerator

The last few years have seen a lot of work to address the challenge of low-latency and high-throughput convolutional neural network inference. Integrated photonics has the potential to dramatically accelerate neural networks because of its low-latency nature. Combined with the concept of Joint Transform Correlator (JTC), the computationally expensive convolution functions can be computed instantaneously (time of flight of light) with almost no cost. This ‘free’ convolution computation provides the theoretical basis of the proposed PhotoFourier JTC-based CNN accelerator. PhotoFourier addresses a myriad of challenges posed by on-chip photonic computing in the Fourier domain including 1D lenses and high-cost optoelectronic conversions. The proposed PhotoFourier accelerator achieves more than 28× better energy-delay product compared to state-of-art photonic neural network accelerators.
TensorFHE: Achieving Practical Computation on Encrypted Data Using GPGPU

In the cloud computing era, privacy protection is becoming pervasive in a broad range of applications (e.g., machine learning, data mining, etc). Fully Homomorphic Encryption (FHE) is considered the perfect solution as it enables privacy-preserved computation on untrusted servers. Unfortunately, the prohibitive performance overhead blocks the wide adoption of FHE (about 10, 000× slower than the normal computation). As heterogeneous architectures have gained remarkable success in several fields, achieving high performance for FHE with specifically designed accelerators seems to be a natural choice. Until now, most FHE accelerators have focused on efficiently implementing one FHE operation at a time based on ASIC and with significantly higher performance than GPU and FPGA. However, recent state-of-the-art FHE accelerators rely on an expensive and large on-chip storage and a high-end manufacturing process (i.e., 7nm), which increase the cost of FHE adoption.In this paper, we propose TensorFHE, an FHE acceleration solution based on GPGPU for real applications on encrypted data. TensorFHE utilizes Tensor Core Units (TCUs) to boost the computation of Number Theoretic Transform (NTT), which is the part of FHE with highest time-cost. Moreover, TensorFHE focuses on performing as many FHE operations as possible in a certain time period rather than reducing the latency of one operation. Based on such an idea, TensorFHE introduces operation-level batching to fully utilize the data parallelism in GPGPU. We experimentally prove that it is possible to achieve comparable performance with GPGPU as with state-of-the-art ASIC accelerators. TensorFHE performs 913 KOPS and 88 KOPS for NTT and HMULT (key FHE kernels) within NVIDIA A100 GPGPU, which is 2.61× faster than state-of-the-art FHE implementation on GPGPU; Moreover, TensorFHE provides comparable performance to the ASIC FHE accelerators, which makes it even 2.9× faster than the F1+ with a specific workload. Such a pure software acceleration based on commercial hardware with high performance can open up usage of state-of-the-art FHE algorithms for a broad set of applications in real systems.
Chimera: An Analytical Optimizing Framework for Effective Compute-intensive Operators Fusion

Machine learning models with various tensor operators are becoming ubiquitous in recent years. There are two types of operators in machine learning: compute-intensive operators (e.g., GEMM and convolution) and memory-intensive operators (e.g., ReLU and softmax). In emerging machine learning models, compute-intensive operators are usually organized in a chain structure. With the continual specialization of hardware, the gap between computing performance and memory bandwidth has become more prominent. Consequently, the implementations of many compute-intensive operator chains are bounded by memory bandwidth, and generating fused kernels to improve locality for these compute-intensive operators becomes necessary. But in existing machine learning compilers, there lack both precise analysis and efficient optimization for compute-intensive operator chains on different accelerators. As a result, they usually produce sub-optimal performance for these operator chains.In this paper, we propose Chimera, an optimizing framework that can efficiently improve the locality of compute-intensive operator chains on different hardware accelerators. In Chimera, each compute-intensive operator is composed of a series of computation blocks. To generate efficient fused kernels for the operator chains, optimizations for both inter-block and intra-block are required. For inter-block optimization, Chimera decides the optimized block execution order by minimizing the data movement volume among blocks using an analytical model. For intra-block optimization, Chimera uses unified replaceable micro kernels to apply hardware-specific optimizations for different accelerators. Finally, Chimera generates fused kernels for compute-intensive operator chains. Evaluation of batch GEMM chains and convolution chains on CPU, GPU, and NPU shows that Chimera achieves up to 2.87×, 2.29×, and 2.39× speedups to hand-tuned libraries. Compared to state-of-the-art compilers, the speedups are up to 2.29×, 1.64×, and 1.14× for CPU, GPU, and NPU.
Co-Designed Architectures for Modular Superconducting Quantum Computers

Noisy, Intermediate Scale Quantum (NISQ) computers have reached the point where they can show the potential for quantum advantage over classical computing. Unfortunately, NISQ machines introduce sufficient noise that even for moderate size quantum circuits the results can be unreliable. We propose a collaboratively designed superconducting quantum computer using a Superconducting Nonlinear Asymmetric Inductive eLement (SNAIL) modulator. The SNAIL modulator is designed by considering both the ideal fundamental qubit gate operation while maximizing the qubit coupling capabilities. First, the SNAIL natively implements $\sqrt[n]{{{\text{iSWAP}}}}$ gates realized through proportionally scaled pulse lengths. This naturally includes $\sqrt {{\text{iSWAP}}} $, which provides an advantage over CNOT as a basis gate. Second, the SNAIL enables high-degree couplings that allow rich and highly parallel qubit connection topologies without suffering from frequency crowding. Building on our previously demonstrated SNAIL-based quantum state router we propose a quantum 4-ary tree and a hypercube inspired corral built from interconnected quantum modules. We compare their advantage in data movement based on necessary SWAP gates to the traditional lattice and heavy-hex lattice used in latest commercial quantum computers. We demonstrate the co-design advantage of our SNAIL-based machine with $\sqrt {{\text{iSWAP}}} $ basis gates and rich topologies against CNOT/heavy-hex and FSIM/lattice for 16-20 qubit and extrapolated designs circa 80 qubit architectures. We compare total circuit time and total gate count to understand fidelity for systems dominated by decoherence and control imperfections, respectively. Finally, we provide a gate duration sensitivity study on further decreasing the SNAIL pulse length to realize $\sqrt[n]{{{\text{iSWAP}}}}$ qubit systems to reduce decoherence times.
ISOSceles: Accelerating Sparse CNNs through Inter-Layer Pipelining

Sparse CNNs dramatically reduce computation and storage costs over dense ones. But sparsity also makes CNNs more data-intensive, as each value is reused fewer times. Thus, current sparse CNN accelerators, which process one layer at a time, are bottlenecked by memory traffic.We present ISOSceles, a new sparse CNN accelerator that dramatically reduces data movement through inter-layer pipelining: overlapping the execution of consecutive layers so that a layer’s output activations are quickly consumed by the next layer without spilling them off-chip. Pipelining greatly increases reuse, but it is challenging to implement with existing approaches, which are limited to dense CNNs. ISOSceles relies on a novel input-stationary output-stationary (IS-OS) dataflow that consumes inputs and produces outputs in the same order, greatly reducing intermediate sizes over existing dataflows. ISOSceles implements IS-OS efficiently and leverages time-multiplexing and dynamic scheduling to pipeline multiple layers despite the large variations in work that sparsity induces.On a wide range of sparse CNNs, ISOSceles outperforms a state-of-the-art accelerator by gmean 4.3× (up to 6.7×), and reduces traffic by 4.7× (up to 8.5×) while using less area.
Quaternion Representation Learning for cross-modal matching

The main challenge of cross-modal matching is to construct a shared subspace reflecting semantic closeness. Asymmetric relevance, especially the one-to-many matching case where multiple correspondences of a given query from the other modality can be obtained, has seriously exacerbated this difficulty. Recently, many approaches based on deep metric learning and probability distribution fitting have made great progress. However, these methods cannot generalize well to other approaches for their too many hyper-parameters or uncontrollable instability. In addition, we argue that the representational ability of common space learned in real space is insufficient and the symmetric similarity calculations adopted by the previous work fail to capture the asymmetric relevance. To seek a remedy for these problems, this work introduces a novel and effective approach called Quaternion Representation Learning (QRL) for better cross-modal matching. Specifically, benefiting from the strong expressive power and richer representation capability of quaternion space with three imaginary components, our proposed QRL method can better represent the shared semantic. Due to the inherent asymmetry of the Hamilton product and its latent inter-dependencies of components, our QRL method can model the asymmetric relevance and capture the complex similarity among intra- and inter-modal interaction. Another advantage is that our QRL method can be utilized in conjunction with other existing methods to promote cross-modal matching. Extensive experiments on two commonly used image-text matching benchmarks, i.e., MSCOCO and Flickr30K, and two widely used video-text retrieval datasets, including MSRVTT and TGIF, demonstrate the effectiveness and superiority of our proposed QRL method.
S-DETR: A Transformer Model for Real-Time Detection of Marine Ships

Due to the ever-changing shape and scale of ships, as well as the complex sea background, accurately detecting multi-scale ships on the sea while considering real-time requirements remains a challenge. To address this problem, we propose a model called S-DETR based on the DETR framework for end-to-end detection of ships on the sea. A scale attention module is designed to effectively learn the weights of different scale information by utilizing the global information brought by global average pooling. We analyzed the potential reasons for the performance degradation of the end-to-end detector and proposed a decoder based on Dense Query. Although the computational complexity and convergence of the entire S-DETR model have not been rigorously proven mathematically, Dense Query can reduce the computational complexity of multi-head self-attention from O(Nq2) into O(Nq). To evaluate the performance of S-DETR, we conducted experiments on the Singapore Maritime Dataset and Marine Image Dataset. The experimental results show that the proposed method can effectively solve the problem of multi-scale ship detection in complex marine environments and achieve state-of-the-art performance. The model inference speed of S-DETR is comparable to that of single-stage target detection models and meets the real-time requirements of shoreside ship detection.
How Do Users Experience Traceability of AI Systems? Examining Subjective Information Processing Awareness in Automated Insulin Delivery (AID) Systems

When interacting with artificial intelligence (AI) in the medical domain, users frequently face automated information processing, which can remain opaque to them. For example, users with diabetes may interact daily with automated insulin delivery (AID). However, effective AID therapy requires traceability of automated decisions for diverse users. Grounded in research on human-automation interaction, we study Subjective Information Processing Awareness (SIPA) as a key construct to research users’ experience of explainable AI. The objective of the present research was to examine how users experience differing levels of traceability of an AI algorithm. We developed a basic AID simulation to create realistic scenarios for an experiment with N = 80, where we examined the effect of three levels of information disclosure on SIPA and performance. Attributes serving as the basis for insulin needs calculation were shown to users, who predicted the AID system’s calculation after over 60 observations. Results showed a difference in SIPA after repeated observations, associated with a general decline of SIPA ratings over time. Supporting scale validity, SIPA was strongly correlated with trust and satisfaction with explanations. The present research indicates that the effect of different levels of information disclosure may need several repetitions before it manifests. Additionally, high levels of information disclosure may lead to a miscalibration between SIPA and performance in predicting the system’s results. The results indicate that for a responsible design of XAI, system designers could utilize prediction tasks in order to calibrate experienced traceability.
Efficient Approach to Color Image Segmentation Based on Multilevel Thresholding Using EMO Algorithm by Considering Spatial Contextual Information

The process of image segmentation is partitioning an image into its constituent parts and is a significant approach for extracting interesting features from images. Over a couple of decades, many efficient image segmentation approaches have been formulated for various applications. Still, it is a challenging and complex issue, especially for color image segmentation. To moderate this difficulty, a novel multilevel thresholding approach is proposed in this paper based on the electromagnetism optimization (EMO) technique with an energy curve, named multilevel thresholding based on EMO and energy curve (MTEMOE). To compute the optimized threshold values, Otsu's variance and Kapur's entropy are deployed as fitness functions; both values should be maximized to locate optimal threshold values. In both Kapur's and Otsu's methods, the pixels of an image are classified into different classes based on the threshold level selected on the histogram. Optimal threshold levels give higher efficiency of segmentation; the EMO technique is used to find optimal thresholds in this research. The methods based on an image's histograms do not possess the spatial contextual information for finding the optimal threshold levels. To abolish this deficiency an energy curve is used instead of the histogram and this curve can establish the spatial relationship of pixels with their neighbor pixels. To study the experimental results of the proposed scheme, several color benchmark images are considered at various threshold levels and compared with other meta-heuristic algorithms: multi-verse optimization, whale optimization algorithm, and so on. The investigational results are illustrated in terms of mean square error, peak signal-to-noise ratio, the mean value of fitness reach, feature similarity, structural similarity, variation of information, and probability rand index. The results reveal that the proposed MTEMOE approach overtops other state-of-the-art algorithms to solve engineering problems in various fields.
Towards Autonomous Driving Model Resistant to Adversarial Attack

Formulae display:?Mathematical formulae have been encoded as MathML and are displayed in this HTML version using MathJax in order to improve their display. Uncheck the box to turn MathJax off. This feature requires Javascript. Click on a formula to zoom.
Tumor Cellularity Assessment of Breast Histopathological Slides via Instance Segmentation and Pathomic Features Explainability

The segmentation and classification of cell nuclei are pivotal steps in the pipelines for the analysis of bioimages. Deep learning (DL) approaches are leading the digital pathology field in the context of nuclei detection and classification. Nevertheless, the features that are exploited by DL models to make their predictions are difficult to interpret, hindering the deployment of such methods in clinical practice. On the other hand, pathomic features can be linked to an easier description of the characteristics exploited by the classifiers for making the final predictions. Thus, in this work, we developed an explainable computer-aided diagnosis (CAD) system that can be used to support pathologists in the evaluation of tumor cellularity in breast histopathological slides. In particular, we compared an end-to-end DL approach that exploits the Mask R-CNN instance segmentation architecture with a two steps pipeline, where the features are extracted while considering the morphological and textural characteristics of the cell nuclei. Classifiers that are based on support vector machines and artificial neural networks are trained on top of these features in order to discriminate between tumor and non-tumor nuclei. Afterwards, the SHAP (Shapley additive explanations) explainable artificial intelligence technique was employed to perform a feature importance analysis, which led to an understanding of the features processed by the machine learning models for making their decisions. An expert pathologist validated the employed feature set, corroborating the clinical usability of the model. Even though the models resulting from the two-stage pipeline are slightly less accurate than those of the end-to-end approach, the interpretability of their features is clearer and may help build trust for pathologists to adopt artificial intelligence-based CAD systems in their clinical workflow. To further show the validity of the proposed approach, it has been tested on an external validation dataset, which was collected from IRCCS Istituto Tumori "Giovanni Paolo II" and made publicly available to ease research concerning the quantification of tumor cellularity.
A Spectral–Spatial Transformer Fusion Method for Hyperspectral Video Tracking

Hyperspectral videos (HSVs) can record more adequate detail clues than other videos, which is especially beneficial in cases of abundant spectral information. Although traditional methods based on correlation filters (CFs) employed to explore spectral information locally achieve promising results, their performances are limited by ignoring global information. In this paper, a joint spectral–spatial information method, named spectral–spatial transformer-based feature fusion tracker (SSTFT), is proposed for hyperspectral video tracking, which is capable of utilizing spectral–spatial features and considering global interactions. Specifically, the feature extraction module employs two parallel branches to extract multiple-level coarse-grained and fine-grained spectral–spatial features, which are fused with adaptive weights. The extracted features are further fused with the context fusion module based on a transformer with the hyperspectral self-attention (HSA) and hyperspectral cross-attention (HCA), which are designed to capture the self-context feature interaction and the cross-context feature interaction, respectively. Furthermore, an adaptive dynamic template updating strategy is used to update the template bounding box based on the prediction score. The extensive experimental results on benchmark hyperspectral video tracking datasets demonstrated that the proposed SSTFT outperforms the state-of-the-art methods in both precision and speed.