Reviewerly/exHarmony huggingface dataset:
Learning From Limited and Imbalanced Medical Images With Finer Synthetic Images From GANs

Chest X-ray is a prevalent medical imaging modality for detecting lung diseases. The clinical analysis of X-ray images is usually conducted by radiologists, who represent valuable human resources. In practice, situations with insufficient radiologists to timely analyze large quantities of X-ray data are very common. Accordingly, developing an automated computer-aided lung disease classification system is beneficial to facilitate diagnoses. However, due to restrictions of costs and time, collecting large amounts of accurately labeled X-ray images to train a machine learning based diagnosis system is challenging. Another limitation is the class imbalances present in datasets. Facing these challenges, we investigate the effectiveness of using generative models, particularly generative adversarial networks (GANs), to synthesize new data to tackle the issue of data paucity and class imbalances. To this end, it should be noted that few existing works have studied the effect of generated image quality on the performance of different learning models, particularly in medical imaging. Therefore, the current paper represents one of the first comprehensive investigations into the impact of synthetic image generation on classifier performance, which is empirically elucidated by a comparative analysis of a simpler deep convolutional GAN to a more complex progressive GAN design. Another contribution of this paper is a multi-scale convolutional neural network (CNN) architecture, which can take advantage of image features at different scales for better learning from scratch. Altogether, to verify the robustness of using GANs to augment datasets, we compare various data augmentation approaches, when applied to different network architectures, including transfer learning, learning-from-scratch CNNs, state-of-the-art ResNet, EfficientNet, DenseNet, and the proposed multi-scale CNN. Specifically, testing on two publicly available datasets, the obtained results show that using finer images synthesized from GANs with the proposed multi-scale CNN achieved good classification performance, under a wide range of operating conditions.
A Novel Hypercube-Based Heuristic for Quantum Boolean Circuit Synthesis

Quantum computation has extraordinary capabilities for solving complicated problems. As quantum computations are reversible by nature, reversible circuits are important for the development of quantum computation techniques. Designing an effective and efficient method for synthesizing reversible circuits to reduce costs and stabilize circuit efficiency is crucial. The traditional synthesis methods of solving reversible circuits focus on the conversion efficiency rather than discussing the properties of the reversible function. Thus, this paper aims to propose a novel synthesis method that directly and efficiently optimizes reversible circuit synthesis with the properties of the reversible circuit. The proposed method converts the reversible function into a hypercube, allowing visual observations of the overall circuit. Two new indicators, the adjacent Hamming distance (AHD) and total cycle distance (TCD), aid in effective decision-making, generating shorter circuits. Furthermore, we use the generalized Toffoli gate set, which without requiring any additional ancilla bits and has applications in error correction and fault tolerance. The experimental results show that our method can find better solutions than traditional methods, significantly reducing the gate count, while the hypercube assists in synthesizing the reversible circuit.
Domain-Specific Quantum Architecture Optimization

With the steady progress in quantum computing over recent years, roadmaps for upscaling quantum processors have relied heavily on the targeted qubit architectures. So far, similarly to the early age of classical computing, these designs have been crafted by human experts. These general-purpose architectures, however, leave room for customization and optimization, especially when targeting popular near-term QC applications. In classical computing, customized architectures have demonstrated significant performance and energy efficiency gains over general-purpose counterparts. In this paper, we present a framework for optimizing quantum architectures, specifically through customizing qubit connectivity. It is the first work that (1) provides performance guarantees by integrating architecture optimization with an optimal compiler, (2) evaluates the impact of connectivity customization under a realistic crosstalk error model, and (3) benchmarks on realistic circuits of near-term interest, such as the quantum approximate optimization algorithm (QAOA) and quantum convolutional neural network (QCNN). We demonstrate up to 59% fidelity improvement in simulation by optimizing the heavy-hexagon architecture for QAOA maxcut circuits, and up to 14% improvement on the grid architecture. For the QCNN circuit, architecture optimization improves fidelity by 11% on the heavy-hexagon architecture and 605% on the grid architecture.
A Self-Attention Mask Learning-Based Recommendation System

The primary purpose of sequence modeling is to record long-term interdependence across interaction sequences, and since the number of items purchased by users gradually increases over time, this brings challenges to sequence modeling to a certain extent.Relationships between terms are often overlooked, and it is crucial to build sequential models that effectively capture long-term dependencies.Existing methods focus on extracting global sequential information, while ignoring deep representations from subsequences.We argue that limited item transfer is fundamental to sequence modeling, and that partial substructures of sequences can help models learn more efficient long-term dependencies compared to entire sequences.This paper proposes a sequence recommendation model named GAT4Rec (Gated Recurrent Unit And Transformer For Recommendation), which uses a Transformer layer that shares parameters across layers to model the user's historical interaction sequence.The representation learned by the gated recurrent unit is used as a gating signal to filter out better substructures of the user sequence.The experimental results demonstrate that our proposed GAT4Rec model is superior to other models and has a higher recommendation effectiveness.
Target drift discriminative network based on deep learning in visual tracking

In visual tracking, sometimes the target response value is high, but it is not the tracking result, which can result in the wrong judgment. Moreover, the threshold to decide the tracking result needs to be set artificially in the traditional discriminative methods. We propose a deep learning-based target drift discriminative network to judge whether the target is lost. We design a lightweight network without the threshold, using four convolutional layers, three full connection layers, and the Softmax function to judge the tracking results. When training the network, the established positive and negative samples are used, and we select difficult samples for further training to achieve a better target discriminative effect. Finally, a target drift discriminative network is introduced into the accurate tracking by overlap maximization. When it is judged that the target is lost, another search area is selected to quickly find the target. Numerous experiments show that our method achieves the best performance on datasets UAV123, UAV20L, and VOT2018-LT, especially on the UAV20L dataset, for which the tracking precision and tracking success rate are improved by 3.7% and 2.8%. Compared with several other classical threshold discriminative criteria, we do not need to set the threshold artificially and have better judgment performance.
Robust Processing Advantage for Binomial Phrases with Variant Conjunctions

Prior research has shown that various types of conventional multiword chunks are processed faster than matched novel strings, but it is unclear whether this processing advantage extends to variant multiword chunks that are less formulaic. To determine whether the processing advantage of multiword chunks accommodates variations in the canonical phrasal template, we examined the robustness of the processing advantage (i.e., Predictability) of binomial phrases with non-canonical conjunctions (e.g., salt and also pepper; salt as well as pepper). Results from the cloze study (Experiment 1) showed that there was a high tendency of producing the canonical conjunct (pepper), even in the binomials that contained non-formulaic conjunctions. Consistent with these findings, results from two eye tracking studies (Experiments 2a and 2b) showed that canonical conjuncts were read faster than novel conjuncts that were matched on word length (e.g., paprika), even in the binomials with variant conjunctions. This robust online processing advantage was replicated in a self-paced reading study that compared all three Conjunction Types (Experiment 3). Taken together, these findings show that binomials with variant function words also receive facilitated processing relative to matched novel strings, even though both types of strings are neither conventional nor relatively frequent. Exploratory analyses revealed that this processing speed advantage was driven by the lexical-semantic association between the canonical conjuncts (salt-pepper), rather than lexical and phrasal frequency. Overall, these results highlight flexibility in the processing of multiword chunks that current models of multiword storage and processing must take into account.
Improving Image Paragraph Captioning with Dual Relations

Image paragraph captioning aims to generate multiple de-scriptive sentences for an image. However, most previous methods ignore the explicit relations among objects resulting in unsatisfactory performance. In this paper, we propose a novel model (i.e., DualRel) to capture spatial and seman-tic relations among objects. Specifically, the spatial relation embedding is obtained solely from images using a predefined geometry pattern. With the help of captions, the semantic relation embedding is learned in a weakly supervised man-ner. These two relation embeddings are then interacted with regional features of objects through a relation-aware attention interaction. It first obtains a visual context vector using regional features. Then with the visual context vector, we obtain the corresponding spatial and semantic relation-aware vectors using attentions. These three vectors are fused with two gates for language decoding to further generate a para-graph. Experimental results on Stanford benchmark dataset show that DualRel achieves remarkable improvements <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Code released at https://github.com/fuyunll07/DualRel.
Generative and Adaptive Multi-Label Generalized Zero-Shot Learning

We address the problem of multi-label generalized zero-shot learning where the task is to predict the labels (usually more than one) of a target image whether each of its labels belongs to the seen or unseen category. To alleviate the extreme data-imbalance problem, in which no annotated images are available for unseen classes during training, state-of-the-art single-label zero-shot learning methods learn to synthesize the class-specific visual features from seen classes. However, synthesizing multi-label visual features from multi-label images has not been extensively studied. By exploring the relationship between an image and its labels, we address the multi-label generalized zero-shot learning problem via a hybrid framework of generative and adaptive learning. We convert an image into a label classifier, which can vary among intra-class samples. The adaptive mechanism facilitates the usage of a single-label feature generating model for creating multi-label features from multi-label images. We show that the proposed method improves the state of the art ZSL/GZSL methods on two benchmark datasets.
Multi-Scale Graph Attention Network for Scene Graph Generation

Scene graph provides a high-level scene understanding of the image, which has a wide range of applications in computer vision. Previous methods elaborately design many message passing strategies and uniformly treat instances in the image to capture contextual information. These methods, however, fail to grasp the salient objects and their relations, which are the basis of understanding the content of images. To capture the interaction among salient instances, we propose a novel Multi-Scale Graph Attention Network (MSGAT) that gradually shrinks the graph scale to retain salient instances, and then expands it to encode the multi-scale context. Our proposed MSGAT contains two sub-modules: Multi-Scale Message Passing (MSMP) and Relationship Filtering Module (RFM), which are designed to enhance features of salient instances and filter redundant relationships, respectively. Extensive experiments demonstrate that MSGAT outperforms previous methods and achieves state-of-the-art performances on Visual Genome.
A Unified Two-Stage Group Semantics Propagation and Contrastive Learning Network for Co-Saliency Detection

Co-saliency detection (CoSOD) aims at discovering the repetitive salient objects from multiple images. Two primary challenges are group semantics extraction and noise object suppression. In this paper, we present a unified Two-stage grOup semantics PropagatIon and Contrastive learning NETwork (TopicNet) for CoSOD. TopicNet can be decomposed into two substructures, including a two-stage group semantics propagation module (TGSP) to address the first challenge and a contrastive learning module (CLM) to address the second challenge. Concretely, for TGSP, we design an image-to-group propagation module (IGP) to capture the consensus representation of intra-group similar features and a group-to-pixel propagation module (GPP) to build the relevancy of consensus representation. For CLM, with the design of positive samples, the semantic consistency is enhanced. With the design of negative samples, the noise objects are suppressed. Experimental results on three prevailing benchmarks reveal that TopicNet outperforms other competitors in terms of various evaluation metrics.
Comparing Interpretable AI Approaches for the Clinical Environment: an Application to COVID-19

Machine Learning (ML) models play an important role in healthcare thanks to their remarkable performance in predicting complex phenomena. During the COVID-19 pandemic, different ML models were implemented to support decisions in the medical settings. However, clinical experts need to ensure that these models are valid, provide clinically useful information, and are implemented and used correctly. In this vein, they need to understand the logic behind the models to be able to trust them. Hence, developing transparent and interpretable models has increasing relevance. In this work, we applied four interpretable ML models including logistic regression, decision tree, pyFUME, and RIPPER to classify suspected COVID-19 patients based on clinical data collected from blood samples. After preprocessing the data set and training the models, we evaluate the models based on their predictive performance. Then, we illustrate that interpretability can be achieved in different ways. First, SHAP explanations are built from logistic regression and decision trees to obtain the features' importance. Then, the potential of pyFUME and RIPPER in providing inherent interpretability are reflected. Finally, potential ways to achieve trust in future studies are briefly discussed.
Hyperpointnet for Point Cloud Sequence-Based 3D Human Action Recognition

Point cloud sequence-based 3D action recognition achieves impressive performance and efficiency. Conventional approaches for modeling point cloud sequences usually perform cross-frame spatio-temporal local encoding, thus resulting in intensive computation and mutual interference between spatial and temporal information extracting. In this work, to avoid spatio-temporal local encoding, we propose a strong parallelized point cloud sequence network referred to as HyperpointNet for 3D action recognition. HyperpointNet is composed of two serial modules, i.e., a hyperpoint sequence embedding module and a hyperpoint sequence encoding module. In the hyperpoint sequence embedding module, employing static point cloud modeling methods, the point cloud sequence is abstracted into a new point data type named hyper-point sequence. In the hyperpoint sequence encoding module, a temporal PointNet (TPN) layer is designed to model the hyperpoint sequence. Extensive experiments conducted on two public datasets show that HyperpointNet outperforms state-of-the-art approaches.
Source Camera Identification with Multi-Scale Feature Fusion Network

Source camera identification (SCI) technology has attracted increasing attentions over the past few years. However, the existing methods suppress image content with denoising filters that are largely agnostic to the specific sensor pattern noise (SPN) signal of interest. Such practices may potentially degrade the performance of SPN-based SCI due to un-reliable SPNs, especially when forensic images are transmitted through social networking platforms. In this paper, we address the problem of SPN-based device identification and propose a multi-scale feature fusion network (MSFFN) to boost the sensor-based source camera identification attribution. Specifically, several image patches of different scales are selected and input into the MSFFN to extract the SPN. The MSFFN is a multi-scale encoder-decoder structure, which is used to suppress image content and improve source attribution. Subsequently, the content-independent SPN features of different scales are fused. At last, the fused features are used for image source identification. Experimental results compared with the state-of-the-art demonstrate that the proposed scheme achieves significant improvements, especially in the accuracy of social networking image source identification.
HDNet: A Hierarchically Decoupled Network for Crowd Counting

Recently, density map regression-based methods have dominated in crowd counting owing to their excellent fitting ability on density distribution. However, further improvement tends to saturate mainly because of the confusing background noise and the large density variation. In this paper, we propose a Hierarchically Decoupled Network (HDNet) to solve the above two problems within a unified framework. Specifically, a background classification sub-task is decomposed from the density map prediction task, which is then assigned to a Density Decoupling Module (DDM) to exploit its highly discriminative ability. For the remaining foreground prediction sub-task, it is further hierarchically decomposed to several density-specific sub-tasks by the DDM, which are then solved by the regression-based experts in a Foreground Density Estimation Module (FDEM). Although the proposed strategy effectively reduces the hypothesis space so as to relieve the optimization for those task-specific experts, the high correlation of these sub-tasks are ignored. Therefore, we introduce three types of interaction strategies to unify the whole framework, which are Feature Interaction, Gradient Interaction, and Scale Interaction. Integrated with the above spirits, HDNet achieves state-of-the-art performance on several popular counting benchmarks.
FPAR: Filter Pruning Via Attention and Rank Enhancement

In recent years, deep convolutional neural networks (CNNs) have become larger than ever, thus their deployment on edge devices becomes difficult. There are numerous popular methods to accelerate the networks; however, many of these methods consider only the importance of a single filter to the network and neglect the coorelation between filters. To solve this problem, we propose a novel filter pruning method, called Filter Pruning via Attention and Rank Enhancement (FPAR), based on the attention mechanism and rank of feature maps. Moreover, the inspiration for it comes from a discovery: For a network with attention modules, irrespective of the batch of input images, the mean of channel-wise weights of the attention module is almost constant. Thus, we can use a few batches of input data to obtain this indicator to guide pruning. With extensive experiments on various datasets, demonstrate that our method outperforms the most advanced methods with similar accuracy. For example, using VGG-16, we removed 62.8% of floating-point operations (FLOPs) even with a 0.24% of the accuracy increase compared with the unpruned network.
Causal Video Summarizer for Video Exploration

Recently, video summarization has been proposed as a method to help video exploration. However, traditional video summarization models only generate a fixed video summary which is usually independent of user-specific needs and hence limits the effectiveness of video exploration. Multi-modal video summarization is one of the approaches utilized to address this issue. Multi-modal video summarization has a video input and a text-based query input. Hence, effective modeling of the interaction between a video input and text-based query is essential to multi-modal video summarization. In this work, a new causality-based method named Causal Video Summarizer (CVS) is proposed to effectively capture the interactive information between the video and query to tackle the task of multi-modal video summarization. The proposed method consists of a probabilistic encoder and a probabilistic decoder. Based on the evaluation of the existing multi-modal video summarization dataset, experimental results show that the proposed approach is effective with the increase of +5.4% in accuracy and +4.92% increase of F1-score, compared with the state-of-the-art method.
RMLANet: Random Multi-Level Attention Network for Shadow Detection

This paper addresses the problem of shadow detection from a single image. Previous approaches have shown that exploiting both global and local contexts in deep convolutional neural network layers can greatly improve performance. However, multi-level contexts remain underexplored. To achieve this, we propose RMLANet, a novel Random Multi-Level Attention Network. Specifically, we leverage shuffled multi-level features simultaneously with guiding features, and employ the transformer to capture global context. Furthermore, to reduce the computational and memory overhead caused by the self-attention mechanism in the vanilla transformer, we propose a random sampling strategy to reduce the number of inputs to the transformer. This is motivated by observing local consistency in images, which suggests that dense attention is unnecessary. Extensive experimental results demonstrate that our method outperforms current state-of-the-art methods on three widely used benchmark datasets SBU, ISTD and UCF.
Deep Blind Image Quality Assessment Using Dual-Order Statistics

Deep convolutional neural networks (CNNs) have become a promising approach to blind image quality assessment (BIQA). Existing CNN-based BIQA methods often employ global average pooling (GAP) to aggregate feature maps into a fixed-size representation for regression, so as to handle input images with varying sizes. However, GAP is only capable of extracting the first-order statistics of feature distributions, which is ineffective for distinguishing complex distortions that cause local degradation or preserve global features. To tackle this problem, we introduce the second-order global covariance pooling (GCP) for aggregating feature maps, leading to a more distortion-sensitive and more discriminative global representation. By incorporating GCP and GAP into a ResNet backbone, we propose an effective deep model for BIQA. The experimental results on five BIQA benchmark datasets, including both the synthetic and authentic ones, have demon-strated the excellent performance of the proposed method.