Reviewerly/exHarmony huggingface dataset:
Traffic sign attack via pinpoint region probability estimation network

Recent work show that Deep Neural Networks (DNNs) have created great performance in many tasks, but they are vulnerable to adversarial examples which trigger Artificial Intelligence (AI) security risks. Especially in the autonomous driving field, attacking a traffic sign classification network results in a serious consequence. Most existing researches prefer to digital level attacks focusing on smaller or more imperceptible adversarial noise. Given that attacks available to real-world implementation usually emerge in more security-critical scenarios, we propose an adaptively adversarial example generation algorithm for physical attacks in the real-world setting. Taking account of the traffic sign classification, our approach is divided into two steps. The first step is to generate a probability map which precisely predicts the probability of being attacked for each pixel in the input image through the proposed Pinpoint Region Probability Estimation Network (PRPEN) and meanwhile, try to reduce the size of the highlighted area in the map. It can also be regarded as a classification problem in which every pixel has two classes, suitable for attacking or not, including the restrictions on the number of items in the suitable sort. The second one is to remake a mask depending on the probability map and optimize adversarial patches only on what mask decides. Experimental results show that our method achieves almost 100% misclassification rate in several widely used networks with even smaller patches. We also find how to effectively disguise as a target class to mislead the DNN classifiers and improve AI security.
A topic modeling and image classification framework: The Generalized Dirichlet variational autoencoder

Latent Dirichlet allocation model (LDA) has been widely used in topic modeling. Recent works have shown the effectiveness of integrating neural network mechanisms with this generative model for learning text representation. However, one of the significant setbacks of LDA is that it is based on a Dirichlet prior that has a restrictive covariance structure. All its variables are considered to be negatively correlated, which makes the model restrictive. In a practical sense, topics can be positively or negatively correlated. To address this problem, we proposed a generalized Dirichlet variational autoencoder (GD-VAE) for topic modeling. The Generalized Dirichlet (GD) distribution has a more general covariance structure than the Dirichlet distribution because it takes into account both positively and negatively correlated topics in the corpus. Our proposed model leverages rejection sampling variational inference using a reparameterization trick for effective training. GD-VAE compares favorably to recent works on topic models on several benchmark corpora. Experiments show that accounting for topics' positive and negative correlations results in better performance. We further validate the superiority of our proposed framework on two image data sets. GD-VAE demonstrates its significance as an integral part of a classification architecture. For reproducibility and further research purposes, code for this work can be found at https://github.com/hormone03/GD-VAE.
Contrastive clustering with a graph consistency constraint

Compared with classical contrastive learning methods, the performance of contrastive clustering is more easily affected by the quality of positive and negative samples, due to the fact that the clustering assumption requires neighbors of points as their positives. In order to reduce the effect of the uncertainty of positives and negatives on contrastive clustering, we propose a new contrastive clustering algorithm with graph consistency constraint. In this algorithm, a loss of graph consistency is proposed to reduce the impact of false negative samples by comparing the neighbor distribution of positive samples. Furthermore, an incremental training method is designed to control the quality of the selected positives. Extensive experiments show that our algorithm outperforms other deep clustering methods on wide-used benchmark data sets.
MinJoT: Multimodal infusion Joint Training for noise learning in text and multimodal classification problems

Amidst the critical role that high-quality labeled data plays in advancing machine learning, the persistence of noise within widely-used datasets remains a challenge. While noise learning has gained traction within machine learning, particularly in computer vision, its exploration in text and multimodal classification domains has lagged. Furthermore, a comprehensive comparison of noise learning techniques in text and multimodal classification has been lacking, partly due to variations in experimental noise settings across prior studies. Addressing these gaps, this research introduces a pioneering Multimodal Infusion Joint Training (MinJoT) framework featuring a novel co-regularized loss function that seamlessly integrates multimodal information during joint training. This framework notably excels in maintaining model robustness amidst noisy data environments. Adapting established noise learning methods from computer vision to text classification, the study conducts extensive experiments across five English and Chinese textual and multimodal datasets, involving four methods, five noise modes, and seven noise rates. Critically, this work challenges the implicit assumption that widely-used datasets are devoid of noise, revealing that these datasets indeed encompass noise levels ranging from 0.61% to 15.77% which is defined as intrinsic noise in this study. For the first time, the study investigates the impact of intrinsic noise on model performance, categorizing it into distinct levels of ambiguity. To facilitate accurate method comparison, a new dataset, Golden-Chnsenticorp (G-Chnsenticorp), is introduced, carefully crafted to be free of intrinsic noise. This research establishes the inaugural noise learning benchmark for text classification, while simultaneously pioneering the first noise learning framework tailored for multimodal sentiment classification. Through these contributions, the study advances the understanding of noise learning in text and multimodal contexts, providing a novel framework, benchmarks, and insights to propel the field forward.
A new fast and efficient dehazing and defogging algorithm for single remote sensing images

Information about the earth's surface is difficult to capture in remote sensing images because bad weather greatly curtails visibility and diminishes visual contrast in the images. For the purpose of military survey and aerial surveillance, these images are crucial for providing information. It is quintessential to eradicate bad weather conditions like haze and fog from remote sensing images. Recently, numerous deweathering initiatives and endeavors have been undertaken to alleviate these limitations. These contemporary deweathering approaches, however, are inadequate to recover dense haze images. This work presented a novel visibility restoration approach based on segmentation and unsharp mask guided filtering method. It consists of the following steps to restore the scene's radiance: First, a segmentation method is employed to determine atmospheric light to quantify contrast and color. Estimation of the transmission map utilizing the dark channel prior is then performed to precisely determine the proximity between objects. The resultant output image contains halo artifacts and inconsistencies in the structure. A guided filter method based on unsharp masking is employed to optimize the transmission map in order to solve this issue. The experimental results demonstrate that the proposed indicators ensures high uniformity regarding qualitative and quantitative evaluation using six performance metrics: mean square error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), universal image quality index (UQI), fog-aware density evaluator (FADE), dehazing algorithm index (DHQI), standard deviation (CC), and blind contrast enhancement assessment (e and r).
Segmentation and thickness calculation of glomerular basement membrane using RADS-Net in glomerular microscopic images

Glomerular ultrastructure needs to be observed under transmission electron microscopy during pathological diagnosis of renal biopsies. Evaluating the glomerular basement membranes (GBM) is necessary for the renal pathological diagnosis. There is little work that performs well for full-flow automated analysis of GBM in whole-glomerular electron microscopy images, including GBM segmentation, location, and thickness calculation. To address this problem, we propose a network architecture, RADS-Net, whose segmentation module combines the advantages of ViT and CNN to achieve better performance in GBM and electron dense deposit (EDD) segmentation tasks. It also contains an uncertainty evaluation module and a location module to exclude unsuitable regions for measurement and extract accurate GBM contours. We propose a GBM thickness calculation method based on statistical learning method, skeletonization algorithm, and improved region growth method. Systematic experiments demonstrate that our proposed segmentation network achieves superior accuracy compared to other competing methods. Additionally, the computed GBM thickness, after an automatic screening of suitable fields of view, meets the requirements for pathology evaluation.
Prototype-oriented contrastive learning for semi-supervised medical image segmentation

Semi-supervised learning has been recently explored to advance medical image segmentation due to the challenges of acquiring sufficient labeled data. However, the mainstream semi-supervised learning methods leverage unlabeled data by enforcing the perturbation-based prediction consistency, which neglects the semantic relations across different images and cannot handle inductive bias problem about the feature distribution. To address the above problems, in this paper, a novel prototype-oriented contrastive learning framework is proposed for semi-supervised medical image segmentation. Specifically, inspired from prototypical learning, a set of prototypes is extracted first to represent the diverse feature distributions of different images. Then we present semi-supervised contrastive learning with a prototype-oriented sampling strategy, encouraging the network to explore voxel-wise semantic relations across different images and learn more discriminative features to enhance the segmentation ability. The effectiveness of the proposed method is illustrated on three public benchmark datasets. Extensive experiments show that the proposed method outperforms several state-of-the-art semi-supervised segmentation methods, demonstrating its effectiveness for the challenging semi-supervised medical image segmentation task.
GAN inversion-based semi-supervised learning for medical image segmentation

The crux of medical image segmentation stems from learning pixel-wise semantic consistency from a mount of labeled samples through exhaustive annotation. Most existing methods rely heavily on labeled image pairs and are optimized in a fully supervised manner, often resulting in unsatisfactory results due to limited paired training samples. In this paper, we propose a GAN inversion-based semi-supervised learning framework InvSSL for medical image segmentation, which generates corresponding variants from labeled images by GAN inversion. Specifically, it first trains a StyleGAN as an inversion generator which is able to generate samples with the same distribution as training samples, and then obtains variant samples from a training sample by performing GAN inversion to strengthen the segmentation performance in a semi-supervised learning manner. In particular, inversion generator inverts a training sample into the latent space and adds interference to the latent codes to evolve variant samples. To model high-level consistency between pixels and decouple semantic components in the latent space, our InvSSL designs a multi-level dense contrastive learning mechanism. Extensive experiments on lung segmentation and skin lesion segmentation demonstrate that our InvSSL outperforms state-of-the-art methods for medical image segmentation. Our code will be released at https://github.com/funkdub/InvSSL.
Reciprocal cross-modal guidance for liver lesion segmentation from multiple phases under incomplete overlap

Liver lesion segmentation is beneficial for computer-aided disease diagnosis, which can be further enhanced by incorporating multi-phase images. Nevertheless, existing segmentation models still need improvement as they build on the unsolid assumption that lesions are well-aligned between different phases. Our goal was to develop a robust model that achieves high accuracy in lesion segmentation, even in cases where incomplete overlap occurs across multiple phases. To overcome the strong assumption of precise alignment between phases, we introduce a novel deep neural network featuring multi-scale feature fusion and cross-phase feature guidance. The former enables the retrieval of effective information from each phase that is helpful in lesion segmentation, while the latter ensures exchanging of guidance maps for interphase segmentation. Comprehensive experiments were carried out on an in-house multi-phase dataset having liver lesions, as well as a multi-modal dataset from the CHAOS challenge. Results show that our model surpasses existing state-of-the-art methods in segmenting liver lesions and abdominal organs from multi-phase images, even if the registration between phases is inaccurate. A competent solution was proposed for accurate liver lesion segmentation from multi-phase images, considering the imperfect registration between phases. The model was proved to be effective in extracting pertinent information from each phase and sharing useful information between phases. The accommodation of incomplete overlap in multi-phase images provides a new paradigm of accurate and robust lesion segmentation, which is particularly helpful in real-world scenarios where registration accuracy may be compromised.
Two-stage fine-grained image classification model based on multi-granularity feature fusion

Fine-grained visual classification (FGVC) is a difficult task due to the challenges of discriminative feature learning. Most existing methods directly use the final output of the network which always contains the global feature with high-level semantic information. However, the differences between fine-grained images are reflected in subtle local regions which often appear in the front of the network. When the texture of the background and object are similar or the proportion of the background is too large, the prediction will be greatly affected. In order to solve the above problems, this paper proposes multi-granularity feature fusion module (MGFF) and two-stage classification based on Vision-Transformer (ViT). The former comprehensively represents images by fusing features of different granularities, thus avoiding the limitations of single-scale features. The latter leverages the ViT model to separate the object from the background at a very small cost, thereby improving the accuracy of the prediction. We conduct comprehensive experiments and achieves the best performance in two fine-grained tasks on CUB-200-2011 and NA-Birds.
Robust Multitarget Tracking in Interference Environments: A Message-Passing Approach

Multitarget tracking in the interference environments suffers from the nonuniform, unknown and time-varying clutter, resulting in dramatic performance deterioration. We address this challenge by proposing a robust multitarget tracking algorithm, which estimates the states of clutter and targets simultaneously by the message-passing (MP) approach. We define the non-homogeneous clutter with a finite mixture model containing a uniform component and multiple nonuniform components. The measured signal strength is utilized to estimate the mean signal-to-noise ratio (SNR) of targets and the mean clutter-to-noise ratio (CNR) of clutter, which are then used as additional feature information of targets and clutter to improve the performance of discrimination of targets from clutter. We also present a hybrid data association which can reason over correspondence between targets, clutter, and measurements. Then, a unified MP algorithm is used to infer the marginal posterior probability distributions of targets, clutter, and data association by splitting the joint probability distribution into a mean-field approximate part and a belief propagation part. As a result, a closed-loop iterative optimization of the posterior probability distribution can be obtained, which can effectively deal with the coupling between target tracking, clutter estimation and data association. Simulation results demonstrate the performance superiority and robustness of the proposed multitarget tracking algorithm compared with the probability hypothesis density (PHD) filter and the cardinalized PHD (CPHD) filter.
Nonexact oracle inequalities, r-learnability, and fast rates

As an extension of the standard paradigm in statistical learning theory, we introduce the concept of r-learnability, 0<r≤1, which is a notion very closely related to that of nonexact oracle inequalities (see Lecue and Mendelson (2012) [7]). The r-learnability concept can enable so-called fast learning rates (along with corresponding sample complexity-type bounds) to be established at the cost of multiplying the approximation error term by an extra (1+r)-factor in the learning error estimate. We establish a new, general r-learning bound (nonexact oracle inequality) yielding fast learning rates in probability (up to at most a logarithmic factor) for proper learning in the general setting of an agnostic model, essentially only assuming a uniformly bounded squared loss function and a hypothesis class of finite VC-dimension (that is, finite pseudo-dimension).
DSCA-Net: Double-stage Codec Attention Network for automatic nuclear segmentation

The rapid and precise segmentation of cell nuclei from hematoxylin and eosin-stained tissue images is an essential clinical undertaking with significant implications for various clinical applications. The segmentation of cell nuclei poses specific challenges due to the inherent instability of nuclear morphology and the complexity of the segmentation environments. Furthermore, previous studies have primarily relied on small-scale and limited-diverse datasets, potentially hindering their applicability to clinical tasks. This study introduces a novel approach, the Double-stage Codec Attention Network, designed to automatically and accurately segment nuclei. Specifically, we present a hierarchical feature extraction module, which maximizes the utilization of cell nuclei's morphological characteristics in the tissue, thereby providing critical semantic information for nucleus segmentation. Furthermore, the feature selection units are employed to enhance relevant features and suppress interfering ones, thereby enhancing the overall expressive capacity of the information. The multi-scale deep feature fusion module utilizes interrelated encoder–decoder connections to jointly optimize and integrate this information, generating a robust hierarchical feature pyramid. Finally, the feature attention fusion mechanism captures spatial and directional information, aiding the model in the accurate localization and recognition of cell nuclei. We rigorously evaluated our proposed method using the PanNuke dataset, the largest comprehensive histology dataset of cancer tissues. In terms of the average F1-score across all segmentation classes in the PanNuke dataset, DSCA-Net outperforms other state-of-the-art models such as DeepLabV3+, TransUNet, Triple U-net, and TransNuSeg by 1.38, 1.44, 2.64, and 1.02, respectively. Additionally, DSCA-Net shows excellent efficiency in generating predictive images, outperforming all comparative models.
Phase Randomization: A data augmentation for domain adaptation in human action recognition

Human action recognition models often suffer from achieving both accurate recognition and subject independence when the amount of training data is limited. In this paper, we propose a data-efficient domain adaptation approach to learning a subject-agnostic action recognition classifier. The core component of our approach is a novel data augmentation called Phase Randomization. On the basis of the observation that individual body size is highly correlated with the amplitude component of the motion sequence, we disentangle the individuality and action features by using contrastive self-supervised learning with data augmentation that randomizes only the phase component of the motion sequence. This enables us to estimate the subject label of each motion sequence and to train a subject-agnostic action recognition classifier by performing adversarial learning with the estimated subject labels. We empirically demonstrate the superiority of our method on two different action recognition tasks (skeleton-based action recognition and sensor-based activity recognition).
Efficient search of comprehensively robust neural architectures via multi-fidelity evaluation

Neural architecture search (NAS) has emerged as one successful technique to find robust deep neural network (DNN) architectures. However, most existing robustness evaluations in NAS only consider l∞ norm-based adversarial noises. In order to improve the robustness of DNN models against multiple types of noises, it is necessary to consider a comprehensive evaluation in NAS for robust architectures. But with the increasing number of types of robustness evaluations, it also becomes more time-consuming to find comprehensively robust architectures. To alleviate this problem, we propose a novel efficient search of comprehensively robust neural architectures via multi-fidelity evaluation (ES-CRNA-ME). Specifically, we first search for comprehensively robust architectures under multiple types of evaluations using the weight-sharing-based NAS method, including different lp norm attacks, semantic adversarial attacks, and composite adversarial attacks. In addition, we reduce the number of robustness evaluations by the correlation analysis, which can incorporate similar evaluations and decrease the evaluation cost. Finally, we propose a multi-fidelity online surrogate during optimization to further decrease the search cost. On the basis of the surrogate constructed by low-fidelity data, the online high-fidelity data is utilized to finetune the surrogate. Experiments on CIFAR10 and CIFAR100 datasets show the effectiveness of our proposed method.