Reviewerly/exHarmony huggingface dataset:
Contrastive Kernel Subspace Clustering

As a class of nonlinear subspace clustering methods, kernel subspace clustering has shown promising performance in many applications. This paper focuses on the kernel selection problem in the kernel subspace clustering model. Currently, the kernel function is typically chosen by the single kernel or multiple kernel methods. The former relies on a given kernel function, which poses challenges in clustering tasks with limited prior information, making it difficult to determine a suitable kernel function beforehand. Multiple kernel methods usually assume that the optimal kernel is near a series of predefined base kernels, which limits the expressive ability of the optimal kernel. Furthermore, multiple kernel methods tend to have higher solution complexity than single kernel methods. To address these limitations, this paper utilizes contrastive learning to learn the optimal kernel adaptively and proposes the Contrastive Kernel Subspace Clustering (CKSC) method. Unlike multiple kernel approaches, CKSC is not constrained by the multiple kernel assumption. Specifically, CKSC integrates a contrastive regularization into the kernel subspace clustering model, encouraging neighboring samples in the original space to stay nearby in the reproducing kernel Hilbert space (RKHS). In this way, the resulting kernel mapping can preserve the cluster structure of the data, which will benefit downstream clustering tasks. The clustering experiments on seven benchmark data sets validate the effectiveness of the proposed CKSC method.
SGLP-Net: Sparse Graph Label Propagation Network for Weakly-Supervised Temporal Action Localization

The present weakly-supervised methods for Temporal Action Localization are primarily responsible for capturing the temporal context. However, these approaches have limitations in capturing semantic context, resulting in the risk of ignoring snippets that are far apart but sharing the same action categories. To address this issue, we propose an action label propagation network utilizing sparse graph networks to effectively explore both temporal and semantic information in videos. The proposed SGLP-Net comprises two key components. One is the multi-scale temporal feature embedding module, a novel method that extracts both local and global temporal features of the videos during the initial stage using CNN and self-attention and serves as a generic module. The other is an action label propagation mechanism, which uses graph networks for feature aggregation and label propagation. To avoid the issue of excessive feature completeness, we optimize training using sparse graph convolutions. Extensive experiments are conducted on THUMOS14 and ActivityNet1.3 benchmarks, among which advanced results demonstrate the superiority of the proposed method. Code can be found at https://github.com/xyao-wu/SGLP-Net .
CSEC: A Chinese Semantic Error Correction Dataset for Written Correction

Existing research primarily focuses on spelling and grammatical errors in English, such as missing or wrongly adding characters. This kind of shallow error has been well-studied. Instead, there are many unsolved deep-level errors in real applications, especially in Chinese, among which semantic errors are one of them. Semantic errors are mainly caused by an inaccurate understanding of the meanings and usage of words. Few studies have investigated these errors. We thus focus on semantic error correction and propose a new dataset, called CSEC, which includes 17,116 sentences and six types of errors. Semantic errors are often found according to the dependency relations of sentences. We thus propose a novel method called Desket (Dependency Syntax Knowledge Enhanced Transformer). Desket solves the CSEC task by (1) capturing the syntax of the sentence, including dependency relations and part-of-speech tagging, and (2) using dependency to guide the generation of the correct output. Experiments on the CSEC dataset demonstrate the superior performance of our model against existing methods.
Robust Multi-view Spectral Clustering with Auto-encoder for Preserving Information

Multi-view clustering is a prominent research topic in machine learning that leverages consistent and complementary information from multiple views to improve clustering performance. The graph-based multi-view clustering methods learn consistent graph with pairwise similarity between data as edges, and generate sample representation using spectral clustering. However, most existing methods seldom consider to recreate the input data using encoded representation during representation learning procedure, which result in information loss. To address this limitation, we propose a robust multi-view clustering with auto-encoder for preserving information (RMVSC-AE) that minimizes the reconstruction error between the input data and the reconstructed representation to preserve knowledge. Specifically, we discover a graph representation by jointly optimizing the graph Laplacian and auto-encoder reconstruction terms. Moreover, we introduce a sparse noisy term to further enhance the quality of the learned consistent graph. Extensive experiments on six multi-view datasets are conducted to verify the efficacy of the proposed method.
AttIN: Paying More Attention to Neighborhood Information for Entity Typing in Knowledge Graphs

Entity types in knowledge graph (KG) have been employed extensively in downstream tasks of natural language processing (NLP). Currently, knowledge graph entity typing is usually inferred by embeddings, but a single embedding approach ignores interactions between neighbor entities and relations. In this paper, we propose an AttIN model that pays more attention to entity neighborhood information. More specifically, AttIN contains three independent inference modules, including a BERT module that uses the target entity neighbor to infer the entity type individually, a context transformer that aggregates information based on different contributions from the neighbor, and an interaction information aggregator (IIAgg) module that aggregates the entity neighborhood information into a long sequence. In addition, we use exponentially weighted pooling to process these predictions. Experiments on the FB15kET and YAGO43kET datasets show that AttIN outperforms existing competitive baselines while it does not need extra semantic information in the sparse knowledge graph.
Generalized Category Discovery with Clustering Assignment Consistency

Generalized category discovery (GCD) is an important open-world task to automatically cluster the unlabeled samples using information transferred from the labeled dataset, given a dataset consisting of labeled and unlabeled instances. A major challenge in GCD is that unlabeled novel class samples and unlabeled known class samples are mixed together in the unlabeled dataset. To conduct GCD without knowing the number of classes in unlabeled dataset, we propose a co-training-based framework that encourages clustering consistency. Specifically, we first introduce weak and strong augmentation transformations to generate two sufficiently different views for the same sample. Then, based on the co-training assumption, we propose a consistency representation learning strategy, which encourages consistency between feature-prototype similarity and clustering assignment. Finally, we use the discriminative embeddings learned from the semi-supervised representation learning process to construct an original sparse network and use a community detection method to obtain the clustering results and the number of categories simultaneously. Extensive experiments show that our method achieves state-of-the-art performance on three generic benchmarks and three fine-grained visual recognition datasets.
Learnable Color Image Zero-Watermarking Based on Feature Comparison

Zero-watermarking is one of the solutions to protect the copyright of color images without tampering with them. Existing zero-watermarking algorithms either rely on static classical techniques or employ pre-trained models of deep learning, which limit the adaptability of zero-watermarking to complex and dynamic environments. These algorithms are prone to fail when encountering novel or complex noise. To address this issue, we propose a self-supervised anti-noise learning color image zero-watermarking method that leverages feature matching to achieve lossless protection of images. In our method, we use a learnable feature extractor and a baseline feature extractor to compare the features extracted by both. Moreover, we introduce a combined weighted noise layer to enhance the robustness against combined noise attacks. Extensive experiments show that our method outperforms other methods in terms of effectiveness and efficiency.
Effi-Seg: Rethinking EfficientNet Architecture for Real-Time Semantic Segmentation

A popular strategy for designing a semantic segmentation model is to utilize a well-established pre-trained Deep Convolutional Neural Network (DCNN) as a feature extractor and replace the classification head with a decoder to generate segmented outputs. The advantage of this strategy is the ability to obtain a ready-made backbone with additional knowledge. However, there are several disadvantages, such as a lack of architectural knowledge, a significant semantic gap among the deep feature maps, and a lack of control over architectural changes to reduce memory overhead. To overcome these issues, we first study the complete architecture of EfficientNetV1 and EfficientNetV2, analyzing the architectural and performance gaps. Based on this analysis, we develop an efficient segmentation model called Effi-Seg by implementing several architectural changes to the backbone. This approach leads to better semantic segmentation results with improved efficiency. To enhance contextualization and achieve accurate object localization in the scene, we introduce a feature refinement module (FRM) and a semantic aggregation module (SAM) in the decoder. The complete segmentation network comprises only 1.49 million parameters and 8.4 GFLOPs. We evaluate the performance of the proposed model using three popular benchmarks, and it demonstrates highly competitive results on all three datasets while maintaining excellent efficiency.
Staged Long Text Generation with Progressive Task-Oriented Prompts

Generating coherent and consistent long text remains a challenge for artificial intelligence. The state-of-the-art paradigm partitions the whole generating process into successive stages, however, the content plan applied in each stage may be error-prone and fine tuning large-scale language models, one for each stage, is resource-consuming. In this paper, we follow the above paradigm and devise three stages: keyphrase decompression, transition paraphrase, and text generation. We leverage task-oriented prompts to direct the producing of text in each stage which improves the quality of the generated text. Further, we propose a new content plan representation with elastic mask tokens to reduce model bias and irregular words. Moreover, we introduce length control and commonsense knowledge prompts to increase the adaptability of the proposed model. Extensive experiments conducted on two challenging tasks demonstrated that our model outperforms strong baselines significantly, and it is able to generate longer high quality texts with fewer parameters.
Text to Image Generation with Conformer-GAN

Text-to-image generation (T2I) has been a popular research field in recent years, and its goal is to generate corresponding photorealistic images through natural language text descriptions. Existing T2I models are mostly based on generative adversarial networks, but it is still very challenging to guarantee the semantic consistency between a given textual description and generated natural images. To address this problem, we propose a concise and practical novel framework, Conformer-GAN. Specifically, we propose the Conformer block, consisting of the Convolutional Neural Network (CNN) and Transformer branches. The CNN branch is used to generate images conditionally from noise. The Transformer branch continuously focuses on the relevant words in natural language descriptions and fuses the sentence and word information to guide the CNN branch for image generation. Our approach can better merge global and local representations to improve the semantic consistency between textual information and synthetic images. Importantly, our Conformer-GAN can generate natural and realistic 512 $$\times $$ 512 images. Extensive experiments on the challenging public benchmark datasets CUB bird and COCO demonstrate that our method outperforms recent state-of-the-art methods both in terms of generated image quality and text-image semantic consistency.
CKR-Calibrator: Convolution Kernel Robustness Evaluation and Calibration

Recently, Convolution Neural Networks (CNN) have achieved excellent performance in some areas of computer vision, including face recognition, character recognition, and autonomous driving. However, there are still many CNN-based models that cannot be deployed in real-world scenarios due to poor robustness. In this paper, focusing on the classification task, we attempt to evaluate and optimize the robustness of CNN-based models from a new perspective: the convolution kernel. Inspired by the discovery that the root cause of the model decision error lies in the wrong response of the convolution kernel, we propose a convolution kernel robustness evaluation metric based on the distribution of convolution kernel responses. Then, we devise the Convolution Kernel Robustness Calibrator, termed as CKR-Calibrator, to optimize key but not robust convolution kernels. Extensive experiments demonstrate that CKR-Calibrator improves the accuracy of existing CNN classifiers by 1%–4% in clean datasets and 1%–5% in corrupt datasets, and improves the accuracy by about 2% over SOTA methods. The evaluation and calibration source code is open-sourced at https://github.com/cym-heu/CKR-Calibrator .
Spatially-Aware Human-Object Interaction Detection with Cross-Modal Enhancement

We propose a novel two-stage HOI detection model that incorporates cross-modal spatial information awareness. Human-object relative spatial relationships are highly relevant for specific HOI species, but current approaches fail to model such crucial cues explicitly. We observed that relative spatial relationships possess properties that can be described in natural language easily and intuitively. Building on this observation and inspired by recent advancements in prompt-tuning, we design a Prompt-Enhanced Spatial Modeling (PESM) module that generates linguistic descriptions of spatial relations between humans and objects. PESM is capable of merging the explicit spatial information obtained by the aforementioned text descriptions with the implicit spatial information of the visual modality. Moreover, we devise a two-stage model architecture that effectively incorporates auxiliary cues to exploit the enhanced cross-modal spatial information. Extensive experiments conducted on the HICO-DET benchmark demonstrate that the proposed model outperforms state-of-the-art methods, indicating its effectiveness and superiority. The source code is available at https://github.com/liugaowen043/tsce .
Isomorphic Dual-Branch Network for Non-homogeneous Image Dehazing and Super-Resolution

Removing non-homogeneous haze from real-world images is a challenging task. Meanwhile, the popularity of high-definition imaging systems and compute-limited smart mobile devices has resulted in new problems, such as the high computational load caused by haze removal for large-size images, or the severe information loss caused by the degradation of both the haze and image downsampling, when applying existing dehazing methods. To address these issues, we propose an isomorphic dual-branch dehazing and super-resolution network for non-homogeneous dehazing of a downsampled hazy image, which produces dehazed and enlarged images with sharp edges and high color fidelity. We quantitatively and qualitatively compare our network with several state-of-the-art dehazing methods under the condition of different downsampling scales. Extensive experimental results demonstrate that our method achieves superior performance in terms of both the quality of output images and the computational load.
Time Series Anomaly Detection with a Transformer Residual Autoencoder-Decoder

Time series anomaly detection is of great importance in a variety of domains such as finance fraud, industrial production, and information systems. However, due to the complexity and multiple periodicity of time series, extracting global and local information from different perspectives remains a challenge. In this paper, we propose a novel Transformer Residual Autoencoder-Decoder Model called $${\textbf {TRAD}}$$ for time series anomaly detection, which is based on a multi-interval sampling strategy incorporating with residual learning and stacked autoencoder-decoder to promote the ability to learn global and local information. Prediction error is applied to calculate anomaly scores using the proposed model from different scales, and the aggregated anomaly scores are utilized to infer outliers of the time series. Extensive experiments are conducted on five datasets and the results demonstrate that the proposed model outperforms the previous state-of-the-art baselines.
Adversarial Example Detection with Latent Representation Dynamic Prototype

In the realm of Deep Neural Networks (DNNs), one of the primary concerns is their vulnerability in adversarial environments, whereby malicious attackers can easily manipulate them. As such, identifying adversarial samples is crucial to safeguarding the security of DNNs in real-world scenarios. In this work, we propose a method of adversarial example detection. Our approach using a Latent Representation Dynamic Prototype to sample more generalizable latent representations from a learnable Gaussian distribution, which relaxes the detection dependency on the nearest neighbour's latent representation. Additionally, we introduce Random Homogeneous Sampling (RHS) to replace KNN sampling reference samples, resulting in lower reasoning time complexity at O(1). Lastly, we use cross-attention in the adversarial discriminator to capture the evolutionary differences of latent representation in benign and adversarial samples by comparing the latent representations from inference and reference samples globally. We conducted experiments to evaluate our approach and found that it performs competitively in the gray-box setting against various attacks with two $$\mathcal {L}_p$$ -norm constraints for CIFAR-10 and SVHN datasets. Moreover, our detector trained with PGD attack exhibited detection ability for unseen adversarial samples generated by other adversarial attacks with small perturbations, ensuring its generalization ability in different scenarios.
Towards High-Performance Exploratory Data Analysis (EDA) via Stable Equilibrium Point

Exploratory data analysis (EDA) is a vital procedure in data science projects. In this work, we introduce a stable equilibrium point (SEP)-based framework for improving the performance of EDA. By exploiting the SEPs to be the representative points, our approach aims to generate high-quality clustering and data visualization for real-world data sets. A very unique property of the proposed method is that the SEPs will directly encode the clustering properties of data sets. Compared with prior state-of-the-art clustering and data visualization methods, the proposed methods allow substantially improving solution quality for large-scale data analysis tasks. For instance, for the USPS data set, our method achieves more than $$10\%$$ clustering accuracy gain over the standard spectral clustering algorithm and 3X speedup for the t-SNE visualization.
Hi-Stega: A Hierarchical Linguistic Steganography Framework Combining Retrieval and Generation

Due to the widespread use of social media, linguistic steganography which embeds secret message into normal text to protect the security and privacy of secret message, has been widely studied and applied. However, existing linguistic steganography methods ignore the correlation between social network texts, resulting in steganographic texts that are isolated units and prone to breakdowns in cognitive-imperceptibility. Moreover, the embedding capacity of text is also limited due to the fragmented nature of social network text. In this paper, in order to make the practical application of linguistic steganography in social network environment, we design a hierarchical linguistic steganography (Hi-Stega) framework. Combining the benefits of retrieval and generation steganography method, we divide the secret message into data information and control information by taking advantage of the fact that social network contexts are associative. The data information is obtained by retrieving the secret message in normal network text corpus and the control information is embedded in the process of comment or reply text generation. The experimental results demonstrate that the proposed approach achieves higher embedding payload while the imperceptibility and security can also be guaranteed. (All datasets and codes used in this paper are released at https://github.com/wanghl21/Hi-Stega .)
CATS: Connection-Aware and Interaction-Based Text Steganalysis in Social Networks

The generative linguistic steganography in social networks have potential huge abuse and regulatory risks, with serious implications for information security, especially in the era of large language models. Many works have explored detecting steganographic texts with progressively enhanced imperceptibility, but they can only achieve poor performance in real social network scenarios. One key reason is that these methods primarily focus on linguistic features, which are extremely insufficient owing to the fragmentation of social texts. In this paper, we propose a novel method called CATS (Connection-aware and interAction-based Text Steganalysis) to effectively detected the potentially malicious steganographic texts. CATS captures social networks connection information by graph representation learning, enhances linguistic features by contrastive learning and fully integrates features above via a novel features interaction module. Our experimental results demonstrate that CATS outperforms existing methods by exploiting social network graph structure features and interactions in social network environments.
PAG: Protecting Artworks from Personalizing Image Generative Models

Recent advances in conditional image generation have led to powerful personalized generation models that generate high-resolution artistic images based on simple text descriptions through tuning. However, the abuse of personalized generation models may also increase the risk of plagiarism and the misuse of artists' painting styles. In this paper, we propose a novel method called Protecting Artworks from Personalizing Image Generative Models framework (PAG) to safeguard artistic images from the malicious use of generative models. By injecting learned target perturbations into the original artistic images, we aim to disrupt the tuning process and introduce the distortions that protect the authenticity and integrity of the artist's style. Furthermore, human evaluations suggest that our PAG model offers a feasible and effective way to protect artworks, preventing the personalized generation models from generating similar images to the given artworks.