Reviewerly/exHarmony huggingface dataset:
KDED: A Knowledge Distillation Based Edge Detector

Deep learning-based edge detectors are successful due to the large amount of supervisory information provided by manual labeling. However, there are inevitably errors in the manually labeled supervisory information (MLSI), which greatly mislead the learning of the models and become the bottleneck of deep learning-based edge detectors. To solve the drawbacks of MLSI, we propose a novel Knowledge Distillation based Edge Detector (KDED). By means of knowledge distillation, MLSI is transformed into edge probability map to supervise the learning of the models, which can effectively correct the errors in MLSI and represents disputed edges by probability. Adapting to the new training strategy and solving the sample imbalance problem, the Sample Balance Loss is proposed, which ensures the stability of the model and improve the accuracy. The experimental results indicate that KDED remarkably improves the accuracy without increasing the parameters and the computational cost. KDED achieves an ODS F-measure of 0.832 with 14.8 M parameters on BSDS500 dataset, which is significantly super to the results of previous methods. The source code is available at this link .
Attention Auxiliary Supervision for Continuous Sign Language Recognition

Continuous Sign Language Recognition (CSLR) is a challenging task in the field of action recognition. It requires splitting a video into an indefinite number of glosses, which belong to different classes. Nowadays, researchers usually use deep learning methods with end-to-end training. One popular CSLR model paradigm is a three-step network, i.e., using a visual module to extract 2D frame features and short-term sequential features, then using a sequential module to analyze contextual associations, and finally Connectionist Temporal Classification (CTC) loss is used to constrain the output. Gloss alignment ability is found to be an important factor affecting CSLR model performance. However, the three-step CSLR paradigm mainly depends on the sequential module to align gloss, the visual module only focuses on local information and contributes little to module alignment ability, leading to training inconsistent between these two modules. This paper proposes an Attention Auxiliary Supervision (AAS) method to optimize the parameter of visual module and help it pay more attention to global information, thereby improving the alignment ability of the whole model. As an external part of the main model, the proposed AAS method has flexible applicability and is expected to be used in other CSLR models without increasing the cost of inference. The model performs well on two largescale CSLR datasets, i.e., PHOENIX14 (21.1% Test) and PHOENIX14-T (20.9% Test), which demonstrates its competitiveness among state-of-the-art models.
Reduced-Resolution Head for Object Detection

Utilizing a multi-scale strategy to address the challenge of significant variations in object sizes is a common approach in object detection. However, this strategy often results in significant computational overhead on high-resolution feature maps, leading to increased model complexity. This paper proposes a novel Reduced-resolution Head for Object Detection (RHOD). It can be seamlessly incorporated into most existing popular detectors, operating in a plug-and-play way. By replacing high-resolution feature maps with lower-resolution ones for regression and classification, our RHOD effectively eliminates the computational costs on high-resolution feature maps. Combined with the proposed Feature Fusion (FF), the final detection accuracy of the model can approach the accuracy of the original detector. Experimental results demonstrate that RHOD reduces the computational cost of the detector by approximately 15%–40% without sacrificing accuracy. This reduction in computational cost leads to a significant improvement in inference speed, with an increase of 1.2–5.0 frames per second (FPS). The code and models will be available at https://github.com/alpc128/RHOD .
PMT-IQA: Progressive Multi-task Learning for Blind Image Quality Assessment

Blind image quality assessment (BIQA) remains challenging due to the diverse types of distortion and variable image content, which complicates the distortion patterns crossing different scales and aggravates the difficulty of the regression problem for BIQA. However, existing BIQA methods often fail to consider multi-scale distortion patterns and image content, and there has limited research on improving the performance of quality regression models through specific learning strategies. In this paper, we propose a simple yet effective Progressive Multi-Task Image Quality Assessment (PMT-IQA) model, which contains a multi-scale feature extraction module (MS) and a progressive multi-task learning module (PMT), to help the model learn complex distortion patterns and better optimize the regression issue to align with the law of human learning process from easy to hard. To verify the effectiveness of the proposed PMT-IQA model, we conduct experiments on four widely used public datasets, and the experimental results indicate that the performance of PMT-IQA is superior to the comparison approaches, and both MS and PMT modules improve the model’s performance. The source code for this study is available at https://github.com/pqy000/PMT-IQA .
Byzantine-Robust Federated Learning via Server-Side Mixtue of Experts

Byzantine-robust Federated Learning focuses on mitigating the impact of malicious clients by developing robust algorithms that ensure reliable model updates while preserving privacy. The key insight of the state-of-the-art approaches entails statistical analysis on the local model updates uploaded by clients, concurrently eliminating any malicious updates prior to their aggregation. Some of these methods also require the assistance of server-side data for a reliable root of trust. However, these methods may not perform well and can even disrupt the normal process when the amount of data on the server-side is limited or the structure of the model is complex. We address this challenge by introducing FLSMoE, a novel Byzantine-robust Federated Learning approach that utilizes a Server-side Mixture of Experts. Our approach introduces a novel methodology by implementing a server-side Mixture of Experts(MoE) model, where the model parameters uploaded by individual clients are considered as expert models. Through the utilization of the gating unit within the MoE, even with low server-side data requirement, we are able to effectively identify and exclude malicious clients by assigning appropriate weights to their contributions. Empirically, we show through an extensive experimental evaluation that FLSMoE with low server-side data requirement can effectively mitigate the threat of malicious clients while also exhibiting greater Byzantine-robustness compared to previous Byzantine-robust Federated Learning approaches.
STN-BA: Weakly-Supervised Few-Shot Temporal Action Localization

Existing Weakly-supervised Few-Shot Temporal Action Localization (WFTAL) methods often process feature snippets with limited information, resulting in prediction errors and poor localization performance. A novel model called Spatial-Temporal Attention Network with Boundary-check Algorithm (STN-BA) for WFTAL is proposed to address this issue. STN-BA enhances the discriminability of snippet features and has a particular fault tolerance mechanism. The proposed approach focuses on two main aspects: (1) a spatial-temporal attention module that establishes spatial-temporal relationships of action movement to enrich the feature information of each video snippet and (2) the implementation of a boundary-check algorithm to correct potential localization boundary errors. The network is trained to estimate Temporal Class Similarity Vectors (TCSVs) that measure the similarity between each snippet of untrimmed videos and reference samples. These TCSVs are then normalized and employed as a temporal attention mask to extract the video-level representation from untrimmed videos, enabling accurate action localization during testing. Experimental evaluations of the widely used THUMOS14 and ActivityNet1.2 datasets demonstrate that the proposed method outperforms state-of-the-art fully-supervised and weakly-supervised few-shot learning methods.
Low-Frequency Aware Unsupervised Detection of Dark Jargon Phrases on Social Platforms

With the development of the Internet, the number of people communicating on social platforms has soared, which means that it is crucial for platform moderators to review and remove illegal content to create a clean network environment for users. However, identifying such content becomes complex due to the use of dark jargons. These jargons are seemingly innocent or newly coined words and phrases, such as “coke” for cocaine or “vanilla sky” for synthetic cathinone, to convey illegal meanings, aiming to evade detection by moderators. Existing methods primarily focus on detecting dark jargons at the word level, yielding commendable results. However, given the prevalence of phrase-level dark jargons in the context, relying solely on word-level detection can introduce ambiguity. For example, “black” is not a dark jargon, but “black bart” is a dark jargon. As a result, there is a growing interest in developing techniques specifically targeting phrase-level dark jargon detection. Unfortunately, such efforts are relatively limited, potentially resulting in the oversight of numerous low-frequency dark jargon phrases. To tackle this challenge, we propose the Low-Frequency Aware Dark Jargon Phrases Detection (DJPD) model. Our approach centers around finding a noun phrasal attention map pattern based on Transformer that enhances the perception of low-frequency phrases, enabling the selection of candidate dark jargon phrases. Subsequently, the candidate dark jargon phrases’ sentence-level context is analyzed to detect dark jargon phrases. Remarkably, our model achieves a significant 84.66% improvement in F1-score compared to the current state-of-the-art method for dark jargon phrase detection in the corpus.
Shareable and Inheritable Incremental Compilation in iOOBN

Object-oriented Bayesian networks (OOBNs) allow modellers to construct compositional and hierarchical models, using an inheritance hierarchy of classes ad subclasses, enabling reuse and supporting maintenance. Reasoning with both ordinary Bayesian networks (BNs) and OOBNs requires the important computational task of inference, the computing of new posterior probability distributions given a set of evidence. A widely used inference technique in ordinary BNs involves compiling the BN into a so-called junction tree (JT) before performing the inference; the compilation step is only performed when the model changes. In current OOBN software, the OOBN is first transformed into the underlying BN, so-called flattening, then the standard inference is performed. Researchers have proposed methods for incremental compilation of BNs, rather than recompiling from scratch for each network modification; these can apply to OOBNs also after flattening. Here, we propose a new incremental compilation technique that reuses existing compiled JTs of both embedded components and superclasses, and does not require flattening. We demonstrate through experimental analysis that this can reduce compilation time, and produces compact JTs that are cost-effective for inference.
Frequency Domain Feature Learning with Wavelet Transform for Image Translation

Image-to-image translation serves as an essential field of research in computer vision. Existing models frequently cause accidental distortion over non-target attributes, leading to overfitting of the generated image to the reference domain and poor visual quality. To address this problem, we propose Frequency Domain Feature Learning with Wavelet Transform, namely FDFL-WT, which with better non-target attributes retention and more precise image capture of style. This method utilizes the wavelet transform to capture the image’s approximation coefficients and diagonal coefficients, then we suggest wavelet reconstruction loss and wavelet translation loss. The former comprehensively records the context information of the source image to make the generated image realistic, whereas the latter improves the generator’s capacity to decouple attributes by assisting the model in efficiently retaining image content attributes. Experimental results on CelebA-HQ dataset indicate that FDFL-WT achieves about a 7.03 $$\%$$ performance improvement comparing with methods in the FID score of realism and disentanglement tests.
Unified Counterfactual Explanation Framework for Black-Box Models

Despite large-scale deployment in industry and daily life scenarios, the black-box nature of Connectionism-based deep neural networks is still criticized. Counterfactual explanation can shed light on the inner mechanism of arbitrary deep-learning model, thus being a preferable local interpretation method. There are a variety of methods for counterfactual generation, however, exist two defects: (1) Disunity. There is no agreement on model architecture and optimization methods of counterfactual generation. (2) Neglect of desiderata. There exist several desiderata for a good counterfactual sample, but most existing works only include a few of them. To address the above problem, we propose UNICE, a unified framework for counterfactual generations. UNICE models the counterfactual generation as a multi-task optimization problem on a dense data manifold learn by auto-encoder. Besides, UNICE addresses counterfactual desiderata to the best of our knowledge. What’s more, one can custom UNICE components regarding specific tasks and data modalities. An UNICE implementation for tabular data is provided and surpasses state-of-the-art methods in five of six metrics, indicating the effectiveness of our proposed method.
CoalUMLP: Slice and Dice! A Fast, MLP-Like 3D Medical Image Segmentation Network

3D medical image segmentation tasks play a crucial role in clinical diagnosis. However, Handling vast data and intricate structures in Point-of-Care (POC) devices is challenging. While current methods use CNNs and Transformer models, their high computational demands and limited real-time capabilities limit their POC application. Recent studies have explored the application of Multilayer Perceptrons (MLP) to medical image segmentation tasks. However, these studies overlook the significance of local and global image features and multi-scale contextual information. To overcome these limitations, we propose CoalUMLP, an efficient vision MLP architecture designed specifically for 3D medical image segmentation tasks. CoalUMLP combines the strengths of CNN, Transformer, and MLP, incorporating three key components: the Multi-Scale Axial Permute Encoder (MSAP), Masked Axial Permute Decoder (MAP), and Semantic Bridging Connection (SBC). We reframe the medical image segmentation problem as a sequence-to-sequence prediction problem and evaluate the performance of our approach on the Medical Segmentation Decathlon (MSD) dataset. CoalUMLP showcases a state-of-the-art performance by significantly reducing the parameter count by 32.8% and computational complexity by 48.5%, all while maintaining a compact structure. Our results highlight the potential of CoalUMLP as a promising backbone for real-time medical image applications. It achieves a superior trade-off between accuracy and efficiency compared to previous Transformer and CNN-based models.
GHGA-Net: Global Heterogeneous Graph Attention Network for Chinese Short Text Classification

As an important research content in the field of natural language processing, Chinese short text classification task has been facing two challenges: (i) existing methods rely on Chinese word segmentation and have insufficient semantic understanding of short texts; (ii) there is lacking of annotated training data in practical applications. In this paper, we propose the Global Heterogeneous Graph Attention Network (GHGA-Net) for few-shot Chinese short text classification. First, we construct the global character and keyword graph representations from the entire original corpus to collect more text information and make full use of the unlabeled data. Then, the hierarchical graph attention network is used to learn the contribution of different graph nodes and reduce the noise interference. Finally, we concatenate embedding with text vector and fuse the keyword and character features to enrich the Chinese semantics. Our method is evaluated on the Chinese few-shot learning benchmark FewCLUE. Extensive experiments show that our method has achieved impressive results in the classification tasks of news text and sentiment analysis, especially in minimal sample learning. Compared with existing methods, our method has an average performance improvement of 5% and less training consumption, which provides a new idea for few-shot Chinese natural language processing without relying on pre-training.
Abbreviated Weighted Graph in Multi-Agent Reinforcement Learning

The cooperation phenomenon in real life can often be viewed as an interaction between multiple agents. The exchange of unstructured information between agents is usually abstracted as the transmission of node information in a graph. Various approaches have been proposed to build graph models and fuse the information of agents in their observation range. However, it is inefficient to directly apply the related algorithms of graph neural networks to integrate the information of each agent, which can not effectively use the observation features of the dominant agent in cooperation, and it may also be overly focused on the weight of the edges. In this paper, we propose AWGmix, which is an abbreviated weighted graph information-enhanced algorithm for multi-agent reinforcement learning. Specifically, we propose a simple and convenient method to calculate the weight of edges between graph nodes modeled from agent connections, and design an attribution module based on attention mechanism to find the dominant agents and enrich the representation of observations. Experimental results demonstrate the superiority and effectiveness of our proposed method on Starcraft II micromanagement benchmark tasks.
Context-Dependent Text-to-SQL Generation with Intermediate Representation

In recent years, the Text-to-SQL task has become a research hotspot in semantic analysis. Among them, context-dependent Text-to-SQL task has received more and more attention as it meets the needs of actual scenarios. The core of the problem is how to use historical interaction information and database schema to understand the context. Most existing research ignores the structure of SQL queries and introduces low-level information such as variable names and parameters, and the mismatch problem between intents expressed in utterance and the implementation details in SQL still exists. In this paper, SemQL is applied to serve as an intermediate representation between utterance and SQL, meanwhile, the Coarse-to-Fine neural architecture is adopted to decompose decoding process of SemQL into two stages. We validated the performance of our model on SParC and CoSQL datasets, which outperforms the existing ones and achieves excellent results on both datasets.
Relational Acceptability Semantics of Abstract Argumentation

We introduce argumentation tuple relational calculus, adapting tuple relational calculus for acceptability semantics, and derive relational acceptability semantics of abstract argumentation. It serves as a theoretical framework to link different types of acceptability semantics relationally. It also allows for refining existing acceptability semantics.
MuHca: Mixup Heterogeneous Graphs for Contrastive Learning with Data Augmentation

Contrastive learning has become a highly promising learning paradigm and demonstrated significant potential when few labels are available. The effectiveness of contrastive learning on graphs is largely dependent on the quality of positive and negative pairs, which can be improved by developing data augmentation (DA). However, the majority of the current DA methods rely on homogeneous graphs while less on heterogeneous graphs. In this paper, we present a method named MuHca, a node augmentation module for the problem of heterogeneity in DA. Concretely, MuHca separately employs nodes embedding of two views, namely meta-path and network schema, into a novel contrasting generative adversarial nets structure to implement data augmentation. By adopting the contrasting generative paradigm of GANs, MuHca can generate and optimize effective negatives. To enhance the robustness of MuHca, we exploit the potential information from the original data and extend our approaches by mixing up generated negatives with original ones. The final stage involves training a generator for edges in parallel with the modeling of edge presence among nodes, culminating in contrastive learning for heterogeneous graphs. The conducted experiments on three datasets validate the effectiveness of our proposed method, surpassing various state-of-the-art and even semi-supervised methods.
AdaptLight: Toward Cross-Space-Time Collaboration for Adaptive Traffic Signal Control

Recent multi-agent deep reinforcement learning (MADRL) approaches have shown notable benefits in traffic signal control. However, the spatial-temporal coupling, hysteresis, and heterogeneity of collaborative agents are usually ignored. States and actions among multiple intersections induce complex coupling and hysteresis in both space and time dimensions, while the actions also present spatial-temporal heterogeneity due to fluctuated traffic. These characteristics impose a critical impact on the efficiency and flexibility of coordinated control. In this paper, we propose AdaptLight, an MADRL-based model to achieve cross-space-time collaboration. It captures the interactions among spatial-temporal traffic components and exploits action repetition to adaptively adjust decision granularity for heterogeneous traffic. For the spatial-temporal coupling and hysteresis issue, AdaptLight first establishes a feature extraction network based on spatial-temporal graph Transformer. To tackle the spatial-temporal action heterogeneity problem, an action-repetition-enabled MADRL module is designed, which can decide asynchronous-cooperative actions spanning multiple timesteps. Experiments present that AdaptLight shows competitive performance on different datasets.
Multi-modal Component Representation for Multi-source Domain Adaptation Method

Multi-source domain adaptation aims to leverage multiple labeled source domains to train a classifier for an unlabeled target domain. Existing methods address the domain discrepancy by learning the invariant representation. However, due to the large difference in image style, image occlusion and missing, etc., the invariant representation tends to be inadequate, and some components tend to be lost. To this end, a multi-source domain adaptation method with multi-modal representation for components is proposed. It learns the multi-modal representation for missing components from an external knowledge graph. First, the semantic representation of the class subgraph, including not only the class but also rich class components, is learned from knowledge graph. Second, the semantic representation is fused with the visual representations of each domain respectively. Finally, the multi-modal invariant representations of source and target domains are learned. Experiments show the effectiveness of our method.