Reviewerly/exHarmony huggingface dataset:
PKGS: A Privacy-Preserving Hitchhiking Task Assignment Scheme for Spatial Crowdsourcing

Privacy-preserving task assignment is vital to assign a task to appropriate workers and protect workers’ privacy or task privacy for spatial crowdsourcing (SC). Existing solutions usually require each worker to travel to the task location on purpose to perform this task, which fails to consider that workers have specific trajectories and carry out the task on their way in a hitchhiking manner. To this end, this paper proposes a privacy-preserving hitchhiking task assignment scheme for SC, named PKGS. Specifically, we formulate the privacy-preserving hitchhiking task assignment as a decision problem of the relationship between dot and line under privacy protection. In particular, we present a privacy-preserving travel distance calculation protocol and a privacy-preserving comparison protocol through the Paillier cryptosystem and the SC framework. Results of theoretical analysis and experimental evaluation show that PKGS can not only protect the location privacy of both each worker and the task simultaneously but also assign the task to the worker holding a minimum travel distance. In contrast to prior solutions, PKGS outperforms in the computation of travel distance and task assignment.
Enhancing Discriminative Ability among Similar Classes with Guidance of Text-Image Correlation for Unsupervised Domain Adaptation

In deep learning, unsupervised domain adaptation (UDA) is commonly utilized when the availability of abundant labeled data is often limited. Several methods have been proposed for UDA to overcome the difficulty of distinguishing between semantically similar classes, such as person vs. rider and road vs. sidewalk. The confusion of the classes results from the collapse of the distance, caused by the domain shift, between classes in the feature space. In this work, we present a versatile approach based on text-image correlation-guided domain adaptation (TigDA), which maintains a distance to properly adjust the decision boundaries between classes in the feature space. In our approach, the feature information is extracted through text embedding of classes and the aligning capability of the text features with the image features is achieved using the cross-modality. The resultant cross-modal features play an essential role in generating pseudo-labels and calculating an auxiliary pixel-wise cross-entropy loss to assist the image encoder in learning the distribution of cross-modal features. Such a guiding process allows the extension of the distance between similar classes in feature space so that a proper distance for adjusting the decision boundary is maintained. Our TigDA achieved the highest performance among other UDA methods in both single-resolution and multi-resolution cases with the help of GTA5 and SYNTHIA for the source domain and Cityscapes for the target domain. The simplicity and versatility of TigDA will be widely applicable for enhancing the self-training capabilities of most UDA methods.
Augmented Spatial Context Fusion Network for Scene Graph Generation

Scene graph generation provides high-order semantic information by understanding the objects and their relations in images. In order to improve the performance of scene graph generation, context fusion has been widely used in scene graph generation tasks, LSTM and Vision-Transformer are commonly used fusion modules. Both LSTM and Vision-Transformer realize context fusion by stacking multiple basic units, which needs to learn a large number of parameters of the units. However, the model computational efficiency of scene graph generation as a mid-level semantic understanding task to support downstream tasks is crucial. To simplify the context fusion computation, this paper proposes ASCF -Net (Augmented Spatial Context Fusion Network) which computes the spatial context of designated object by searching the nearest neighbor objects with high relevance and strengthens the context with random noise. Without learning parameters, the above computational process essentially simulates the attention mechanism. Experiments on VG dataset show that ASCF -Net uses 15.26% of the parameters of Bi-LSTM and 13.34% of the parameters of Vision-Transformer for context fusion based on the same baseline and achieves higher performance than using the two fusion modules. At the same time, ASCF -Net uses simple fusion module to obtain competitive results on VG dataset compared with the mainstream scene generation models.
GAN Latent Space Manipulation Based Augmentation for Unbalanced Emotion Datasets

Learning with small or highly unbalanced datasets have long been an important area of research in computer vision. Extracting features from these datasets is more difficult and the training process of deep learning models could be hindered. In order to solve this problem many approaches have been proposed, among which data augmentation is widely used since it can balance the dataset by generating new samples for small classes. Current data augmentation methods like image manipulation can only generate samples with low diversity, generative model based methods can not guarantee the quality and distribution of generated images. This paper proposes an offline data augmentation method based on generative adversarial network (GAN) latent space manipulation to create new samples for small classes in unbalanced image datasets. This method helps fill the sparse data manifold of small classes with new samples and can guarantee data quantity, quality, diversity and distribution at the same time. More specifically, we choose the facial expression recognition (FER) task as an example where linear interpolation, combination and concatenation of StyleGAN latent codes are used to generate new samples. We further apply latent space vector arithmetic to perform inter-class translation for different facial expressions. The proposed methods are evaluated through extensive experiments on two different FER databases.
Convex Polygon Triangulation Based on Symmetry

A polygon with n nodes can be divided into two subpolygons by an internal diagonal through node n. Splitting the polygon along diagonal δi,n and diagonal δn−i,n, i∈{2,…,⌊n/2⌋} results in mirror images. Obviously, there are ⌊n/2⌋−1 pairs of these reflectively symmetrical images. The influence of the observed symmetry on polygon triangulation is studied. The central result of this research is the construction of an efficient algorithm used for generating convex polygon triangulations in minimal time and without generating repeat triangulations. The proposed algorithm uses the diagonal values of the Catalan triangle to avoid duplicate triangulations with negligible computational costs and provides significant speedups compared to known methods.
SWTA: Sparse Weighted Temporal Attention for Drone-Based Activity Recognition

Drone-camera based human activity recognition (HAR) has received significant attention from the computer vision research community in the past few years. A robust and efficient HAR system has a pivotal role in fields like video surveillance, crowd behavior analysis, sports analysis, and human-computer interaction. What makes it challenging are the complex poses, understanding different viewpoints, and the environmental scenarios where the action is taking place. To address such complexities, in this paper, we propose a novel Sparse Weighted Temporal Attention (SWTA) module to utilize sparsely sampled video frames for obtaining global weighted temporal attention. The proposed SWTA is divided into two components. First, temporal segment network that sparsely samples a given set of frames. Second, weighted temporal attention, which incorporates a fusion of attention maps derived from optical flow, with raw RGB images. This is followed by a basenet network, which comprises a convolutional neural network (CNN) module along with fully connected layers that provide us with activity recognition. The SWTA network can be used as a plug-in module to the existing deep CNN architectures, for optimizing them to learn temporal information by eliminating the need for a separate temporal stream. It has been evaluated on three publicly available benchmark datasets, namely Okutama, MOD20, and Drone-Action. The proposed model has received an accuracy of 72.76%, 92.56%, and 78.86% on the respective datasets thereby surpassing the previous state-of-the-art performances by a margin of 25.26%, 18.56%, and 2.94%, respectively.
Joint Node Representation Learning and Clustering for Attributed Graph via Graph Diffusion Convolution

In recent years, the representation learning method based on graph convolution network has made the latest achievements in attributed graph clustering. However, these methods only deal with clustering as a downstream task, and better performance can be achieved if clustering is combined with the node learning representation process. This paper proposes a novel method of graph clustering based on graph diffusion convolution network, which jointly conducts node representation learning and clustering. The graph diffusion strategy is applied on the node attributes to assign the near-by nodes with high weights in the step of feature propagation. The output nodes after diffusion were reconstructed by a linear encoder-decoder, and also could be represented by the clusters. The joint learning is achieved by minimizing both errors of reconstruction and representation. Experiments conducted on three public datasets and three real datasets from Zhengzhou Commodity Exchange demonstrate the effectiveness of the proposed method in the task of node clustering.
RTOD-YOLO: Traffic Object Detection in UAV Images Based on Visual Attention and Re-parameterization

Unmanned Aerial Vehicles (UAVs) are widely deployed in transportation. Detection of traffic objects like vehicles in UAV images is helpful for traffic monitoring, urban transport planning, and estimation of traffic flow. However, detecting objects in UAV images has some inevitable problems, such as large changes in object scale, occlusion between objects in dense detection scenes, and interference caused by complex backgrounds. To reduce the influence of the above factors and achieve real-time detection, we propose RTOD-YOLO based on YOLOv5s in this study. Firstly, a small object detection layer is added to improve the model's performance to detect small objects. Secondly, the Global Context Attention Module (GCAM) is introduced to enable RTOD-YOLO to concentrate more on the target itself and reduce the interference caused by complex backgrounds. Thirdly, the YOLO detection head is replaced by the Shift Double Prediction Head (SDPH) to improve the bounding box regression and classification ability of the proposed model. Finally, RepBlock and CSPRepBottleneck are introduced to re-parameterize the model structure, for improving feature extraction capabilities and adequately mining semantic information in images. The experimental result shows that the mAP@0.5 of RTOD-YOLO reaches 44.6% on the VisDrone dataset, which is 10.7% higher than that of the baseline YOLOv5s, while still accommodating real-time detection.
A Simple Semantics and Topic-aware Method to Enhance Abstractive Summarization

In recent years, Transformer-based abstractive summarization models have been proven to be effective and widely used in multiple domains. However, most of these Transformer-based methods are based on maximum likelihood estimation and they still focus on token-level optimization. In addition, the generated summary should have the same semantics as the gold summary. Therefore, we propose a new abstractive summarization model in this paper. Specifically, we optimize the traditional model from the semantic perspective, so that the semantics of the generated summary is more similar to that of the gold summary. We also add topic information so that more key information in the text can be retained in the model. We prove in the CNN/DailyMail dataset that the method proposed in this paper greatly improve the classic abstractive model based on BART, and achieve SOTA results in the SAMSUM dataset.
Enhancing source code summarization from structure and semantics

Source code summarization aims to generate concise and high-quality natural language descriptions for code snippets. High-quality code summaries can help developers better understand source codes. Researchers have made great efforts to generate more accurate summaries; however, due to the lack of preservation of source code structure and semantics, previous approaches have struggled to generate summaries that accurately describe the functionality or other major characteristics of codes. In this paper, we propose a novel approach called Code Structure and Semantic Fusion (CSSF) for automatically generating summaries for source code. CSSF can utilize both the structural and semantic information of source codes. To achieve this, we extract the overall structure of the Abstract Syntax Tree (AST) by expanding the AST and using a heterogeneous graph attention network. Furthermore, we use an additional sequence model to obtain the semantic information of the code fragment. Finally, we fuse the two kinds of information through a novel modality fusion method. We evaluate our approach on a widely used Java dataset; experimental results confirm that our approach outperforms existing methods.
Neuromorphic tuning of feature spaces to overcome the challenge of low-sample high-dimensional data

For learning algorithms, accessing large volumes of annotated data is highly desirable but not always available, especially in real-world scenarios. Accordingly, learning in the high-dimensional and low-sample size (HDLS) domain is recognised as one of the core challenges for modern AI systems. In this work, we consider a particular but very practical scenario in the HDLS domain where the number of training samples is not limited to mere few observations but yet it is not large enough to reliably build models with high degrees of expressivity. To address the problem, we present a new neuromorphic algorithm capable of fine-tuning existing feature spaces via learning relevant associations in high dimensional data with high probability. The algorithm is based on the idea of Concept Cells [1] and mimics properties attributed to memory and learning inherent to live neural systems. We demonstrate, through numerous numerical experiments, that the algorithm can “fine-tune” and “adapt” the feature space of pre-trained neural networks for better performance on new tasks in the HDLS domain. In addition, we study the impact of this “tuning” on quasi-orthogonal measures, which correlates with classification and calibration metrics.
Gradient Descent Optimizes Normalization-Free ResNets

Recent empirical studies observe that even without normalization, a deep residual network can be trained reliably. We call such a structure as normalization-free Residual Networks (N-F ResNets), which add a learnable parameter <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\alpha$</tex> to control the scale of the residual block instead of normalization. However, the theoretical understanding on N-F ResNets is still limited despite their empirical success. In this paper, we provide the first theoretical understanding of N-F ResNets from two perspectives. Firstly, we prove that the gradient descent (GD) algorithm can find the global minimum of the training loss at a linear rate for over-parameterized N-F ResNets. Secondly, we prove that N-F ResNets can avoid the gradient exploding or vanishing problem, by initializing the key parameter <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\alpha$</tex> to be a small constant. Notably, we demonstrate that the gradients of N-F ResNets are more stable than those of ResNets with Kaiming initialization. Moreover, empirical experiments on benchmark datasets verify our theoretical results.
Image-to-Image Translation on Defined Highlighting Regions by Semi-Supervised Semantic Segmentation

Image-to-image translations made remarkable performance in Generative Adversarial Network (GAN). While recent advances are easily generated a high-quality synthesized images, it usually remains a problem to recognize complicated scenarios. We believe that a few human annotations can greatly reduce the problems. In this paper, we propose Highlight-IT, which generates synthesized images and its corresponding pixel-level semantic segmentation. In addition, segmentation can be viewed as a strong prior and guide our framework to focus on human-defined important regions. In evaluation, we experiment with various categories of unlabeled and labeled datasets. The results show that our method achieves the quality of images of the state-of-the-art framework and also the performance of the famous semantic segmentation framework. In the end, we demonstrate the qualitative results of our work and the approaches proposed by others.
Discriminative Domain Adaptation Network for Fine-grained Disease Severity Classification

Unsupervised Domain Adaptation (UDA) has shown promise in improving medical diagnosis tasks on the unlabeled target domain by utilizing rich labels on the source domain. However, in real medical scenarios, it is crucial to obtain a fine-grained classification of the disease, in order to support physicians in making accurate diagnoses and treatment plans for patients. Unfortunately, accurately labeling medical data at a fine-grained level is challenging because of the diversity of patients, and the variety of diseases. This leads to a difference between the given label and the true label, referred to as label bias. The existing UDA methods are based on the premise that the given label on the source domain is the true label, so it is easy to transfer the biased knowledge to the target domain in Fine-grained Unsupervised Domain Adaptation (FUDA), which leads to the poor performance of the model on the target domain. We find the key factor of FUDA is the sample with a large label bias (bias-sample) which is located near the decision boundary of adjacent fine-grained classes. To solve this problem, we propose Discriminative domain adaptation Network for Fine-grained classification (DNF) with Discriminative Cross Entropy (DCE) and Discriminative Local multi-kernel Maximum Mean Discrepancy (DLMMD). DNF employs two classifiers with different parameters to discriminate bias-samples and reduce their weight for classification, and use the outputs of the classifiers to modify the expectation of each class made by the given label to make it closer to the expectation of the true label. Therefore, DNF transfers non-bias knowledge from the source domain to the target domain. Experiments on Hand Tremor (HT) and Gait Freezing (GF) show that our approach outperforms SOTA.
Feature selection for regression tasks base on explainable artificial intelligence procedures

In modern data analysis and data mining, there are many applicable regression procedures, however, the most frequently used ones include classical linear regression, decision trees, random forest model, support vector method and artificial neural networks. Most of these algorithms are powerful approximation tools that can model the complex relationships and patterns in data. One of the reasons for redundant structure within such procedures is the use of a too-large feature vector for a given task. This paper aimed to investigate and compare the selected methods of significance analysis and then reduce the size of the feature vector through supervised learning models. Based on the obtained results, it seems that the Shap analysis and the Sobol global sensitivity analysis method are the most reliable methods for determining the significance of feature vector elements and thus for reducing the feature vector dimension in regression problems.
CCCSpell: A Consistent and Contrastive Learning Approach with Character Similarity for Chinese Spelling Check

Chinese Spelling Check (CSC) is a task to detect and correct Chinese misspellings. Existing methods have adopted pre-trained language models (PLMs) and achieved state-of-the-art performances. However, due to the Dropout strategy, the PLM-based models suffer from inconsistency between the training and testing stages, i.e., the semantic representation during training is inconsistent with the counterpart during testing. Besides, the PLM-based models prefer to correct the erroneous characters to semantically proper or commonly used ones rather than the ground truths. And they usually neglect phonological and visual similarity knowledge between characters. In this paper, we propose a consistent and contrastive learning approach with character similarity for CSC (CCCSpell). CCCSpell is composed of a BERT-based detector and a character similarity-based corrector. The detector uses bidirectional Kullback-Leibler divergence to minimize the gap between the output distributions of two sub-models sampled by Dropout, which can enhance the consistency of the semantic representations when training and inferring to improve the testing performance. Besides, the detector employs a contrastive optimization objective to increase the confidence of targets and decrease the counterpart of common characters, which can help avoid predicting common ones. After that, the corrector tries to incorporate phonological and visual similarity knowledge into detection results by calculating the levenshtein edit distance between the shape/pronunciation representations of characters, and utilizes the confusion set to help select golden characters. Experiments are conducted on SIGHAN and OCR datasets, and the results demonstrate that CCCSpell outperforms all baseline models and achieves new state-of-the-art performances.
Engagement Recognition in Online Learning Based on an Improved Video Vision Transformer

Online learning has gained wide attention and application due to its flexibility and convenience. However, due to the separation of time and space, the level of students' engagement is not easily informed by teachers, which affects the effectiveness of teaching. Automatic detection of students' engagement is an effective way to solve this problem. It can help teachers obtain timely feedback from students and adjust the teaching schedule. In this paper, transformer is first applied in engagement recognition and a novel network based on an improved video vision transformer (ViViT) is proposed to detect student engagement. A new transformer encoder, named Transformer Encoder with Low Complexity (TELC) is proposed. It adopts unit force operated attention (UFO-attention) to eliminate the nonlinearity of the original self-attention in standard ViViT and Patch Merger to fuse the input patches, which allows the network to significantly reduce computational complexity while improving performance. The proposed method is evaluated on the Dataset for Affective States in E-learning Environments (DAiSEE) and achieves an accuracy of 63.91% in the four-level classification task, which is superior to state-of-the-art methods. The experimental results demonstrate the effectiveness of our method, which is more suitable for the practical application of online learning.