Reviewerly/exHarmony huggingface dataset:
Customizing Synthetic Data for Data-Free Student Learning

Data-free knowledge distillation (DFKD) aims to obtain a lightweight student model without original training data. Existing works generally synthesize data from the pretrained teacher model to replace the original training data for student learning. To more effectively train the student model, the synthetic data shall be customized to the current student learning ability. However, this is ignored in the existing DFKD methods and thus negatively affects the student training. To address this issue, we propose Customizing Synthetic Data for Data-Free Student Learning (CSD) in this paper, which achieves adaptive data synthesis using a self-supervised augmented auxiliary task to estimate the student learning ability. That is, data synthesis is dynamically adjusted to enlarge the cross entropy between the labels and the predictions from the self-supervised augmented task, thus generating the hard samples for the student model. The experiments on various datasets and teacher-student models show the effectiveness of our proposed method. Code is available at: https://github.com/luoshiya/CSD
Towards Efficient Large Mask Inpainting via Knowledge Transfer

Image inpainting aims to complete the damaged image with plausible and realistic content. Recently some methods demonstrate promising results by incorporating auxiliary generators or co-learning branches. However, they typically require a large amount of computational resources. Moreover, due to the insufficient receptive field of conventional convolution layers, they usually struggle to deal with large holes. To address these problems, we propose an efficient inpainting framework dubbed as EFill that can cost-effectively capture the global context information via the Light-weight Fast Fourier Convolution (LFFC). Besides, it leverages an Inference-Free Co-learning Branch (IFCB) to acquire unbiased structural prior knowledge avoiding unnecessary computation overheads in testing phase. Furthermore, we introduce knowledge distillation to provide alternative content clues about the missing areas which alleviate the inherently ill-posed nature of inpainting and propel the learning process. Consequently, EFill surmounts the tiresome efficiency aversion and performs surprisingly well in processing the large-scale missing areas. Extensive experiment results demonstrate the superiority of EFill compared to state-of-the-art methods in terms of model efficiency.
Unmanned Aerial Vehicle Forensics Investigation Performance under Different Attacks

Unmanned Aerial Vehicles (UAVs), also called drones, have grown tremendously in recent years and have been adopted in various sectors. With the continuous development of machine learning algorithms to detect various attacks on UAVs. The attackers also focused on utilizing this machine-learning algorithm to disrupt the UAV's activities by using predictions to generate the attack route. In digital forensics, the forensics experts focus on the communication data between the controller and the drones, the multimedia files, and the flight data to make a complete cybersecurity report. This paper reviews various ways the drone may be compromised and its performance evaluation using the existing machine learning detection algorithm to make the forensics report about the drones. The machine learning algorithms investigated in this study include Multilayer Perceptron (MLP) for the detection technique.
Sparse Representation based Deep Residual Geometry Compression Network for Large-scale Point Clouds

The increasing applications of 3D point clouds require efficient compression techniques to achieve high-quality and low-delay services. However, the computational efficiency and rate-distortion performance for large-scale dense point clouds are still challenging, and the phenomenon of reconstruction ability degradation also exists when the network is deep. To solve these challenges, we propose a novel fully end-to-end point cloud compression model based on sparse convolution. Specifically, we adopt a long-range-residual aided architecture to avoid the reconstruction degradation and high computational complexity of deep networks. Further, we propose a multi-scale geometry compression module to construct an end-to-end network that avoids the accumulation of reconstruction distortion during decoding. Experiments on the large-scale Moving Picture Experts Group (MPEG) PCC benchmarks show that our model outperforms the latest Video-based Point Cloud Compression (V-PCC) scheme in terms of lossy geometry compression by 50.4% in D1 BD-rate and 50.8% in D2 BD-rate, while maintaining affordable processing speed and memory consumption.
Synthetic Feature Assessment for Zero-Shot Object Detection

Zero-shot object detection aims to simultaneously identify and localize classes that were not presented during training. Many generative model-based methods have shown promising performance by synthesizing the visual features of unseen classes from semantic embeddings. However, these synthetic features are inevitably of varied quality, which may be far from the ground truth. It degrades the performance of trained unseen classifier. Instead of tweaking the generative model, a new idea of feature quality assessment is proposed to utilize both the good and bad features to optimize the classifier in the right direction. Moreover, contrastive learning is also introduced to enhance the feature uniqueness between unseen and seen classes, which helps the feature assessment implicitly. To demonstrate the effectiveness of the proposed algorithm, comprehensive experiments are conducted on the MS COCO dataset and PASCAL VOC dataset, the state-of-the-art performance is achieved. Our code is available at: https://github.com/Dai1029/SFA-ZSD.
Self-supervised Cross-stage Regional Contrastive Learning for Object Detection

Cross-stage object similarity is a vital property of generic supervised object detectors, which maintains similar feature responses to the same object across feature maps of different intermediate stages of the backbone network. Since an object can be predicted by multiple stages, this similarity is beneficial for accurate object classification and localization. Inspired by this property, we introduce Cross-stage regional Contrastive Learning (CrossCL) to learn the cross-stage object similarity during the model pre-training. Since labels are unavailable in self-supervised learning, we treat the regions sharing the same position in different stages as the same object and constrain them to have similar feature responses across stages to achieve cross-stage object similarity. The learned feature representations of CrossCL share a similar property with supervised detectors, thus showing strong transfer capability to object detection tasks. Besides, we also provide in-depth discussions, ablation studies, and visualizations to understand better how CrossCL works. Code is available at https://github.com/yanjk3/CrossCL.
Dual Control Inspired Active Sensing for Bearing-Only Target Tracking

Automating sensing processes is of high interest to both the target tracking and the control community. Active sensing is focused on solving this task, usually with information based or task driven selection of optimal sensing actions. This paper presents an active sensing formulation that combines task based, in the form of standoff tracking, and information based active sensing by implementing the dual control for exploitation and exploration (DCEE) concept to control a mobile sensor platform with a limited field-of-view. The DCEE based cost function is integrated into the Monte Carlo tree search (MCTS) framework for non-myopic decision making. Using the Bernoulli particle filter for single target tracking with bearing-only measurements, the DCEE observer control method is benchmarked against the popular Rényi divergence information metric with two different parameterisations. Whilst the Rényi divergence performs marginally better when considering existence estimation, spatial results clearly demonstrate that our formulation is able to outperform the benchmark algorithm with improved target localisation performance resulting from outmanoeuvring of the target.
A Simple Stochastic Neural Network for Improving Adversarial Robustness

The vulnerability of deep learning algorithms to malicious attack has garnered significant attention from researchers in recent years. In order to provide more reliable services for safety-sensitive applications, prior studies have introduced Stochastic Neural Networks (SNNs) as a means of improving adversarial robustness. However, existing SNNs are not designed from the perspective of optimizing the adversarial decision boundary and rely on complex and expensive adversarial training. To find an appropriate decision boundary, we propose a simple and effective stochastic neural network that incorporates a regularization term into the objective function. Our approach maximizes the variance of the feature distribution in low-dimensional space and forces the feature direction to align with the eigenvectors of the covariance matrix. Due to no need of adversarial training, our method requires lower computational cost and does not sacrifice accuracy on normal examples, making it suitable for use with a variety of models. Extensive experiments against various well-known white- and black-box attacks show that our proposed method outperforms state-of-the-art methods.
Variational Information Bottleneck for Cross Domain Object Detection

Cross domain object detection leverages a labeled source domain to learn an object detector which performs well in a novel unlabeled target domain. Most existing works mainly align the distribution utilizing the entire image knowledge ignoring the obstacles of task-uncorrelated information to alleviate the domain discrepancy. To tackle this issue, we propose a novel module called Variational Instance Disentanglement (VID) based on information theory which aims to decouple the information of task-correlated while filtering out the task-uncorrelated factors at the instance level. Notably, the proposed VID can be used as a plug-and-play module without bringing extra network parameter cost. We equip it with adversarial network and self-training network forming Variational Instance Disentanglement Adversarial Network (VIDAN) and Variational Instance Disentanglement Self-training Network (VIDSN), respectively. Extensive experiments on multiple widely-used scenarios show that the proposed method improves the performance of the popular frameworks and outperforms state-of-the-art methods.
PSR: Probabilistic Serendipitous Recommendations

Recent recommender systems studies have started focusing on accelerating serendipity to offer users diverse perspectives. However, recommending serendipitous items is known to decrease users' satisfaction due to low recommendation accuracy. In this paper, we propose a probabilistic serendipitous recommendations (PSR) model that aims to discover serendipitous items through the fully-connected combinations of unrelated user-provided information. The proposed model provides an unexpected result, while user's answers are maintained not to decrease recommendation accuracy. An online user study with 40 participants revealed that the proposed PSR model provides serendipitous items by using the intersection of usefulness and unexpectedness from the recommended results.
Image Layer Modeling for Complex Document Layout Generation

Document layout analysis (DLA) plays an essential role in information extraction and document understanding. At present, DLA has reached the milestone achievement; however, DLA of non-Manhattan is still challenging because of annotation data limitations. In this paper, we propose an image layer modeling method to mitigate this issue. The image layer modeling method generates document images of non-Manhattan layouts by superimposing images under pre-defined aesthetic rules. Due to the lack of evaluation benchmark for non-Manhattan layout, we have constructed a manually-labeled non-Manhattan layout fine-grained segmentation dataset. To the best of our knowledge, this is the first manually-labeled non-Manhattan layout fine-grained segmentation dataset. Extensive experimental results verify that our proposed image layer modeling method can better deal with the fine-grained segmented document of the non-Manhattan layout.
Multi-stream Adaptive Offloading of Joint Compressed Video Streams, Feature Streams, and Semantic Streams in Edge Computing Systems

Edge computing (EC) is a promising paradigm for serving latency-sensitive video applications. However, massive compressed video transmission and analysis require considerable bandwidth and computing resources, posing enormous challenges for current multimedia frameworks. Novel multi-stream frameworks that incorporate feature streams are more practical. The reason is that feature streams containing compact video frame feature data have a lower bitrate and better serve machine vision tasks. Nevertheless, feature extraction by devices increases the latency and energy consumption of local computing. Therefore, how to offload suitable streams according to video task requirements and system resources is a challenging issue. This paper studies EC-based multi-stream adaptive offloading. We model the multi-stream offloading and computation problem to maximize system utility by jointly optimizing offloading decisions, computation resource allocation, and video frame sampling rates. Frame sampling rates, processing latency, and energy consumption are considered in system utility modeling. The formulated optimization problem is a mixed-integer programming (MIP) problem. We propose an efficient algorithm to address this MIP problem. The proposed algorithm relies on the Hungarian algorithm and improved greedy Markov approximation. The simulation results validate our proposed algorithm’s superior performance.
Adversarially Robust Source-free Domain Adaptation with Relaxed Adversarial Training

Unsupervised Domain Adaptation (UDA) learns a model for an unlabeled target domain, utilizing a labeled source domain. Most existing works on UDA assume the availability of source data and neglect the adversarial robustness of the models, hindering security-sensitive real-world applications. In this paper, we study adversarially robust source-free UDA, aiming to train a robust target model by adapting a non-robust source model without using source data. A basic approach is to train a non-robust teacher model via conventional source-free UDA to predict pseudo-labels for target data, and then train a robust student model via adversarial training (AT). However, AT tends to magnify the errors of the teacher model, reducing the accuracy. Hence, we propose Relaxed Adversarial Training (RAT) that relieves the constraints on the confidence of predictions in AT to balance the robustness and accuracy. Extensive experiments validate that RAT can improve the accuracy on clean and adversarial samples, and is superior to related methods. Our code is available at https://github.com/Coxy7/RAT.
SPTNET: Span-based Prompt Tuning for Video Grounding

When a Pre-trained Language Model (PLM) is adopted in video grounding task, it usually acts as a text encoder without having its knowledge fully utilized. Also, there exists an inconsistency problem between the pre-training and downstream objectives. To solve the issues, we propose a new paradigm, named Span-based Prompt Tuning (SPTNet). It can convert the video grounding task into a cloze form. Specifically, a query is first changed into a form with mask token by a template, then the video and the query embeddings are integrated through a cross-modal transformer. The start and end points of the query matching time span are predicted with the embedding of the mask token. Experimental results on two public benchmarks ActivityNet Captions and Charades-STA show that our SPTNet achieves surpassing performance compared with state-of-the-art methods.
Improving Quality of Software Requirements by Using a Triplet Structure

The development of a quality software system is a priority in the domain of software engineering. Quality requires to define unambiguous and consistent requirements that are not conducive to various interpretations. Indeed, the success of the realization of a software system depends largely on the phase of software requirements specification. The requirements specification phase consists, among other things, in describing in a precise and unambiguous manner the characteristics of the system to be developed. Moreover, the techniques for writing software specification documents used in the industry don't facilitate to define unambiguous and consistent requirements. In industry, software requirements are often written in natural language, and no technical details are specified. Thus, software requirements are incomplete, inconsistent and prone to ambiguities, and therefore interpretation errors can easily be made by analysts. This article introduces a new technique for writing and developing software requirements that could help to reduce the ambiguities and inconsistencies in the document of specification software requirements. Our technique is validated by the development of a new tool that detects the ambiguities and inconsistencies in the software requirements, and generates the potential methods and classes from requirements written in natural language. Our tool integrates a set of techniques in natural language processing (NLP), and in artificial intelligence which helps to improve the software requirements quality.
Flow-Guided Attention Deformation for Person Image Generation

Pose-guided person image generation aims to transfer reference images to target poses while preserving the source appearance. Recent approaches achieve considerable improvement by using spatial transformation modules such as attention operation. However, the commonly used vanilla attention tends to generate a dense correlation matrix which means that the value of a target position is the weighted sum of many source positions, resulting in blurry appearance. In this paper, we propose a novel model named Flow-guided Attention Deformation (FAD) to perform the spatial transformation. Our model first establishes the correlation between sources and targets with a flow-guided attention operation. Then, with the obtained correlation matrix, we perform an accurate deformation for source features to generate the predicted image. Extensive results demonstrate the superiority of the proposed method, outperforming state-of-the-art methods quantitatively and qualitatively. Ablation studies clarify the efficiency of the proposed modules and verify our hypothesis.
Rethinking Overfitting of Multiple Instance Learning for Whole Slide Image Classification

Multiple instance learning(MIL) is widely used for whole slide image(WSI) classification. However, these methods suffer from severe overfitting. In this paper, we introduce two main causes of such overfitting problems by rethinking the MIL task and formulation of attention-based MIL models: (i) The model is sensitive to the proportion of positive regions, and (ii)incorrectly learns the positional relationship of patches (i.e., the order of instances). To this end, we propose recurrent random padding(RRP) module and patch shuffle(PS) module to tackle these two issues, respectively. Furthermore, we present random alignment(RA) algorithm to solve these two overfitting problems simultaneously. On CAMELYON16 and TCGA-NSCLC, the proposed plug-and-play modules improve the performance of six baselines by large margins. The significant and consistent refinement demonstrates the correctness of our theories and the effectiveness of our modules.
Comparison of Machine Learning Methods for Human Activity Recognition Using Pseudo Free-Living Data

Recognition of human activities has become more critical in recent years. Practical approaches for classifying physical activities are becoming required in the scientific community, and such models are needed explicitly in health promotion and behavior. The data collected in real-world conditions differ from laboratory data, which has been the main focus of much research. As a result, in this work, we analyzed several machine-learning techniques and models for identifying human activity using a pseudo-free-living dataset obtained at the University of Georgia. We found that hierarchical meta-classifiers outperformed deep learning and classical models by 6% for classifying seven activities. Model personalization is promoted since it lowers the inter-subject variability of the dataset. We divided activities based on the Metabolic Equivalent of Task (MET), and we achieved 80% inter-subject accuracy and 87% accuracy by including 50% of the participant's data. Achieving high performance for machine learning models is challenging using real-world data.
A Scalable Rao-Blackwellised Sequential MCMC Sampler for Joint Detection and Tracking in Clutter

This paper addresses the joint detection and tracking of an unknown and time-varying number of targets in clutter. Here we formulate the tracking task in a variable-dimension state space, under which the reversible jump sequential Markov chain Monte Carlo sampling methods can be utilised to online estimate the target number, their kinematic states, and the association variables. In particular, a fast Rao-Blackwellisation scheme is devised to improve the tracking accuracy and sampling efficiency for linear Gaussian models. Based on the nonhomogeneous Poisson process measurement model, the developed tracker enjoys a partially parallel sampling structure, thereby being able to efficiently tackle the data association under massive measurements and clutter. The simulation results demonstrate that the developed tracker exhibits superior tracking performance in comparison to existing trackers in both accuracy and computational efficiency when tracking multiple targets under heavy clutter.
Meaningful ciphertext image encryption based on histogram shift and ND-ICM hyperchaos

A novel method for meaningful ciphertext image encryption is proposed in this paper. The method involves a series of steps, including image compression, histogram shift, and disordered cyclic shift scrambling under M-ary decomposition to generate noise-like ciphertext, followed by generalized least significant bit (GLSB) embedding and correction to produce meaningful ciphertext. In addition, a new N-dimensional infinite collapse discrete chaotic map (ND-ICM) is designed to generate pseudo-random sequences for controlling ciphertext generation. The combination of histogram shift and GLSB embedding significantly improves the quality of the ciphertext, and the ND-ICM is capable of adjusting dimensions as needed, exhibiting good chaotic properties. Experimental results demonstrate that the quality of the ciphertext produced by this algorithm is significantly superior to that of existing visual meaning image encryption algorithms, indicating a broad range of potential applications.