Reviewerly/exHarmony huggingface dataset:
A Node Agent for Fast and Safe Execution of Computing Tasks under Kubernetes

Current cloud platforms host workloads by creating a container instance and destroying the instance when the call is complete. However, the use of containers brings much overhead, which is not suitable for performance-sensitive computing tasks such as emerging IoT applications with real-time latency constraints. In this paper, we discuss the existing solutions for performing computational tasks in Kubernetes, a software that can automatically manage, scale, and maintain the required state of multiple workloads, as well as unresolved performance challenges in resource-constrained nodes. To achieve higher performance, we propose and implement a node agent that performs computing tasks quickly and safely. Compared with traditional container-based node agent, the performance of our node agent is improved by about 100 times and 1.1 times in the cold start and hot start workload scenarios, respectively. Compared with the latest solution using Webassembly, it improves the cold start and hot start time by about 10%-50%, and can be applied to richer scenarios.
Multi-Step Hindsight Experience Replay with Bias Reduction for Efficient Multi-Goal Reinforcement Learning

Multi-goal reinforcement learning has emerged as a powerful approach for planning and robot manipulation tasks, but it faces challenges such as sparse rewards and sample inefficiency. Hindsight Experience Replay (HER) has been proposed as a solution to these challenges by relabeling goals, but it still requires a large number of samples and significant computation. To address these issues, we propose Multi-step Hindsight Experience Replay (MHER), which incorporates multi-step relabeling to improve sample efficiency. Despite the advantages of $n -$step relabeling, we theoretically and experimentally prove the off-policy $n -$step bias introduced by $n -$step relabeling may lead to poor performance in many environments. To address this issue, two bias-reduced MHER algorithms, MHER $( \lambda )$ and Model-based MHER (MMHER) are presented. MHER $( \lambda )$ exploits the $\lambda$ return while MMHER benefits from model-based value expansions. Experimental results on numerous multi-goal robotic tasks show that our solutions can successfully alleviate the off-policy $n -$step bias and achieve significantly higher sample efficiency than previous multi-goal RL baselines with little additional computation beyond HER.
Fast Multivariate Time Series Anomaly Detection Based on Matrix Completion

Increasingly complex systems contain large numbers of devices that generate great number of multivariate time series that are monitored and recorded. For anomaly detection of these complex time series, deep learning technology is often used to build a learning model for anomaly detection. The method based on deep learning usually takes a long time to train data in practice, and the online service system is deployed or changed very frequently, which makes the online service system need to frequently spend a lot of time updating the anomaly detection model. To overcome this challenge, this paper proposes a fast multivariate time series anomaly detection (FMAD) approach based on matrix completion. After sampling the primitive low-rank matrix, FMAD uses matrix completion to reconstruct the original low-rank matrix to detect anomalies in multivariate time series. We conducted related experimental analysis and discussion. The experimental results on three real data sets show that FMAD has improved the problem of too long model training time and improved the detection efficiency.
RDEPD: Re-Exploring Depth Estimation for Pedestrian Detection

Pedestrian detection is a fundamental task in computer vision field. It remains challenging due to perspective affine and object occlusion. To alleviate these problems, this paper re-thinks the assistance of depth estimation, and then proposes a novel pedestrian detection method named RDEPD. The method consists of a data augmentation module based on depth (DamD) and a detection framework with learnable attention module based on depth(LAmD) and self-suitable NMS(S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> NMS). DamD provides hard examples with mosaics at the depth gradient, which can improve the generalization ability. The detection framework pays more attention on these instances with occlusion or perspective affine by LamD and S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> NMS. Where LAmD is responsible for integrating depth and RGB cues to guide localization, and S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> NMS exploits every possible predicting box to improve the detecting precision. Extensive experimental results demonstrate that the proposed RDEPD significantly outperforms most state-of-the-art methods on the authoritative MOT17det, CrowdHuman, and Citypersons datasets, especially for occlusion situations.
ADA-VIT: Attention-Guided Data Augmentation for Vision Transformers

The limitations of a machine learning model can often be traced back to the existence of under-represented regions in the feature space of the training data. Data augmentation is a common technique that has been used to inflate training datasets with new samples to improve the model performance. However, these techniques usually focus on expanding the data in size and do not necessarily aim to cover the under-represented regions of the feature space. In this paper, we propose an Attention-guided Data Augmentation technique for Vision Transformers (ADA-ViT). Our framework exploits the attention mechanism in vision transformers to extract visual concepts related to misclassified samples. The retrieved concepts describe under-represented regions in the training dataset that contributed to the misclassifications. We lever-age this information to guide our data augmentation process by identifying new samples and using them to augment the training data. We hypothesize that this focused data augmentation populates under-represented regions and improves the model’s accuracy. We evaluate our framework on the CUB dataset and CUB-Families. Our experiments show that ADA-ViT outperforms state-of-the-art data augmentation strategies, and can improve the accuracy of a transformer by an average margin of 2.5% on the CUB dataset and 3.3% on CUB-Families.
Joint Demosaicing and Denoising with Gradient Guidance in Quad Bayer CFA

In this study, we introduce a challenging task called Joint Demosaicking and Denoising in Quad Bayer CFA (JDD-QBC). Inspired by the effectiveness of gradient prior in previous tasks, we design a gradient prior extraction algorithm to extract robust gradient profiles directly from Quad Bayer raw images. By leveraging contextual information, our algorithm overcomes the limitations of previous gradient extraction algorithms. Building on this prior, we propose an end-to-end JDD-QBC framework called Gradient Guidance Network (G <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2-</sup> Net), consisting of two specifically designed components: a Gradient Refinement Branch (GRB) to transform the extracted gradient prior, and a Multi-Scale Backbone Branch (MSB <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) to recover all missing values under the guidance of transformed prior. Extensive experiments demonstrate our proposed G <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2-</sup> Net outperforms state-of-the-art methods across a wide range of noise levels while requiring 50% less memory cost.
Functional Knowledge Transfer with Self-supervised Representation Learning

This work investigates the unexplored usability of self-supervised representation learning in the direction of functional knowledge transfer. In this work, functional knowledge transfer is achieved by joint optimization of self-supervised learning pseudo task and supervised learning task, improving supervised learning task performance. Recent progress in self-supervised learning uses a large volume of data, which becomes a constraint for its applications on small-scale datasets. This work shares a simple yet effective joint training framework that reinforces human-supervised task learning by learning self-supervised representations just-in-time and vice versa. Experiments on three public datasets from different visual domains, Intel Image, CIFAR, and APTOS, reveal a consistent track of performance improvements on classification tasks during joint optimization. Qualitative analysis also supports the robustness of learnt representations. Source code and trained models are available on GitHub <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Prompt Prototype Learning Based on Ranking Instruction For Few-Shot Visual Tasks

Querying large language models (LLMs), such as GPT-3, for high-quality prompts and utilizing pre-trained vision-language models, such as CLIP, to construct a zero-shot visual classification model, offer promising performance across various downstream visual tasks. However, when applied to specific domains, their efficacy is restricted due to the gap between the general prompts they generate and the required domain-specific knowledge. In this paper, we propose a novel, lightweight method for prompt prototype learning through ranking instruction, specifically designed to bridge this gap in the context of few-shot visual classification. We generate domain-specific prompts leveraging the knowledge contained in LLMs and then fine-tune the prompt prototype with effective ranking instructions from several domain images. Our few-shot experiments on facial expression benchmarks demonstrate the efficacy of the prompt prototype. Notably, our method delivers results that are on par with state-of-the-art few-shot image classification techniques and can be integrated with them to further improve performance in the facial expression domain. Our approach provides a promising solution to few-shot visual classification, making use of the knowledge contained in LLMs to generate domain-specific prompts.
Consistent and Multi-Scale Scene Graph Transformer for Semantic-Guided Image Outpainting

The task of image outpainting extends an image beyond its boundaries with semantically plausible content. Recently, Scene Graph Transformer (SGT) introduced a transformer architecture to leverage scene graph guidance for image outpainting. Despite its success, we identified two shortcomings: (a) SGT uses a positional encoding that was originally proposed for 1D signal; (b) SGT uses a scene graph attention layer that propagates information between neighboring nodes which limited the model to learning local graph features. To address these issues, we propose incorporating Laplacian positional encoding and introducing a multiscale scene graph attention into SGT. Extensive results on MS-COCO and Visual Genome show that our proposed approach generates more plausible outpainted images with higher quality.
ADFA: Attention-Augmented Differentiable Top-K Feature Adaptation for Unsupervised Medical Anomaly Detection

The scarcity of annotated data, particularly for rare diseases, limits the variability of training data and the range of detectable lesions, presenting a significant challenge for supervised anomaly detection in medical imaging. To solve this problem, we propose a novel unsupervised method for medical image anomaly detection: Attention-Augmented Differentiable top-k Feature Adaptation (ADFA). The method utilizes Wide-ResNet50-2 (WR50) network pre-trained on ImageNet to extract initial feature representations. To reduce the channel dimensionality while preserving relevant channel information, we employ an attention-augmented patch descriptor on the extracted features. We then apply differentiable top-k feature adaptation to train the patch descriptor, mapping the extracted feature representations to a new vector space, enabling effective detection of anomalies. Experiments show that ADFA outperforms state-of-the-art (SOTA) methods on multiple challenging medical image datasets, confirming its effectiveness in medical anomaly detection.
Part Aware Graph Convolution Network with Temporal Enhancement for Skeleton-Based Action Recognition

In recent years, skeleton-based human action recognition has attracted broad research interests, and methods based on graph convolution networks have demonstrated excellent performance. However, how to extract the distinguishing spatio-temporal information effectively remains an essential problem. To address the problem, we propose a novel part aware graph convolution network with temporal enhancement, which can adaptively evaluate the activity level of each part of the body in the action sequence and enhance the extraction of temporal information. Considering that the range of motion of body parts in the action sequence is greater than that of joints, we manually divide the five major parts of the body and generate a skeleton sequence with different attention weights by using part-based attention module. Then, a temporal enhanced module is used to model actions with different duration. Experiments show that our method achieves the state-of-the-art performance.
Deep Unsupervised Reflection Removal Using Diffusion Models

Reflections caused due to surfaces such as glass affect the aesthetics of an image, and are hence undesirable. Most of the recent works on reflection removal use supervised learning based approaches using deep neural networks. However, most of these methods require large amount of paired data for training, which is difficult to obtain. Moreover, it is difficult to deploy existing deep learning based algorithms on multiple devices with different computational power, since it is very hard to control the trade-off between the strength of reflection removal and computational complexity during inference. To address these challenges, we propose a novel deep learning based approach for reflection removal, that is both unsupervised and controllable. We use Denoising Diffusion Probability Models to learn a distribution of reflection-free images. The learnt model is then used to generate reflection-free images using an input conditioned forward diffusion process during inference. We also perform qualitative and quantitative comparison and show that our method is at par or better than existing methods for deep supervised reflection removal, while outperforming unsupervised method by ~ 6.5 dB.
Weighted Anisotropic–Isotropic Total Variation for Poisson Denoising

Poisson noise commonly occurs in images captured by photon-limited imaging systems such as in astronomy and medicine. As the distribution of Poisson noise depends on the pixel intensity value, noise levels vary from pixels to pixels. Hence, denoising a Poisson-corrupted image while preserving important details can be challenging. In this paper, we propose a Poisson denoising model by incorporating the weighted anisotropic–isotropic total variation (AITV) as a regularization. We then develop an alternating direction method of multipliers with a combination of a proximal operator for an efficient implementation. Lastly, numerical experiments demonstrate that our algorithm outperforms other Poisson denoising methods in terms of image quality and computational efficiency.
PS-NeRV: Patch-Wise Stylized Neural Representations for Videos

We study how to represent a video with implicit neural representations (INRs). Classical INRs methods generally utilize MLPs to map input coordinates to output pixels. While some recent works have tried to directly reconstruct the whole image with CNNs. However, we argue that both the above pixel-wise and image-wise strategies are not favorable to video data. Instead, we propose a patch-wise solution, PS-NeRV, which represents videos as a function of patches and the corresponding patch coordinate. It naturally inherits the advantages of image-wise methods, and achieves excellent reconstruction performance with fast decoding speed. The whole method includes conventional modules, like positional embedding, MLPs and CNNs. We also introduce AdaIN to enhance intermediate features. Extensive experiments have demonstrated its effectiveness in several video-related tasks, such as video compression and video inpainting.
OnDA-DETR: Online Domain Adaptation for Detection Transformers with Self-Training Framework

This paper presents a novel method for online domain adaptation (OnDA) for DEtection TRansformer (DETR)-based object detection models called OnDA-DETR. OnDA is a domain adaptation paradigm that adapts a model trained on the source domain data to perform well on the target domain in an online manner during testing, using only the unlabeled test data from the target domain. Due to challenging and realistic problem settings, OnDA has garnered significant attention. However, OnDA methods for DETR-based models, which have demonstrated excellent performance in object detection research fields, had not been developed. OnDA-DETR is the first OnDA method specifically designed for DETR-based models. OnDA-DETR incorporates a self-training framework that generates pseudo-labels for the unlabeled target domain data. To effectively incorporate the self-training framework into DETR-based models, we leverage recall-aware pseudo-labeling and quality-aware training in OnDA-DETR. Experimental results indicate that OnDA-DETR improves the performance of the source-trained model by about 3.0 % points through OnDA.
A No-Reference Quality Assessment Method for Digital Human Head

In recent years, digital humans have been widely applied in augmented/virtual reality (A/VR), where viewers are allowed to freely observe and interact with the volumetric content. However, the digital humans may be degraded with various distortions during the procedure of generation and transmission. Moreover, little effort has been put into the perceptual quality assessment of digital humans. Therefore, it is urgent to carry out objective quality assessment methods to tackle the challenge of digital human quality assessment (DHQA). In this paper, we develop a novel no-reference (NR) method based on Transformer to deal with DHQA in a multi-task manner. Specifically, the front 2D projections of the digital humans are rendered as inputs and the vision transformer (ViT) is employed for the feature extraction. Then we design a multi-task module to jointly classify the distortion types and predict the perceptual quality levels of digital humans. The experimental results show that the proposed method well correlates with the subjective ratings and outperforms the state-of-the-art quality assessment methods.
Structure-Aware Generative Adversarial Network for Text-to-Image Generation

Text-to-image generation aims at synthesizing photo-realistic images from textual descriptions. Existing methods typically align images with the corresponding texts in a joint semantic space. However, the presence of the modality gap in the joint semantic space leads to misalignment. Meanwhile, the limited receptive field of the convolutional neural network leads to structural distortions of generated images. In this work, a structure-aware generative adversarial network (SaGAN) is proposed for (1) semantically aligning multimodal features in the joint semantic space in a learnable manner; and (2) improving the structure and contour of generated images by the designed content-invariant negative samples. Experimental results show that SaGAN achieves over 30.1% and 8.2% improvements in terms of FID on the datasets of CUB and COCO when compared with the state-of-the-art approaches.
Cross-Scale Query-Support Alignment Approach for Small Object Detection in the Few-Shot Regime

Small object detection is a challenging task in computer vision. In the few-shot regime, this problem is reinforced. Leveraging useful information from only a few examples is difficult, in particular with small objects. We hypothesize that features extracted from small objects are noisy and often dominated by background information. In addition, recent detectors rely on multi-scale features and visually similar objects of different sizes may have unaligned representations. We address these issues with Cross-Scale Query-Support Alignment (XQSA), a novel attention mechanism that combines features from query and support images at different scales. This allows matching objects of different sizes and therefore improves Few-Shot Object Detection (FSOD) performance. Extensive experiments are conducted on four distinct datasets, including natural images (Pascal VOC and MS COCO) and aerial images (DOTA and DIOR). XQSA improves the detection of small objects on all tested datasets. In aerial images, which contain smaller objects, it yields significant gains for the overall detection and outperforms the state-of-the-art results on DOTA and DIOR.
Image Stitching Based on Multi-Scale Meshes

Generating high-quality stitching images with a natural structure is a challenging task in computer vision. Recent image stitching methods based on warps failed to suppress the distortion of the images. They often bend the salient lines in the image, which is inconsistent with human perception. In this paper, we succeed in proposing a novelty model called multi-perspective warps for natural image stitching which is related to the density of feature points in the images. With it we can get more precise matching results. Three new energy terms are developed to stitch quality to specify and balance the expected for aligning the vertices of the multi-scale mesh, which can constrain the transformation of the mesh. We also explore and introduce three feature point reconstruction algorithms to enrich the features in the images. Extensive experiments demonstrate that the proposed method outperforms most state-of-the-arts by effectively preserving the linear structure in the image and improving the robustness.
MDFD: Study of Distributed Non-IID Scenarios and Frechet Distance-Based Evaluation

With the development of distributed machine learning and federated learning, the solution to the data island problem is promoted. People use computer clusters to train machine learning models on data distributed in different regions. In the early stage of research, researchers usually assume that the data sets of each node are independent identically distribution (IID), but this is a strong assumption, which is challenging to meet in practical applications. Therefore, research on non-IID has become a hot spot in recent years. However, there is no uniform standard for designing and evaluating non-IID scenarios. This paper proposes a Frechet distance-independent non-IID distribution dataset metric MDFD. And we conducted experiments on different types of distributed machine-learning methods in different non-IID scenarios to verify the effectiveness of MDFD.