Reviewerly/exHarmony huggingface dataset:
Covariance-Aware Feature Alignment with Pre-Computed Source Statistics for Test-Time Adaptation to Multiple Image Corruptions

Real-world image recognition systems often face corrupted input images, which cause distribution shifts and degrade the performance of models. These systems often use a single prediction model in a central server and process images sent from various environments, such as cameras distributed in cities or cars. Such single models face images corrupted in heterogeneous ways in test time. Thus, they require to instantly adapt to the multiple corruptions during testing rather than being re-trained at a high cost. Test-time adaptation (TTA), which aims to adapt models without accessing the training dataset, is one of the settings that can address this problem. Existing TTA methods indeed work well on a single corruption. However, the adaptation ability is limited when multiple types of corruption occur, which is more realistic. We hypothesize this is because the distribution shift is more complicated, and the adaptation becomes more difficult in case of multiple corruptions. In fact, we experimentally found that a larger distribution gap remains after TTA. To address the distribution gap during testing, we propose a novel TTA method named Covariance-Aware Feature alignment (CAFe). We empirically show that CAFe outperforms prior TTA methods on image corruptions, including multiple types of corruptions.
Correlation and Foreground Attention to Improve Object Detection

Object Detection (OD) can be viewed as a multi-objective task to achieve object localization and class recognition. With the rapid development of the deep neural networks (DNNs), on the one hand, the performance of OD has been significantly improved by relying on the high-quality feature extraction and representation of DNNs. On the other hand, it can be challenging to accurately detect and recognize objects with non-salient or confusing features. In this paper, we propose an efficient and pluggable OD method by using attention mechanism to solve these issues from two aspects. Firstly, we exploit the semantic relationship between objects as a prior knowledge to reduce the incorrect recognition of objects with confusing features, where the relationship is encoded as an attention map by using a graph convolutional network, and then this attention map is used to reweight the feature intensities of objects belonging to different classes. Then, based on the feature map extracted from DNNs, we extract a sub-feature map containing foreground information and use this map to generate foreground attention map to improve the feature saliency of the objects. The qualitative and quantitative experimental results well verify the effectiveness of our method.
Context-Aware Inpainter-Refiner for Skeleton-Based Human Motion Completion

In this paper, we propose a context-aware inpainter-refiner (CAIR) framework for skeleton-based human motion completion, which aims to restore the damaged area of a given corrupted sequence in spatio-temporal domain. Conventional methods usually utilize convolutional neural networks to inpaint the corrupted sequence treated as a image. In comparison, we devise a two-stage "inpainting-refining" framework and further design a context-aware graph convolutional module (CA-GCM), which employs graph convolutional operation and mask-aware residual connection to update values of corrupted area. In this way, our CAIR is capable of mapping and associating intact and corrupted joints implicitly, and thus optimizes the result generated by our inpainting network effectively. Extensive experimental results on NTU60 and Human3.6M datasets demonstrate that our method outperforms our own baseline and previous methods.
Unsupervised Domain Adaptation with Imbalanced Character Distribution for Scene Text Recognition

Recent deep learning based methods have demonstrated promising results in scene text recognition. One of the major difficulty is the lack of manually annotated data. Synthetic data are then used to eliminate the requirement for human annotation. However, the domain gap between synthetic and real-world data remains a challenging issue. To bridge the gap, unsupervised domain adaptation (UDA) was introduced to transfer knowledge from a labeled source domain to a target domain. In this work, we introduce an unsupervised domain adaptation method based on a sequence-to-sequence attention model. We take into account imbalanced distribution of characters to optimize the adaptation process. We propose to use focal loss as the classification loss for the labeled source domain and focal entropy as the entropy loss for the unlabeled target domain. Our proposed method, named ICD-DA, outperforms other UDA methods on official benchmarks.
HQRetouch: Learning Professional Face Retouching Via Masked Feature Fusion and Semantic-Aware Modulation

Face retouching is a crucial technique for many consumer-level products. The goal of face retouching is to remove skin imperfections and preserve facial details simultaneously. However, it usually requires tedious manual work to achieve professional retouching effect. With the advent of Deep Neural Networks (DNNs), some methods were recently proposed to complete the task of face retouching automatically by using DNNs. They divide a portrait photo into local patches and train a DNN for face retouching. Although they can produce professional results automatically, there are still some limitations. Firstly, the network architecture fails to preserve sufficient facial details. Secondly, the facial semantic information is ignored when dividing a photo into some local patches. In this paper, we propose a novel method to solve these limitations. We first introduce the Masked Feature Fusion (MFF) module to a UNet, enabling the network to better preserve details in facial regions. Then, we exploit the semantic information by the Semantic-Aware Modulation (SAM) module, further boosting the retouching performance. Experiments on the recent public dataset Flickr-Faces-HQ-Retouched (FFHQR) demonstrate the effectiveness of our method. The code will be released.
Distributed Control for Teams of Non-holonomic Mobile Robots Executing Competitive Tasks

Multi-robot systems have been widely used in a variety of applications to perform tasks cooperatively. A greater challenge is to design control when two teams of robots have to compete with each other when performing tasks. This paper presents a control algorithm for teams of differentially-driven mobile robots that perform such a task. The goal of each team is to follow an individual reference trajectory keeping the desired shape of the formation. In the transition state, the robots of one group must penetrate the other team to reach desired poses. Artificial potential functions are used to avoid collisions. They are shaped to reduce the risk of deadlocks between groups of robots. Numerical simulations illustrate the effectiveness of the proposed algorithm.
Learned Image Compression with Multi-Scan Based Channel Fusion

While classical compression standards heavily rely on scan methods, few learned compression methods tend to use scan. The lack of scan has led to an increase in the encoding bits and the loss of relative information. Besides, recent works also struggle to choose between convolutional neural networks and transformers due to the presence of both local redundancy and global semantic information. CNN methods raise the distortion problem while the transformers increase additional encoding bits. To solve these problems, in this paper, we novelly introduce the Multi-Scan based Channel Fusion (MSCF) to the hyper-prior VAE structure to reduce the redundant bits. We propose a novel Residual Local-Global Consolidation (RLGC) module that utilizes the latest ConvNeXt and Swin transformer to enhance the image quality without additional bits. Experiments have shown that our model outperforms the state-of-the-art methods in terms of PSNR, MS-SSIM and BD-rate.
Feature Fusion Enhanced Super Resolution for Low Bitrate Screen Content Compression

Screen content image/video has been widely applied into cloud game, AR/VR, online education etc., while the transmission and storage of screen content data is limited, and existing compression methods tend to lose details when encoding screen content with low bitrate. To address this challenge, a low bitrate screen content compression method based on super-resolution is proposed in this paper. The method combines super-resolution (SR) and down-sampling techniques to compress screen content while preserving its details. A novel super-resolution network is designed, which enhances detail preservation during feature fusion and eliminates compression distortion artifacts. The proposed method was evaluated against the state-of-the-art HEVC-SCC and can save bit rate by 19.77% and 26.18% on the SCID and SIQAD datasets, respectively while maintaining similar subjective quality.
Uncertainty Aware Implicit Image Function for Arbitrary-Scale Super-Resolution

The most integral step in computer vision and image processing tasks is the representation of images. Recently, continuous image parameterization using Implicit Neural Representations (INR) has shown a great advantage over discrete representations due to their spatial invariance. This property immediately finds application in the context of single-image super-resolution (SISR) at an arbitrary scale. However, most super-resolution models, including the INR-based Local Implicit Image Function (LIIF), produce only a single output, failing to address the ill-posedness of SISR. Moreover, these models tend to optimize a mean-squared-error (MSE) based loss function which causes blurring and structural distortion in regions exhibiting a high degree of variance (details). Our work proposes a novel uncertainty-aware self-supervised methodology (U-LIIF) that extends on LIIF, to reduce the blurriness and deals with the ill-posedness of SISR. Our U-LIIF does not require any re-training and yields diversified high-resolution images by leveraging model uncertainty. The efficacy of the proposed method is validated by substantial experiments on various benchmark datasets.
A Joint Model-Driven Unfolding Network for Degraded Low-Quality Color-Depth Images Enhancement

Inspired by multi-task learning, degraded low-quality color-depth images enhancement tasks are transformed as a joint color-depth optimization model by using maximum a posteriori estimation. This model is optimized alternatively in an iterative way to get the solutions of CGD-SR task and Low-Brightness Color Image Enhancement (LBC-IE) task. The whole iterative optimization procedure is expanded as a joint model-driven unfolding network. Many experimental results have confirmed that high-resolution reconstruction of the depth map and the enhancement of low-brightness image can be realized simultaneously in one network. Furthermore, the proposed method with network interpretability can exceed that of many inexplicable CGD-SR methods and LBC-IE methods.
Measure4dhand: Dynamic Hand Measurement Extraction from 4D Scans

Hand measurement is vital for hand-centric applications such as glove design, immobilization design, protective gear design, to name a few. Vision-based methods have been previously proposed but are limited in their ability to only extract hand dimensions in a static and standardized posture (open-palm hand). However, dynamic hand measurements should be considered when designing these wearable products since the interaction between hands and products cannot be ignored. Unfortunately, none of the existing methods are designed for measuring dynamic hands. To address this problem, we propose a user-friendly and fast method dubbed Measure4DHand, which automatically extracts dynamic hand measurements from a sequence of depth images captured by a single depth camera. Firstly, the ten dimensions of the hand are defined. Secondly, a deep neural network is developed to predict landmark sequences for the ten dimensions from partial point cloud sequences. Finally, a method is designed to calculate dimension values from landmark sequences. A novel synthetic dataset consisting of 234K hands in various shapes and poses, along with their corresponding ground truth landmarks, is proposed for training the proposed methods. The experiment based on real-world data captured by a Kinect illustrates the evolution of the ten dimensions during hand movement, while the mean ranges of variation are also reported, providing valuable information for the hand wearable product design. (The video abstract is available here.)
High Dynamic Range Image Tone Mapping Based on Layer Decomposition and Image Fusion

Common displays have limited dynamic range and therefore cannot support direct reproduction of high dynamic range (HDR) images. Generally, this gap can be effectively solved by image tone mapping. One of the typical tone mapping methods is known as layer decomposition based algorithms. They decompose an HDR image into base and detail layers, and reduce the dynamic range by employing gamma functions on the base layer. However, simple gamma curves are not sufficient to produce good local contrasts. This paper proposes a layer decomposition based method by utilizing image fusion techniques to enhance local contrasts. Our method constructs virtual images to stretch dark, normal and bright regions of the base layer respectively and fuses them together for contrast enhancement. Experimental results validate that the proposed method produces enhanced local contrasts and more natural colors in the tone-mapped images.
Unknown Class Feature Transformation for Open Set Domain Adaptation Without Source Data

Utilizing deep neural network for Domain Adaptation (DA) has made great progress on learning knowledge from source domain to solve tasks in other relevant target domains. However, conventional DA methods have several restrictions: first, the category set from the source and target domain should be identical; second, during the training process data from different domains are fed into the network simultaneously, which may not be practical in real-world applications. Consequently, we aim at tackling the Source Free Open Set Domain Adaptation scenario: source and target data cannot meet with each other and the target domain contains exclusive unknown classes. Specifically, we propose a method enhancing the unknown class identification ability by synthesizing unknown class data: a feature modifier with multiple Gated Recurrent Units (GRU) is designed to learn and modify key features from the input data in order that the modified data ’looks like’ the real unknown class class data. Both real and synthesis data are used to train the classifier such that it can identify each class correctly. We evaluate our method in multiple benchmarks and the proposed framework outperforms other methods in comparison.
Automated Diagnosis of Breast Cancer Using Deep Learning-Based Whole Slide Image Analysis of Molecular Biomarkers

Breast cancer is a prevalent and diverse type of cancer that exhibits unique clinicopathologic characteristics, making the correct identification of its subtype critical to providing targeted treatment and increasing survival rates. This identification process involves testing for the presence of four key molecular biomarkers, namely estrogen receptor (ER), progesterone receptor (PR), human epidermal growth factor receptor 2 (HER2), and antigen Ki67. For accurate diagnosis ,the expertise of a pathologist and immunohistochemistry is required. To overcome this diagnostic challenge, we present a novel approach based on a deep learning pipeline for automated classification. Our approach can detect tumor and non-tumoral regions of the HER2 biomarker. Our deep learning framework comprises a Dense Convolutional Network (DenseNet), which process whole slide images (WSIs) of breast tissues, dividing them into patches for input into the network. Moreover, our approach provides both patchwise and pixelwise classification and analyzes ten WSIs of breast cancer histology. Our proposed approach generates an image map that classifies slide images on the pixel-level, detecting the status of hormone HER2 receptor as either positive or negative. The obtained results show that our deep learning-based approach has the potential to enhance the pathologist’s capabilities in diagnosing histopathological images with automated classification.
Disparity Error in Advanced Vision Sensors

This paper reports research in which feature-based disparity error is reported against corresponding features identified in a subset of images in the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) stereo 2012 benchmark. The dataset provides real-world imagery. The theory behind finding corners or features, matching them across image pairs, and then calculating the resultant disparity is explained. These disparity results are compared against disparity data provided by the KITTI stereo 2012 dataset, to produce disparity errors. The disparity errors are graphed. Concluding remarks are discussed.
Generalized Gaussian Distribution with Augmented Pure Quaternion Random Variable

Quaternion random signals can be analyzed using augmented second-order statistics. If these signals are treated as the augmented quaternion valued random variables, then they can be modeled with the quaternion generalized Gaussian probability density function. In the case of pure quaternions, this probability density function becomes a limiting case. Therefore, for this case, this article introduces the dedicated generalized Gaussian distribution of an augmented quaternion random variable for pure quaternions.
EFNQL: An Efficient Method Based On Q-learning And Fuzzy Neural Networks In Robocup Soccer Simulation

Robocup Soccer Simulation is considered to be one of the world’s most prestigious and large-scale robotics events, attracting researchers from all over the world. In Robocup Soccer Simulation 2D, the challenge has always been to organize intelligent agents to collaborate and analyze scenarios in order to develop more effective strategies for launching shots and dribbling. To address this challenge, an efficient algorithmic architecture called EFNQL has been developed that utilizes fuzzy control, fuzzy neural networks, and Q-learning to optimize the strategy of the agents in Robocup Soccer Simulation competitions. The architecture first establishes fuzzy rules using fuzzy control theory, then processes more complex information on the playing field using fuzzy neural networks, and finally completes fuzzy reasoning by applying Q-learning related theory for learning. Through extensive experimentation, this thesis found that the scoring rate and ball control rate were significantly improved, ultimately resulting in a second-place finish at the 2022 RoboCup China Open.
Make Your Conference Secure: A Robust Method to Replace Private Backgrounds

The visual conference has become one of the important parts of our daily life as the result of the worldwide pandemic. People will attend the increasing number of visual conferences with their colleagues, co-workers, or even strangers, so they are more and more concerned about how to protect their privacy when they have these meetings at home or in other personal places. To dispel people’s misgivings, the most efficient way is to hide the environmental information around attendees, or in other words, is to replace the background containing the private information with some online pictures. So based on this motivation, we propose a robust method to help people protect their private information by replacing the background with some non-privacy pictures. The proposed method, Portrait Segmentation Network(PSN), can efficiently use internet images to substitute the private background. In addition, the lack of a suitable dataset for segmentation in the visual conference scenario is another challenge. Therefore, we utilize a training method to train our proposed model by using images instead of videos in training. Meanwhile, we use the adequate experiments on EG1800 [1] and a new dataset formed by Supervisely [2] to prove the robustness of our proposed method.
Learning Mutually in Crowd Scenes for Pedestrian Detection

Pedestrian detection in crowded scenes is a challenging problem due to the diverse occlusion patterns and highly overlap. To tackle this critical problem, we propose a mutual learning detection network. First, a self-attention mechanism is proposed to achieve mutual learning between individuals by capturing similar semantics among pedestrians. Feature representation of occluded individuals is enhanced by locally fusing similar semantics. Second, mutual loss is designed to improve the consistency of regression and classification. Specifically, regression results are leveraged to make classification score aware of the quality of predicted boxes, and the classification scores help the regression head to accelerate convergence of redundant boxes. Finally, we evaluate our proposed method on MOT20 and CityPersons datasets and achieve comparable state-of-the-art performance using less data. Compared to baseline, our detector obtains 14.4% AP and 11.7% AR gains on challenging MOT20 dataset.
GTLMA: Generalizable Hierarchical Learning for Tasks with Variable Entities

The current ALlocator-Actor Multi-Agent Architecture (ALMA) has shown convincing results on some complex composite tasks. The two-level decision-making process used by ALMA, along with a high-level subtask allocation strategy and low-level agent policies, is the secret to its success. Although ALMA’s modularity allows it to generalize to different environment settings to some level, we believe there is still opportunity for improvement in this area. Due to the intrinsic property of ALMA, the fixed number of entities in the model architecture for a specific task, it may even get stuck in scaling to similar task which only differs in the number of entities. Therefore, we introduce a new module called Knowledge Container, which has a significant effect on improving the performance and generalizability of agents. We demonstrate our method can obtain and keep the considerable performance without any re-training or fine-tuning.
A Directed Jump Point Search with Improved Preprocess for Path Planning

We introduce a new algorithm called directed JPS+(Jump Point Search Plus) that addresses the large memory overheads of JPS+ and improves its performance in cases where the obstacle scale is small. Our approach involves two main modifications. First, we reduce memory usage by recording only jump points in the preprocessing stage instead of eight labels for each nodes in JPS+. And we introduce a technique to connect these precomputed jump points into jump point set. Second, we apply an optimization to JPS+ that speeds up path planning in maps with small obstacle scale and it has no negative impact on other cases. We implement our algorithm in Python and evaluate its performance in terms of memory usage and planning time. Our results demonstrate that it saves memory and speeds up the path planning, particularly in maps with small obstacle scale.