Reviewerly/exHarmony huggingface dataset:
OEST: Outlier Exposure by Simple Transformations for Out-of-Distribution Detection

Although the previous works for out-of-distribution(OOD) detection have achieved great improvements, they are still highly dependent on the specific selection of the outliers from external datasets or that transformed by certain data augmentations, and hence cannot be applied in a wide range of domains. To solve this problem, in this paper, we propose a simple, yet effective method called Outlier Exposure by Simple Transformations (OEST), which aims at exposing the outliers by the composition of several simple transformations of data augmentations via energy score. In addition, our training scheme can make full use of nearly all considered data augmentations in previous works, even though some of them are generally regarded as useless. And we also find that, for simple data augmentation, our training scheme is less time-consuming and better in performance than relative works. Furthermore, our experiments validate that our method outperforms the state-of-the-art methods.
Cooperative Navigation of Differential-Drive Mobile Robots in Crowd Environments

Mobile robotics has gained a lot of attention due to its wide range of applications in various domains such as warehouses, factories, and hospitals. In crowded environments, cooperative navigation is essential to ensure the safe and efficient movement of mobile robots. The cooperative navigation strategies must enable robots to coordinate their movements and adapt to the dynamic environment. In recent years, significant progress has been made in developing cooperative navigation algorithms for mobile robots. The algorithms can be grouped into centralised and decentralised methods, each with its advantages and disadvantages. However, most deep-learning-based methods have been designed for holonomic robots, while differential-drive mobile robots pose more challenges due to their limited degrees of freedom. The paper presents a method that uses deep reinforcement learning to enable differential-drive mobile robots to navigate cooperatively in crowded environments. The method employs a cooperation module to model the interactions among the robot and other agents, and a self-attention mechanism to prioritize the interaction features. The experimental outcomes show that the method outperforms several existing algorithms, which obtains better success rate and lower collision rate, indicating its effectiveness in guiding robots to their destinations while avoiding collisions.
Nonlinear, adaptive model-following control using numerical inverse of a fuzzy model

A novel, adaptive, nonlinear model-following control using an on-line, numerical inversion of a fuzzy model is proposed. We consider a nonlinear plant (slave system) with unknown parameters and implicit, nonlinear dependance of control and state variables. The plant is supposed to follow a nonlinear master system. The appropriate function of control and state of the plant is represented by a fuzzy model, not necessarily being a one-to-one map. The model is numerically inverted on-line to get the desired control value. Consequent functions of the model are changed by robust adaptive laws, to cope with any modelling inaccuracies, while the other adaptive parameters are changed to follow the unknown parameters of the plant. This approach achieves a short execution time and can be applied in standard DSP-based controllers.
Adaptive Graph Convolution Module for Salient Object Detection

Salient object detection (SOD) is a task that involves identifying and segmenting the most visually prominent object in an image. Existing solutions can accomplish this using a multi-scale feature fusion mechanism to detect the global context of an image. However, as there is no consideration of the structures in the image nor the relations between distant pixels, conventional methods cannot deal with complex scenes effectively. In this paper, we propose an adaptive graph convolution module (AGCM) to overcome these limitations. Prototype features are initially extracted from the input image using a learnable region generation layer that spatially groups features in the image. The prototype features are then refined by propagating information between them based on a graph architecture, where each feature is regarded as a node. Experimental results show that the proposed AGCM dramatically improves the SOD performance both quantitatively and quantitatively.
Visual Explanation of Object Detectors via Saliency Maps

In recent years, rapid progress of deep learning has resulted in object detection algorithms based on deep neural networks nearly completely replacing traditional methods. This shift has led to significant improvements in both accuracy and speed of object detection. However, deep neural networks that support decision-making inobject detection are complex black boxes with hidden internal logic and operations. This lack of transparency results in deep learning models being unexplainable, which limits their practical deployment. In this paper, we analyze the current state and future trends of interpretable technology in object detection and propose an effective object detection interpretable algorithm. This algorithm aims to explain the underlying logic of AI decision-making and provide support for the high performance and credibility of object detection algorithms based on deep neural networks through quantitative evaluation indicators.
BITS-Net: Blind Image Transparency Separation Network

This research presents a new approach for blind single-image transparency separation, a significant challenge in image processing. The proposed framework divides the task into two parallel processes: feature separation and image reconstruction. The feature separation task leverages two deep image prior (DIP) networks to recover two distinct layers. An exclusion loss and deep feature separation loss are used to decompose features. For the image reconstruction task, we minimize the difference between the mixed image and the re-mixed image while also incorporating a regularizer to impose natural priors on each layer. Our results indicate that our method performs comparably or outperforms state-of-the-art approaches when tested on various image datasets.
Asymmetric Scalable Cross-Modal Hashing

Cross-modal hashing is a practical approach to solving the problem of large-scale multimedia retrieval. However, there are still specific issues that the current methods cannot solve, such as how to construct binary codes rather than relax them to continuity effectively and how to prevent n × n problem. This paper proposes a novel Asymmetric Scalable Cross-Modal Hashing (ASCMH) to address these issues. It learns a common latent space from the kernelized features of different modalities. It then transforms the similarity matrix optimization to a distance-distance difference minimization problem with the help of semantic labels and common latent space. Additionally, we use an orthogonal constraint of label information to construct hash codes necessary for search accuracy. Extensive experiments on three benchmark datasets show that our ASCMH outperforms the SOTA cross-modal hashing methods.
A Contrastive Learning Approach for Screenshot Demoiréing

We propose a contrast learning-based approach for screenshot demoiréing based on the assumption that a moiré image can be separated into two layers in deep latent space: moiré artifacts and latent clean image. First, we develop a multiscale network, called SDN, that extracts multiscale feature maps of an input image and then separates them into moiré and clean image components. To improve the separation of the features, we develop a contrast learning approach that separates and clusters moiré and clean image features in the latent space in supervised and unsupervised manners, respectively. Experimental results on a misaligned real-world screenshot dataset show that the proposed algorithm provides better demoiréing performance than state-of-the-art algorithms.
JPEG Compliant Compression for DNN Vision

Conventional image compression techniques are mostly developed for the human visual system. However, with the extensive use of deep neural networks (DNNs), more and more images will be consumed by DNN-based intelligent machines, which makes it crucial to develop image compression techniques customized for DNN vision while being JPEG compliant. In this paper, we first propose a new distortion measure, dubbed the sensitivity weighted error (SWE). Then, we develop OptS, a DNN-oriented compression algorithm with full JPEG compatibility, which designs optimal quantization tables for DNN models based on SWE. To test the performance of our algorithm, experiments of image classification are conducted on the ImageNet dataset for two prevailing DNN models. Results demonstrate that our algorithm achieves better rate-accuracy (R-A) performance than the default JPEG. For some DNN model, the compression ratio of our algorithm can reach 8.3× <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> , reducing the compression rate (bits per pixel, bpp) of the default JPEG by 57.4% with no accuracy loss. Our source code is available at https://github.com/zkxufo/OptS.git.
Lite-HRNet Plus: Fast and Accurate Facial Landmark Detection

Facial landmark detection is an essential technology for driver status tracking and has been in demand for real-time estimations. As a landmark coordinate prediction, heatmap-based methods are known to achieve a high accuracy, and Lite-HRNet can achieve a fast estimation. However, with Lite-HRNet, the problem of a heavy computational cost of the fusion block, which connects feature maps with different resolutions, has yet to be solved. In addition, the strong output module used in HRNetV2 is not applied to Lite-HRNet. Given these problems, we propose a novel architecture called Lite-HRNet Plus. Lite-HRNet Plus achieves two improvements: a novel fusion block based on a channel attention and a novel output module with less computational intensity using multi-resolution feature maps. Through experiments conducted on two facial landmark datasets, we confirmed that Lite-HRNet Plus further improved the accuracy in comparison with conventional methods, and achieved a state-of-the-art accuracy with a computational complexity with the range of 10M FLOPs.
Improving Learned Invertible Coding with Invertible Attention and Back-Projection

Learned image compression (LIC) is developing rapidly. Due to the high-frequency information preservation property of reversible mapping, invertible neural networks (INNs) have been applied to the transformation of LIC and successfully surpassed the latest classical coding standard Versatile Video Coding (VVC) in rate-distortion performance. However, the nonlinearity of the used INNs is limited and the dimensionality reduction method before quantization is not refined enough, respectively resulting in redundancy and information loss. We use attention instead of convolution in the coupling layers of INNs to improve information extraction while maintaining reversibility. In addition, inspired by Back-Projection (BP), we design a BP mechanism in the dimensionality adjustment to reduce information loss. Combined with the advanced channel-wise autoregressive entropy model, our model achieves significant performance improvement compared to the original model and surpasses the SOTA transformation model in the high bitrate range.
Viva: a Variational Image Vectorization Algorithm on Dual-Primal Graph Pairs

We propose a novel variational image vectorization algorithm (VIVA) which alternatively smooths contours by affine shortening flow and eliminates spurious regions by minimizing a Mumford-Shah-type functional. We introduce dual-primal graphs representing domain partitions which allows for effective iterative computation. The method provides varying levels of simplicity on the topology of the resulted vector graphics while effectively removing pixelization. It compares favorably to the state-of-the-art (SOTA) vectorization methods.
Data Augmentation Using Corner CutMix and an Auxiliary Self-Supervised Loss

Deep convolutional neural networks (CNNs) have achieved remarkable success in computer vision tasks, but their training is susceptible to overfitting when the training sample size is insufficient. In this paper, we introduce Corner CutMix, a novel data augmentation technique for CNN training. During training, Corner CutMix randomly selects a region from one of four corner areas in an image and replaces it with a randomly chosen region from a distractor image. Additionally, we design an auxiliary self-supervised loss function to learn the position of the selected corner region, thereby improving the transferability and generalizability of the learned representation. Corner CutMix is easy to implement, adding little computational overhead, and can be combined with other augmentation methods such as random cropping, color distortion, and flipping. Our extensive classification task experiments in self-supervised learning on public datasets (e.g., CIFAR10, CIFAR100, and STL10) demonstrate the effectiveness of Corner CutMix, which consistently outperforms strong baselines such as CutOut and CutMix.
Adaptive Camouflage Pattern Generation to Different Environments Via Content-Aware Style Transfer

Visual camouflage is an effective means of protecting valuable assets that are vulnerable to theft, espionage, or other forms of malicious activity. To overcome the limitation of standardized camouflage patterns in certain environments, we need an innovative approach that adapts the camouflage pattern to the specific surroundings of the asset to be concealed. In this paper, we present a novel camouflage image generation method whose results change by the circumstances around the asset to be concealed using the style transfer. In order to remove the influence of stuff and objects that are not advantageous for camouflage in the surrounding, we propose a novel mechanism that introduces the contents-aware information into the calculation of style representation. Experimental results in various situations, including the snowy natural scene, show that the proposed method provides excellent adaptive camouflage outcomes while effectively suppressing conspicuous elements in the surrounding.
Underwater Image Enhancement Based on Improved U-Net Convolutional Neural Network

Aiming at the problems of low contrast, dim and color distortion of underwater images caused by the attenuation and scattering of light in water propagation, an underwater image enhancement algorithm, which is based on a U-Net convolutional Neural Network is proposed. The algorithm consists of an image synthesis module and an image enhancement module. Firstly, a style transfer network based on transfer learning is built and employed to synthesize underwater images from clear images, image pairs are composed of these synthesized and original images for data expansion. Secondly, an underwater image enhancement network with a U-shaped convolutional variational autoencoder is constructed, and the image pairs serve as input to the second module for training. The qualitative and quantitative analysis results show that the proposed algorithm has good performs favorably in the color fidelity and detailed reservation of the target object.
Can we Distill Knowledge from Powerful Teachers Directly?

Knowledge distillation efficiently improves a small model’s performance by mimicking the teacher model’s behavior. Most existing methods assume that distilling from a large and accurate teacher model leads to better student models. However, several studies show the difficulty of distillation from large teacher models and opt for heuristics to address this. In this work, we demonstrate that large teacher models can still be effective in knowledge distillation. We show that the spurious features learned by large models are the cause of difficulty in knowledge distillation for small students. To overcome this issue, we propose employing ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> regularization to prevent teacher models from learning an excessive number of spurious features. Our method alleviates the poor learning for small students when there is a significant disparity in size between the teachers and students. We achieve substantial improvement on various architectures, e.g., ResNet, WideResNet, and VGG, on diverse datasets, including CIFAR-100, Tiny-ImageNet, and ImageNet.
Soft-IntroVAE for Continuous Latent Space Image Super-Resolution

Continuous image super-resolution (SR) recently receives a lot of attention from researchers, for its practical and flexible image scaling for various displays. Local implicit image representation is one of the methods that can map the coordinates and 2D features for latent space interpolation. Inspired by Variational AutoEncoder, we propose a Soft-introVAE for continuous latent space image super-resolution (SVAE-SR). A novel latent space adversarial training is achieved for photo-realistic image restoration. To further improve the quality, a positional encoding scheme is used to extend the original pixel coordinates by aggregating frequency information over the pixel areas. We show the effectiveness of the proposed SVAE-SR through quantitative and qualitative comparisons, and further, illustrate its generalization in denoising and real-image super-resolution.
Processing Energy Modeling For Neural Network Based Image Compression

Nowadays, the compression performance of neural-network-based image compression algorithms outperforms state-of-the-art compression approaches such as JPEG or HEIC-based image compression. Unfortunately, most neural-network based compression methods are executed on GPUs and consume a high amount of energy during execution. Therefore, this paper performs an in-depth analysis on the energy consumption of state-of-the-art neural-network based compression methods on a GPU and show that the energy consumption of compression networks can be estimated using the image size with mean estimation errors of less than 7%. Finally, using a correlation analysis, we find that the number of operations per pixel is the main driving force for energy consumption and deduce that the network layers up to the second downsampling step are consuming most energy.
Implicit Attention-Based Cross-Modal Collaborative Learning for Action Recognition

Human action recognition is an active research topic in recent years. Multiple modalities often convey heterogeneous but potentially complementary action information that single modality does not hold. Some efforts have been resoted to explore cross-modal representation to promote the modeling capability, but with limited improvement due to the simple fusion of different modalities. To this end, we propose an impliCit attention-based Cross-modal Collaborative Learning (C3L) for action recognition. Specifically, we apply a Modality Generalization network with Grayscale enhancement (MGG) to learn specific modality representation and interaction (infrared and RGB). Then, we construct a unified representation space through the Uniform Modality Representation module (UMR), which preserves the modality information while enhancing the overall representation ability. Finally, feature extractors adaptively leverage modality-specific knowledge to realize cross-modal collaborative learning. Extensive experiments conducted on three widely-used public benchmarks InfAR, HMDB51, and UCF101, demonstrate the effectiveness and strength of our proposed method.
Robust RGB-T Tracking via Consistency Regulated Scene Perception

RGB-T tracking has received increasing attention due to its significant advantage under severe weather conditions. Existing RGB-T tracking methods pay close attention to the representation of target appearance, ignoring the importance of scene information. In this paper, we propose a global reasoning-oriented method for RGB-T tracking. In particular, within a multi-task learning framework, our approach adopts a nested global reasoning model to regulate the consistency of scene perception (reasoning the relation between targets and the surrounding semantic regions) in different image domains. Moreover, a meta-unsupervised learning strategy is designed to enforce the nested global reasoning model to utilize partial multi-domain target information for the updating of scene perception. Extensive experiments on GTOT, RGBT210 and LasHeR datasets show the superior performance of our method when compared with related works.
Revisiting Modality Imbalance In Multimodal Pedestrian Detection

Multimodal learning, particularly for pedestrian detection, has recently received emphasis due to its capability to function equally well in several critical autonomous driving scenarios such as low-light, night-time, and adverse weather conditions. However, in most cases, the training distribution largely emphasizes the contribution of one specific input that makes the network biased towards one modality. Hence, the generalization of such models becomes a significant problem where the non-dominant input modality during training could be contributing more to the course of inference. Here, we introduce a novel training setup with regularizer in the multimodal architecture to resolve the problem of this disparity between the modalities. Specifically, our regularizer term helps to make the feature fusion method more robust by considering both the feature extractors equivalently important during the training to extract the multimodal distribution which is referred to as removing the imbalance problem. Furthermore, our decoupling concept of output stream helps the detection task by sharing the spatial sensitive information mutually. Extensive experiments of the proposed method on KAIST and UTokyo datasets shows improvement of the respective state-of-the-art performance.
Multi Hybrid Extractor Network for 3D Human Pose Estimation

Monocular image or video based 3D human pose estimation remains a very challenging task because of depth ambiguity and occluded joints. To relieve this limitation, we propose a Multiple Hybrid Extraction Network (MHENet), which obtains three different representations of pose hypotheses features by multiple hybrid extractors with different structures, and uses pose interaction and fusion to obtain accurate 3D pose. The Hybrid Extraction Module obtains three hypotheses features: base features correspond to structural information, diverse features correspond to detail information, and condensed features correspond to action information. Hypotheses Interaction Fusion Modul builds relationships across hypotheses feature to generate more accurate 3D poses. Extensive qualitative and quantitative experimental results on a large-scale publicly available dataset demonstrate that our approach achieves competitive performance compared to state-of-the-art methods. The code will be made publicly.
The First Comprehensive Dataset with Multiple Distortion Types for Visual Just-Noticeable Differences

Recently, with the development of deep learning, a number of Just Noticeable Difference (JND) datasets have been built for JND modeling. However, all the existing JND datasets only label the JND points based on the level of compression distortion. JND models learned from such datasets can only be used for image/video compression. Hence, a generalized JND modeling should take more kinds of distortion types into account. To benefit JND modeling, this work establishes a generalized JND dataset with a coarse-to-fine JND selection, which contains 106 source images and 1,642 JND maps, covering 25 distortion types. To this end, we proposed a coarse JND candidate selection scheme to select the distorted images from the existing Image Quality Assessment (IQA) datasets as JND candidates instead of generating JND maps ourselves. Then, a fine JND selection is carried out on the JND candidates with a crowdsourced subjective assessment.