Reviewerly/exHarmony huggingface dataset:
Dynamic Dual-Graph Fusion Convolutional Network for Alzheimer’s Disease Diagnosis

In this paper, a dynamic dual-graph fusion convolutional network is proposed to improve Alzheimer’s disease (AD) diagnosis performance. The following are the paper’s main contributions: (a) propose a novel dynamic Graph Convolutional Network (GCN) architecture, which is an end-to-end pipeline for diagnosis of the AD task; (b) the proposed architecture can dynamically adjust the graph structure for GCN to produce better diagnosis outcomes by learning the optimal underlying latent graph; (c) incorporate feature graph learning and dynamic graph learning, giving those useful features of subjects more weight while decreasing the weights of other noise features. Experiments indicate that our model provides flexibility and stability while achieving excellent classification results in AD diagnosis.
Micro-Expression Recognition with Layered Relations and More Input Frames

Micro-expressions (MEs) which are types of spontaneous facial movements, are difficult to recognize due to their short duration and low intensity. Recent ME recognition methods typically depend on spatio-temporal features around the eyebrow and mouth regions where MEs occur. And these features are often extracted from the onset and apex frames in which the intensities of MEs are considered to be zero and high respectively. In this paper, we improve the effectiveness of the spatio-temporal features by proposing a two-layer encoder of Transformer to model the features’ relations. In addition, the novel recognition scheme captures the more detailed motion dynamics of MEs by employing more frames rather than the onset and apex frames. The recognition scheme is further refined by developing a graph convolution network with a trainable adjacency matrix for Action Units (AUs). Extensive experiments on multiple public datasets demonstrate that our method has better or comparable performance to SOTA methods on multiple evaluation metrics.
Efficient Joint Video Denoising and Super-Resolution

Denoising and super-resolution are two important tasks for video enhancement. Despite recent progress for each task, there are very few works that target both tasks simultaneously. In this paper, we propose an efficient noise-robust video super-resolution method that is trained end-to-end for an input video containing observable noises. We investigate current approaches to address this joint denoising and super-resolution task and compare them to our proposed method. Experimental results show that our method achieves competitive reconstruction performance with existing solutions on various datasets while maintaining a low computation cost and a small model size which prove the effectiveness of our joint model design and training. Our code is available at "https://github.com/Eventhyn/EVDSRNet.".
Evaluation of the Effectiveness Of Physical Protection Systems with Consideration of its Cyber-Resilience

The importance of security in the modern, dynamically changing world seems to be understood across the globe. Different types of organizations, both from the public and the private sector, are creating strategies and investing large amounts of funds to implement Physical Protection Systems (PPS) in order to protect their assets. It appears relatively obvious that assessment of the effectiveness of PPS is important and required. Moreover, the result of the assessment should be reliable and cover as much as possible current threats, including those that are coming from cyberspace. In this paper, we propose a new approach to evaluating PPS. Our proposition as a starting point is using well-known EASI (Estimate of Adversary Sequence Interruption) methodology but reinforced with methods aiming to cover the cybersecurity of PPS considering all potential adversary paths.
Flow-Based One-Class Anomaly Detection with Multi-Frequency Feature Fusion

Anomaly detection in computer vision seeks to identify samples outside of a predefined distribution, including texture defect detection and semantic anomaly detection. However, existing methods are difficult to simultaneously achieve high performance for both types of anomaly detection. To address this issue, we propose a new flow-based anomaly detection method. Firstly, we use semantic features extracted from a pre-trained backbone to learn the distribution of normal data from a semantic perspective. Secondly, we introduce a multi-frequency feature fusion module to aggregate semantic and texture information, which substantially improves performance for both types of anomaly detection at the same time. Extensive experiments on multiple well-known datasets demonstrate that our proposed method performs well in both types of anomaly detection, specially, achieves state-of-the-art performance in one-class anomaly detection. The codes will be available at https://github.com/SYLan2019/FOAD-MFFF.
MENAS: Multi-trial Evolutionary Neural Architecture Search with Lottery Tickets

Neural architecture search (NAS) has brought significant progress in recent image recognition tasks. Most existing NAS methods apply restricted search spaces, which limits the upper-bound performance of searched models. To address this issue, we propose a new search space named MobileNet3-MT. By reducing human-prior knowledge in omni dimensions of networks, MobileNet3-MT accommodates more potential candidates. For searching in this challenging search space, we present an efficient Multi-trial Evolution-based NAS method termed MENAS. Specifically, we accelerate the evolutionary search process by gradually pruning models in the population. Each model is trained with an early stop and replaced by its Lottery Tickets (the explored optimal pruned network). In this way, the full training pipeline of cumbersome networks is prevented and more efficient networks are automatically generated. Extensive experimental results on ImageNet-1K, CIFAR-10, and CIFAR-100 demonstrate that MENAS achieves state-of-the-art performance.
A Global-Local Contrastive Learning Framework for Video Captioning

In this paper, a global-local contrastive learning framework is proposed to leverage global contextual information from different modalities and then effectively fuse them with the supervision of contrastive learning. First, a global-local encoder is proposed to sufficiently explore the salient contextual information from different modalities, which generates the global contextual information. Second, contrastive learning is used to minimize the semantic distance between the paired modalities, which can improve the content matching between videos and the predicted captions. Finally, an attention-based multimodal encoder is presented to effectively fuse different modalities, thereby generating the multimodal representations that include global contextual information from different modalities. Extensive experimental results on benchmark datasets indicate that our proposed method is superior to the state-of-the-art approaches.
Sandwiched Video Compression: Efficiently Extending the Reach of Standard Codecs with Neural Wrappers

We propose sandwiched video compression – a video compression system that wraps neural networks around a standard video codec. The sandwich framework consists of a neural pre- and post-processor with a standard video codec between them. The networks are trained jointly to optimize a rate-distortion loss function with the goal of significantly improving over the standard codec in various compression scenarios. End-to-end training in this setting requires a differentiable proxy for the standard video codec, which incorporates temporal processing with motion compensation, inter/intra mode decisions, and in-loop filtering. We propose differentiable approximations to key video codec components and demonstrate that, in addition to providing meaningful compression improvements over the standard codec, the neural codes of the sandwich lead to significantly better rate-distortion performance in two important scenarios. When transporting high-resolution video via low-resolution HEVC, the sandwich system obtains 6.5 dB improvements over standard HEVC. More importantly, using the well-known perceptual similarity metric, LPIPS, we observe 30% improvements in rate at the same quality over HEVC. Last but not least, we show that pre- and post-processors formed by very modestly-parameterized, light-weight networks can closely approximate these results.
Reuse Non-Terrain Policies for Learning Terrain-Adaptive Humanoid Locomotion Skills

Terrain-adaptive locomotion skills are prerequisites for humanoid to traverse complex environments including barriers, gaps, caves and etc. Previous imitation learning methods require large amounts of motion sampling for an optimal policy adapted to the new terrain. However, we reveal an interesting fact: the effectiveness of such a costly policy on certain complex terrain is quite similar to that of existing policies pretrained on the flat ground. Inspired by this finding, we present a few-shot imitation learning (FSIL) framework to reuse these pretrained policies as learning primitives for new terrain adaptation. Specifically, a meta-controller, trained with a few control sequences, is proposed to allocate combination weights of each primitive for different terrains. Empirical studies in mainstream problem settings show that our method maintains a high passing rate within a few shots of new terrain while avoiding massive unnecessary data sampling. Further in-depth theoretical analysis shows that, compared to other methods that sample data in new terrain environments hundreds or thousands of times, our method requires only a maximum of 52 shots to achieve optimal control policy for certain terrain adaptation.
SDWD: Style Diversity Weighted Distance Evaluates the Intra-Class Data Diversity of Distributed GANs

Due to the distributed storage of massive data, efficient deployment of Generative Adversarial Networks (GANs) in distributed scenarios has become a hot topic. This paper analyzes the characteristics of existing distributed GANs. Moreover, it discusses the evaluation methods of these models. Through experiments, we found that Fréchet Inception Distance (FID) cannot sensitively reflect the diversity of intra-class data, and the evaluation effect of different distributed GANs is greatly affected by different datasets, so we cannot accurately rank the models. To more accurately evaluate the generation capability of distributed GANs, we propose a novel evaluation model, Style Diversity Weighted Distance (SDWD), which introduces a Siamese network to measure data similarity. We conduct experiments on two datasets, and the results prove that SDWD can sensitively reflect the intra-class diversity of data and can reliably evaluate different models and frameworks.
Weakly Supervised Disentanglement with Triplet Network

Variational Autoencoders have gained considerable attention due to their capacity of encoding high dimensional data into a lower dimensional latent space. In this context, several methods have been proposed with the objective of producing disentangled representations. In this work, we propose a weakly supervised model that explicitly disentangles the factors of variation of a dataset in separate subspaces using a pairwise architecture. We also create a framework that encourages conditional image generation according to the desired factor of variation, by controlling these subspaces. This is achieved by introducing an additional network trained with a triplet loss. Its output approximates representations of images generated from the same factor and push the ones of images generated from different factors apart. Experiments are carried out on widely used datasets, and show that our model is able to disentangle specified factors of variation, and to generate new data while constraining desired properties, even when these factors have small influence on reconstruction loss.
Arithmetic Billiard Paths Revisited: Escaping from a Rectangular Room

In this work, we consider a problem where a robot can move in a straight line inside a 2D rectangular room with integer lengths until it hits any part of the wall of the room. If the robot hits any part of a wall other than the corners or any point of an opening, then the robot bounces off the wall and follows a new direction in another straight line following the laws of symmetric reflection. The robot needs to escape through an opening on the wall that has a minimum length of one unit. The robot can only escape through the opening if it reaches any point of the opening with a non-zero angle.We present an efficient algorithm for which the robot is guaranteed to find the opening if there is any or declare that there is none. We prove that the algorithm works if and only if the sides of the rectangle are co-prime. As a by-product of our main result, we also provide some interesting results related to the coverage of the interior of the rectangle when the robot follows similar algorithms to escape from the rectangular room.
Locally Accumulated Adam For Distributed Training With Sparse Updates

The high bandwidth required for gradient exchange is a bottleneck for the distributed training of large transformer models. Most sparsification approaches focus on gradient compression for convolutional neural networks (CNNs) optimized by SGD. In this work, we show that performing local gradient accumulation when using Adam to optimize transformers in distributed fashion leads to a misled optimization direction and we address this problem by accumulating the optimization direction locally. We also empirically demonstrate most sparse gradients do not overlap and thus show that sparsification is comparable to an asynchronous update. Our experiments with classification and segmentation tasks show that our method can still maintain the correct optimization direction in distributed training event under highly sparse updates.
Geometry-Aware Video Quality Assessment for Dynamic Digital Human

Dynamic Digital Humans (DDHs) are 3D digital models that are animated using predefined motions and are inevitably bothered by noise/shift during the generation process and compression distortion during the transmission process, which needs to be perceptually evaluated. Usually, DDHs are displayed as 2D rendered animation videos and it is natural to adapt video quality assessment (VQA) methods to DDH quality assessment (DDH-QA) tasks. However, the VQA methods are highly dependent on viewpoints and less sensitive to geometry-based distortions. Therefore, in this paper, we propose a novel no-reference (NR) geometry-aware video quality assessment method for DDH-QA challenge. Geometry characteristics are described by the statistical parameters estimated from the DDHs’ geometry attribute distributions. Spatial and temporal features are acquired from the rendered videos. Finally, all kinds of features are integrated and regressed into quality values. Experimental results show that the proposed method achieves state-of-the-art performance on the DDH-QA database.
Classification Task Assisted Segmentation Network for Breast Tumor Segmentation in Ultrasound Images

Accurate and effective segmentation of breast masses plays an important role in the early stages of breast cancer treatment. However, the irregular shape of mass, the blurred mass boundary and speckle noise in breast ultrasound (BUS) images make automatic segmentation still challenging. In this paper, we propose a classification task assist (CTA) module for boosting the performance of the commonly used deep segmentation models on BUS images. This module can be easily inserted into representative segmentation models to focus on a learning task of BUS image-level classification. The incorporation of this classification learning task enable the segmentation model achieve additional supervision information to enhance the extraction of semantic context information and the localization of segmentation object. We added the CTA module to four state-of-the-art segmentation models, including codec and non-codec structures, and the experimental results show that the insertion of the CTA module into the original model results in the lowest improvement of 1.48% and the highest improvement of 4.29% in the intersection over union (IoU) metric.
Learning Torso Prior for Co-Speech Gesture Generation with Better Hand Shape

Co-speech gesture generation is the task of synthesizing gesture sequences synchronized with an input audio signal. Previous methods try to estimate upper body gesture as a whole, ignoring the different mapping relations between audio and different body parts, which leads to poor overall results especially bad hand shapes. In this paper, we propose a novel three-branch co-speech gesture generation framework to obtain better results. In particular, we propose a Torso2Hand Prior Learning module (T2HPL) to leverage torso information as an extra prior to enhance hand pose prediction, and carefully design a hand shape discriminator to improve the authenticity of generated hand shape. In addition, an arm orientation loss is designed to encourage the network to generate torso part with better semantic expressiveness. Experiments on dataset of four different speakers demonstrate the superiority of our method over the state-of-the-art approaches.
Neural Field Real-Time Transmission Using Multiple Description Coding with Random Position Sampling

Neural fields are a new signal representation and are widely used now to represent various forms of multimedia. However, due to their size, they could introduce latency when delivered over the network. A way to mitigate that is to use Multiple description coding (MDC), that provides multiple descriptions of the same content which can then be transmitted along different paths to improve reliability/efficiency. In this paper, we introduce a novel MDC framework that is based on neural fields. We first apply a randomly sampled neural field (with small model size) to generate multiple descriptions based on random initializations. We leverage the fact that neural fields are continuous functions (constructed using a multi-layer perceptron (MLP)) and use that to reconstruct the entire original content and progressively improve the quality as more descriptions are obtained. We validate the effectiveness of the proposed method by showing results on public data sets.
Cross-domain Person Re-Identification in Low Resolution Non-visual Sensor Network

Person Re-identification (RE-ID) is a computer vision technique for detecting specific persons in image or video sequences, enabling the recognition of target persons in non-overlapping camera views. This paper focuses on person re-identification under the FOVs of two infrared thermopile array sensors, which are similar to the low-resolution infrared cameras. A convolutional-recurrent neural network is constructed for closed-set identification and can achieve a cross-validation average accuracy of 91.3%. To solve the more challenging open-set identification problem, auxiliary methods such as outlier detection and manual feature extraction have been combined with neural networks, which address the limitations of neural networks in processing unknown data, leading to an average recognition accuracy of 88.6% for known classes and 83.2% for unknown class. The experimental results have demonstrated that low-resolution infrared images can be used for cross-domain person re-identification and achieve a relatively good recognition accuracy.
Online Pedestrian Tracking Using A Dense Fisheye Camera Network With Edge Computing

This paper describes a dense fisheye camera network for pedestrian tracking in an indoor environment, with the target scenario being an unmanned store. Each fisheye camera is connected to a single-board computer for local tracking using its own images. The local tracks are integrated and global tracks generated at a central computer in an online manner. The local trackers are based on the popular DeepSORT algorithm, and the global tracker combines distance and novel specialization based factors to update global tracks from local tracks, avoiding the need of matching local tracks. Experiments on a self-collected dataset demonstrate highly accurate tracking over several minutes of videos.
Restoration of Extremely Compressed Background for VCM Using Guided Generative Priors

We propose a learning-based image restoration algorithm for a single decoded image with a high-quality foreground and an extremely degraded background for video coding for machines (VCM). First, we develop an encoder that extracts multiscale features and learns latent vectors. Then, a background generator with style and feature fusion blocks generates guided features that contain the prior background information in the input image. Finally, the decoder restores the degraded background region by merging the image features from the encoder and prior background information from the generator. Experimental results show that the proposed algorithm achieves better performance than state-of-the-art algorithms.
Nighttime Haze Removal with Spatially Variant Ambient Light and Saliency-Weighted Fused Transmission

Different from hazy images captured in the daytime, the ambient illumination of nighttime hazy images is not globally homogeneous. Highlight and lowlight regions have different transmission properties in nighttime hazy scenes. In this paper, we propose a nighttime dehazing model without using the dark channel prior. We estimate the ambient illumination via the Difference of Gaussian (DoG), which selectively retains the high-frequency edges of brightness mutation. We propose a coefficient fusion algorithm in the LAB color space using Homomorphic filtering to estimate the transmission of highlight regions. And we propose a Retinex-like transmission estimation model for lowlight regions. Then we acquire the global saliency-weighted fused transmission. Finally, we get the haze-free results via the nighttime atmospheric scattering model. Experimental results show that our method outperforms other state-of-the-art methods in both color and detail recovery.
HRFNet: High-Resolution Forgery Network for Localizing Satellite Image Manipulation

Existing high-resolution satellite image forgery localization methods rely on patch-based or downsampling-based training. Both of these training methods have major drawbacks, such as inaccurate boundaries between pristine and forged regions, the generation of unwanted artifacts, etc. To tackle the aforementioned challenges, inspired by the high-resolution image segmentation literature, we propose a novel model called HRFNet to enable satellite image forgery localization effectively. Specifically, equipped with shallow and deep branches, our model can successfully integrate RGB and resampling features in both global and local manners to localize forgery more accurately. We perform various experiments to demonstrate that our method achieves the best performance, while the memory requirement and processing speed are not compromised compared to existing methods.
Deep Variational Segmentation of Topology-Constrained Object Sets, with Correlated Uncertainty Models, for Robustness to Degradations

Some key applications in medical image segmentation involve object complexes having specific topologies, but typical deep neural networks (DNNs) ignore such topologies. We propose a novel DNN framework to model topology-constrained object boundaries, incorporating both individual-object and multi-object topology constraints. Unlike typical DNNs, our topology-constrained DNN makes the learning significantly more robust to out-of-distribution images. Moreover, our DNN combines variational modeling in latent-space with uncertainty modeling of boundary points along with inter-point correlations. Results on publicly available datasets show our framework to outperform existing methods.