Reviewerly/exHarmony huggingface dataset:
Scene Augmentation Methods for Interactive Embodied AI Tasks

The emerging embodied AI paradigm enables intelligent robots to learn like humans from interaction, and is thus considered an effective way approach to general artificial intelligence. Unfortunately, even the best performing agents still overfit and generalize poorly to unseen scenes, due to the limited scenes provided by embodied AI simulators. To alleviate this issue, we propose a scene augmentation strategy to scale up the scene diversity for interactive tasks and make the interactions more like the real world. Compared to existing methods focusing on improving diversity in observation space, our approach aims to automatically derive a new distribution of scene layout or object states to provide sufficient conditional transfer models for the agent to learn environmental invariant and irrelevant features through interaction. Specifically, we provide four representative and systematical scene augmentation methods that can derive scene variants for entities from different levels of a scene graph. We demonstrate the efficiency of our methods in the popular embodied AI simulator iGibson. To verify the effectiveness for interactive agents, we also conduct two representative interactive tasks with a proposed continuous action parameterized method. The evaluation results show that our scene augmentation strategy can boost the performance of interactive agents and generalize well to unseen scenes. Finally, we present a systematic generalization analysis using the proposed methods to explicitly estimate the ability of agents to generalize to new layouts, new objects, and new object states. We claim that the proposed methods are not limited to iGibson and can be extended to other interactive simulators. The code and additional information are available at: https://github.com/sanghongrui/SceneAug.
Twin Broad Learning System for Fault Diagnosis of Rotating Machinery

As rotating machines are more and more widely used in modem equipment, their fault diagnosis is important to guarantee the instrument’s reliability and safety. Although intelligent fault diagnosis based on deep learning (DL) has achieved great performance in many fault diagnosis tasks, these models are highly dependent on the large-scale training dataset and can be time-consuming during training and testing. These limitations largely affect the efficiency of diagnosis in real-world applications. An effective alternative way is to develop the single-layer feedforward networks (SLFNs) based Broad Learning System (BLS) for fault diagnosis tasks, which enjoy fast training speed as well as strong generalization ability. However, we found that the least square classifier used in classical BLS can have a problem distinguishing the possibly overlapping fault patterns. In this paper, we propose a novel twin broad learning system (TBLS) for fault diagnosis of rotating machinery. Rather than using the classical least square method, the proposed TBLS learns to find two nonparallel hyper-planes to deal with the classification problem, which shows stronger generalization ability in fault diagnosis problems. Experimental results on two fault diagnosis benchmark datasets for typical rotating machinery further illustrate the effectiveness of the TBLS methods, which offer a high-efficiency solution for rotating machinery fault diagnosis.
Linear Functionality Equivalence Attack Against Deep Neural Network Watermarks and a Defense Method by Neuron Mapping

As an ownership verification technique for deep neural networks, the white-box neural network watermark is being challenged by the functionality equivalence attack. By leveraging the structural symmetry within a deep neural network and manipulating the parameters accordingly, an adversary can invalidate almost all white-box watermarks without affecting the network’s performance. This paper introduces the linear functionality equivalence attack, which can adapt to different network architectures without requiring knowledge of either the watermark or data. We also propose NeuronMap, a framework that can efficiently neutralize linear functionality equivalence attacks and can be easily combined with existing white-box watermarks to enhance their robustness. Experiments conducted on several deep neural networks and state-of-the-art white-box watermarking schemes have demonstrated not only the destructive power of linear functionality equivalence attacks but also the defense capability of NeuronMap. Our result shows that the threat of basic linear functionality equivalence attacks against deep neural network watermarks can be effectively solved using NeuronMap.
Exploring Spatio–Temporal Graph Convolution for Video-Based Human–Object Interaction Recognition

Video-based human-object interaction recognition is a challenging task since the state of objects as well as their correlations change constantly in the video. Existing methods mainly use 3DCNN or use separate components (e.g., GCN + RNN) to model the spatial correlation or the temporal correlation respectively, but ignore modeling spatio-temporal correlations simultaneously and long-term temporal dynamics of objects. In this paper, we propose a novel model, named Spatio-Temporal Interaction Graph Parsing Networks (STIGPN), for human-object interaction recognition in videos. STIGPN captures both spatial and temporal correlations simultaneously and thus can capture intra-frame and inter-frame dependencies efficiently and effectively. To model long-term temporal dynamics of objects, we introduce spatio-temporal feature enhancement, which can improve the detection of the salient human-object interaction pairs. We explore three types of spatio-temporal graph convolutions to simultaneously capture the spatio-temporal correlations and assess their effectiveness as the basic building block of STIGPN. Extensive experiments on CAD-120, Something-Else and Charades datasets show that our proposed solution leads to competitive results compared with the state-of-the-art methods. Code for STIGPN is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/NingWang2049/STIGPN2</uri>
Complete The Last Mile

In social networks, delivery-related concerns are perennially significant and challenging. Another noteworthy and captivating development is the development of quantum computing or quantum-inspired computing. Combining these two fields is not only timely but natural and necessary in the current environment of data explosion. The current strengths and challenges of quantum computing were put together in addressing delivery problems as an example and provide a collection and analysis of existing approaches to last-mile delivery. The purpose of this paper is to help people to understand how to use quantum computing to quickly solve delivery problems and find the potential niche market in highly competitive big data environments.
Supporting Wider Baseline Light Fields in JPEG Pleno With a Novel Slanted 4D-DCT Coding Mode

Light fields are one of the emerging 3D representation formats with an effective potential to offer very realistic and immersive visual experiences. This capability comes at the cost of a very large amount of acquired data which practical use requires efficient coding solutions. This need was already addressed by the JPEG Pleno Light Field Coding standard for static light fields, which has specified two coding modes, named 4D-Transform and 4D-Prediction. While the first offers better compression performance for smaller baseline light fields, the second excels for larger baseline light fields. This paper intends to propose a novel light field coding mode, the Slanted 4D-Transform coding mode, which extends the 4D-Transform coding mode based on the conventional 4D-DCT, to offer better compression performance than both the available JPEG Pleno coding modes, independently of the baseline. The key idea is to apply first to each 4D block in the light field an adaptive, hierarchical geometric transformation, which makes the data in the block more energy-compaction friendly for the following 4D-DCT. The rate-distortion performance results show that the proposed Slanted 4D-Transform codec is able to outperform both the already standardized JPEG Pleno coding modes, showing BD-Rates gains of 31.03% and 28.30% for the 4D-Transform and 4D-Prediction modes, respectively, thus implying that a single coding mode can efficiently code all types of light fields.
Color Transfer

Image or video appearance features (e.g., color, texture, tone, illumination, and so on) reflect one’s visual perception and direct impression of an image or video. Given a source image (video) and a target image (video), the image (video) color transfer technique aims to process the color of the source image or video (note that the source image or video is also referred to as the reference image or video in some literature) to make it look like that of the target image or video, i.e., transferring the appearance of the target image or video to that of the source image or video, which can thereby change one’s perception of the source image or video. This chapter will introduce the techniques about image and video color transfer.
Analysis of the Effects of the Two-Photon Temporal Distinguishability on Measurement-Device-Independent Quantum Key Distribution

Measurement-device-independent quantum key distribution (MDI-QKD) protects legitimate users from attacks on measurement devices.The decoy method allows for unconditionally secure quantum key generation using lasers.One of the most important issues in MDI-QKD is that photons from two independent lasers must be indistinguishable.Because Alice and Bob send Charlie laser pulses through separate fiber links, the arrival times of the pulses fluctuate independently.According to the Hong-Ou-Mandel (HOM) interference at Charlie's relay, the time delay between two photons has the greatest effect on distinguishability.However, to date, these effects have not been analyzed.Our study uses simulations to investigate the effects of two-photon temporal distinguishability in terms of the visibility of the HOM-dip of two photons on the final key rate of finite-size MDI-QKD.Furthermore, an acceptable time delay range was estimated based on photons with Gaussian spectral amplitude functions.
NIRPed: A Novel Benchmark for Nighttime Pedestrian and Its Distance Joint Detection

In nighttime driving scenes, due to insufficient and uneven lighting, and the scarcity of high-quality datasets, the miss rate of nighttime pedestrian detection (PD) is much higher than that of daytime. Vision-based distance detection (DD) has the advantages of low cost and good interpretability, but the existing methods have low precision, poor robustness, and the DD is mostly performed independently of PD. A narrowband near-infrared (NIR) camera and an NIR lamp to relieve the impact of noisy visible light (VIS) and improve imaging quality have been selected; a LiDAR to obtain distance information has been embedded in the system; a nighttime driving scene pedestrian and its distance joint detection dataset NIRPed has been built. The NIRPed includes 146k pedestrian annotations, which is 3 times that of NightOwls, the largest VIS nighttime pedestrian dataset. Based on Faster-RCNN, a joint PD and DD method using monocular imaging has been proposed. The propo- sed method realizes the joint detection of pedestrian and its distance on the NIRPed dataset with the PD log-average miss rate and DD mean absolute error rate of 6.5% and 5.5%, respectively. For comparison, the joint detection method was implemented on other large-scale VIS pedestrian datasets as well. Moreover, compared with the existing vision-based DD methods, the proposed method is less affected by pedestrian distance and height, exhibits higher accuracy and robustness, and satisfied the ISO requirements for intelligent transportation systems (absolute error rate < 15%).
Location Privacy-Aware and Energy-Efficient Offloading for Distributed Edge Computing

Driven by the ever-increasing scale and intensity of the computing tasks arising from various mobile applications, distributed edge computing has fostered wide research interests. It can effectively reduce the task processing delay by partitioning the original large-scale task into several small subtasks and offloading them to multiple edge nodes (ENs) for parallel computing. In edge computing, as the mobile user usually tends to offload computing tasks to closer ENs to save transmit power, the attacker may stealthily infer user location by exploiting this feature. Although there have been some pioneering works on offloading related location privacy, they mainly focused on the scenario where each task can only be offloaded to a single EN, and may not be directly applicable to distributed edge computing. Besides, the privacy issues considered in existing works are mainly based on good heuristics, and there is still a lack of concrete examples of location privacy attacks in edge computing. To the best of our knowledge, the location privacy issue in distributed edge computing still remains largely unexplored in existing literature. With this consideration, a location inference attack based on matrix sequential probability ratio test (MSPRT) is identified in this work. Besides, a countermeasure based on dynamic multi-EN selection is proposed, together with a location privacy-aware and energy-efficient distributed offloading scheme based on the generic Lyapunov optimization framework. Both theoretic analysis and simulations based on real-world channel measurements are employed to validate the feasibility of the identified MPSRT attack and the effectiveness of the proposed defense scheme.
Style Transfer

As an extension of color transfer, style transfer refers to rendering the content of a target image or video in the style of an artist with either a style sample or a set of images through a style transfer model (Fig. 6.1). As an emerging field, the study of style transfer has attracted the attention of a large number of researchers. After decades of development, it has become a highly interdisciplinary research with a variety of artistic expression styles that can be achieved, such as pencil painting, oil painting, watercolor painting, cartoon painting, and other artistic effects.
Decolorization

Image or video decolorization, also known as grayscale transformation, converts a three-channel color image or video into a single-channel grayscale one. Decolorization is actually a process of dimension reduction, so that the resulting grayscale image or video often only contains the most important information, which greatly saves storage space. A grayscale image or video can better display the texture and contour of objects. Decolorization can also be widely applied in the field of image compression, medical image visualization, and image or video art stylization. Black-and-white digital printing of images, with the advantages of low cost and fast printing, is common in daily life, one important process of which is decolorization, i.e., a color image sent to a monochrome printer must undergo a color-to-grayscale transformation. Below we will summarize various image and video decolorization methods in the last decade.
Leveraging Domain Features for Detecting Adversarial Attacks Against Deep Speech Recognition in Noise

In recent years, significant progress has been made in deep model-based automatic speech recognition (ASR), leading to its widespread deployment in the real world. At the same time, adversarial attacks against deep ASR systems are highly successful. Various methods have been proposed to defend ASR systems from these attacks. However, existing classification based methods focus on the design of deep learning models while lacking exploration of domain specific features. This work leverages filter bank-based features to better capture the characteristics of attacks for improved detection. Furthermore, the paper analyses the potentials of using speech and non-speech parts separately in detecting adversarial attacks. In the end, considering adverse environments where ASR systems may be deployed, we study the impact of acoustic noise of various types and signal-to-noise ratios. Extensive experiments show that the inverse filter bank features generally perform better in both clean and noisy environments, the detection is effective using either speech or non-speech part, and the acoustic noise can largely degrade the detection performance.
Low-Exposure Image and Video Enhancement

Low-exposure image and video enhancement aims to process color and details of an underexposed image or video to improve their appearance. With the rapid development of digital photography technology, the acquisition of digital images and videos becomes more and more convenient. However, in daily life, due to low lighting, equipment configuration, and other reasons, the images and videos obtained often suffer from being underexposed and show dark scenes. Low-exposure images and videos have very low visibility, and the details in the image and video scenes cannot be displayed well, which greatly affects the aesthetics and has an influence on the downstream image and video processing. Especially in the public security management, the monitoring system usually suffers from capturing low-exposure night vision video, which seriously affects the public transport monitoring and the monitoring of criminal activities. In the downstream processing of images and videos, the processing (e.g., tone adjustment and pattern recognition) would be greatly influenced by insufficient exposure and will be affected. Therefore, these images and videos must be enhanced by effective methods to make their exposure uniform and display enough details.
Block Optimization and Outlier Detection in GEO SAR Turbulent Tropospheric Error Compensation

Geosynchronous Synthetic Aperture Radar (GEO SAR) has important potential applications in the fields of atmospheric water cycle and digital weather forecast, and establishing the connection between GEO SAR imaging and atmospheric water vapor inversion is a key step. The turbulent variation of atmospheric phase delay will cause the SAR image defocus, and turbulent tropospheric delay phase estimation based on block autofocus method in low-orbit high resolution space-borne SAR can be used for reference, but this method is very sensitive to the data block selection, modulation frequency (FM) outlier detection and correction. This Letter gives some considerations to these two problems of block selection and outlier correction. Based on the idea of Extreme Learning Machine (ELM), a mathematical model with nonlinearity as the core and linear model as easy to solve is established. When the parameters affecting the turbulent troposphere are known, the model can be directly used to obtain the block optimization selection, namely, the number and size of blocks. When the parameters are unknown, the model can be used to roughly inverse the parameters and provide initial values for iteration. For the second problem, using the prior information of the correlation of the FM estimated by the adjacent sub-blocks, we proposes a Local Outlier Factor (LOF) method to identify the error point by defining the distance threshold. The effectiveness of the proposed algorithm is verified by Sentinel-1 data injected with tropospheric error.
Introduction

An image or video is a similar representation and description of the real world with the human visual system, which is among the most important information carriers in human daily activities. An image or video contains many different features of a scene, such as color, texture, illumination, motion, and so on. They are the main sources of information for one to understand the world.
Multi-Scale Frequency Separation Network for Image Deblurring

Image deblurring aims to restore the detailed texture information or structures from the blurry images, which has become an indispensable step in many computer vision tasks. Although various methods have been proposed to deal with the image deblurring problem, most of them treated the blurry image as a whole and neglected the characteristics of different image frequencies. In this paper, we present a new method called multi-scale frequency separation network (MSFS-Net) for image deblurring. MSFS-Net introduces the frequency separation module (FSM) into an encoder-decoder network architecture to capture the low- and high-frequency information of image at multiple scales. Then, a simple cycle-consistency strategy and a sophisticated contrastive learning module (CLM) are respectively designed to retain the low-frequency information and recover the high-frequency information during deblurring. At last, the features of different scales are fused by a cross-scale feature fusion module (CSFFM). Extensive experiments on benchmark datasets show that the proposed network achieves state-of-the-art performance.
Spatio-Temporal Feature Pyramid Interactive Attention Network for Egocentric Gaze Prediction

Over the past few years, gaze prediction technology has rapidly developed in human visual attention mechanism research. However, there is still a large gap between the prediction models and the human visual system (HVS). This work makes four contributions to video egocentric gaze prediction research. First, we introduce a new benchmark named ECVG (egocentric video gaze), which consists of 3K high-quality, elaborately selected video sequences spanning a large range of scenes. Existing eye-tracking datasets limit human activities. In contrast, ECVG achieves the collection of eye movement data under real human activity. Second, we establish a visual attention-selection and gaze-tracking model, which summarizes the human visual attention process. Third, we propose a novel feature pyramid interactive attention 3D network (FPIANet) for egocentric gaze prediction, which directly captures the long-term relationship between spatiotemporal features at different time steps and achieves feature nonlocal interactions across temporal, spatial, and scales. In addition, we propose a multiscale interactive attention (MSIA) module, which first explicitly integrates the human top-down and bottom-up visual attention mechanisms to transform any spatiotemporal feature into another feature of the same size but with richer contexts. Furthermore, we propose a new gaze transfer path consistency evaluation and thoroughly examine the performance of our model on two large datasets (ECVG and LEDOV). The experimental results show that our model outperforms the existing state-of-the-art methods.
A Deep Reinforcement Learning Based Real-Time Solution Policy for the Traveling Salesman Problem

The rapid development of logistics and navigation has led to increasing demand for solving route optimization problems in real-time. The traveling salesman problem (TSP) tends to require fast and reliable online solutions, which may not be met by traditional iterative optimization algorithms. In this work, a real-time solution policy is proposed for TSP. The idea is to build a mapping between city information and optimal solutions using deep neural networks. Therefore, when given a new set of city coordinates, the optimal route can be directly and quickly calculated without iteration. Considering the recent advancement in computer vision with deep convolutional neural networks (DCNNs), an image representation is proposed to convert TSP to a computer vision problem. A problem decomposition method is introduced to reduce the mapping complexity. Taking advantage of the powerful fitting capabilities of DCNN, a deep reinforcement learning method is designed without any labeling requirement. The proposed method is superior for real-time applications compared with other algorithms.