Reviewerly/exHarmony huggingface dataset:
Automated Segmentation of Prohibited Items in X-Ray Baggage Images Using Dense De-Overlap Attention Snake

Prohibited item segmentation has a wide range of applications in the security check field, such as computer-aided screening, threat image projection and material discrimination. However, the severe object overlapping in X-ray baggage images restricts the performance of common CNN-based segmentation methods greatly. Worse, no public dataset can be used to promote research in this challenging and promising area. In this paper, to cope with these problems, we present the first Prohibited Item X-ray segmentation dataset named PIXray. PIXray comprises 5,046 X-ray images, in which 15 classes of 15,201 prohibited items are annotated as instance-level masks. Besides, we contribute a dense de-overlap attention snake (DDoAS) in the context of deep learning for automated and real-time prohibited item segmentation. DDoAS mainly includes a dense de-overlap module (DDoM) and an attention deforming module (ADM). Specifically, DDoM is designed to infer prohibited item information accurately from extreme background overlaps through dense reversed connections. ADM aims to improve the low learning efficiency introduced by large variations in shapes and sizes among different prohibited items. Comprehensive evaluation on the PIXray shows the effectiveness and superiority of DDoM and ADM. DDoM excels at recognizing prohibited items from complex backgrounds than other in-domain methods and achieves consistent performance gain over various network backbones, extending the idea of tackling overlapping images data. ADM can ease the model training and further refine the mask quality. Furthermore, out-of-domain experiments prove that DDoAS can also be applied to natural images and achieves comparable performance to the state-of-the-art methods, which implies its potential applications in other fields. The dataset and source code are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Mbwslib/DDoAS</uri> .
A Data-Driven Cyber–Physical System Using Deep-Learning Convolutional Neural Networks: Study on False-Data Injection Attacks in an Unmanned Ground Vehicle Under Fault-Tolerant Conditions

Leveraging the benefits of deep-learning convolutional neural networks, we introduce a new data-driven cyber–physical system specifically designed to address the vulnerability of middleware software, namely, Robot Operating System (ROS), widely implemented in robotics in both civilian and military domains. As for our research platform, we employ the GVR-BOT unmanned ground vehicle, which is a replicate of the U.S. Army ground robot. We focus our study on the ability of the ground robot to operate under fault-tolerant conditions, making it challenging from the standpoint of cybersecurity to differentiate between legitimate and malicious operations. The GVR-BOT ground vehicle belongs to a class of differential drive ground robots and employs ROS in its onboard computer to interact with users. To facilitate deep learning, we develop a large database of images based on the network-traffic data of ROS, describing the dynamics of the GVR-BOT ground robot under legitimate and malicious operations. We use our image database to train and validate the performance of our deep-learning CNN system. Given a set of RGB/grayscale images describing the normalized time-series data representing the dynamics of the GVR-BOT ground robot, the objective of our proposed cybersecurity algorithm is to safeguard the legitimate operation of the ground robot under fault-tolerant conditions, such that any attempts to compromise its performance (e.g., malicious attacks) can be prevented within the minimum detection time. Our research indicates a promising result as our system is capable of detecting malicious attacks with high accuracy while recognizing its legitimate operations with reasonably small false-positive rates.
Teacher-Student Learning: Efficient Hierarchical Message Aggregation Hashing for Cross-Modal Retrieval

Inspired by the powerful representation capability of deep neural networks, deep cross-modal hashing methods have recently drawn much attention and various deep cross-modal hashing methods have been developed. However, two key problems have not been solved well yet: 1) With advanced neural network models, how to seek the multi-modal alignment space which can effectively model the intrinsic multi-modal correlations and reduce the heterogeneous modality gaps. 2) How to effectively and efficiently preserve the modelled multi-modal semantic correlations into the binary hash codes under the deep learning paradigm. In this paper, we propose a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Hierarchical Message Aggregation Hashing</i> (HMAH) method within an efficient teacher-student learning framework. Specifically, on the teacher end, we develop hierarchical message aggregation networks to construct a multi-modal complementary space by aggregating the semantic messages hierarchically across different modalities, which can better align the heterogeneous modalities and model the fine-grained multi-modal correlations. On the student end, we train a couple of student modules that learn hash functions to support cross-modal retrieval. We design a cross-modal correlation knowledge distillation strategy which seamlessly transfers the modelled fine-grained multi-modal semantic correlations from the teacher to the lightweight student modules. With the fine-grained knowledge supervision from teacher module, the semantic representation capability of hash functions can be enhanced. In addition, the whole learning framework avoids the time-consuming finetuning on the pre-trained deep models as existing methods and it is computationally efficient. Experimental results demonstrate the significant performance improvement of the proposed method on both retrieval accuracy and efficiency, compared with the state-of-the-art deep cross-modal hashing methods.
Untrimmed Action Anticipation

Egocentric action anticipation consists in predicting a future action the camera wearer will perform from egocentric video. While the task has recently attracted the attention of the research community, current approaches assume that the input videos are "trimmed", meaning that a short video sequence is sampled a fixed time before the beginning of the action. We argue that, despite the recent advances in the field, trimmed action anticipation has a limited applicability in real-world scenarios where it is important to deal with "untrimmed" video inputs and it cannot be assumed that the exact moment in which the action will begin is known at test time. To overcome such limitations, we propose an untrimmed action anticipation task, which, similarly to temporal action detection, assumes that the input video is untrimmed at test time, while still requiring predictions to be made before the actions actually take place. We propose an evaluation procedure for methods designed to address this novel task, and compare several baselines on the EPIC-KITCHENS-100 dataset. Experiments show that the performance of current models designed for trimmed action anticipation is very limited and more research on this task is required.
Learning to Learn Better Unimodal Representations via Adaptive Multimodal Meta-Learning

Multimodal sentiment analysis is an emerging field of artificial intelligence. The most predominant approaches have made notable progress by designing sophisticated fusion architectures, exploring inter-modal interactions between modalities. However, these works tend to utilize a uniform optimization strategy for each modality, so that only sub-optimal unimodal representations are obtained for multimodal fusion. To address this issue, we propose a novel meta-learning based paradigm that can retain the advantages of unimodal existence and further boost the performance of multimodal fusion. Specifically, we introduce the Adaptive Multimodal Meta-Learning (AMML) to meta-learn the unimodal networks and adapt them for multimodal inference. AMML can (1) effectively obtain more optimized unimodal representation via meta-training on unimodal tasks, which adaptively adjusts the learning rate and assigns a more specific optimization procedure for each modality; (2) and adapt the optimized unimodal representations for multimodal fusion via meta-testing on multimodal tasks. Considering multimodal fusion often suffers from the distributional mismatches between features of different modalities due to heterogeneous nature of the signals, we implement a distribution transformation layer on unimodal representations to regularize the unimodal distributions. In this way, distribution gaps can be reduced to achieve a better effect of fusion. Extensive experiments on two widely-used datasets demonstrate that AMML achieves state-of-the-art performance.
Invariant Structure Representation for Remote Sensing Object Detection Based on Graph Modeling

Due to the characteristics of vertical orthophoto imaging, the apparent structural features of the object in the remote sensing image are relatively stable, such as the cross-shaped structure of the aircraft, the rectangular structure of the vehicle, etc. Compared with the traditional visual features, using these features is conducive to improving the accuracy of object detection. However, there are few studies on such characteristics. In this paper, we systematically study the invariant structural features of remote sensing objects and propose a Graph Focusing Aggregation Network (GFA-Net) to represent the structural features of remote sensing objects. Among them, in view of the problem that traditional convolutional neural networks (CNNs) are sensitive to the changes in rotation, scale, and other factors, which makes it difficult to extract structural features, we propose the Graph Focusing Process (GFP) based on the idea of graph convolution. Analysis and experiments show that graph structure has significant advantages over Euclidean feature space under CNN in expressing such structural features. In order to realize the end-to-end efficient training of the above model, we design Graph Aggregation Network (GAN) to update the weight of nodes. We verify the effectiveness of our method on the proposed multi-task datasets ACSD and large-scale fine-grained remote sensing dataset FAIR1M. Experiments conducted on the object detection data sets of DOTA and HRSC2016 prove that the proposed method is superior to the current state-of-the-art method.
The Unfairness of Active Users and Popularity Bias in Point-of-Interest Recommendation

Point-of-Interest (POI) recommender systems provide personalized recommendations to users and help businesses attract potential customers. Despite their success, recent studies suggest that highly data-driven recommendations could be impacted by data biases, resulting in unfair outcomes for different stakeholders, mainly consumers (users) and providers (items). Most existing fairness-related research works in recommender systems treat user fairness and item fairness issues individually, disregarding that RS work in a two-sided marketplace. This paper studies the interplay between (i) the unfairness of active users, (ii) the unfairness of popular items, and (iii) the accuracy (personalization) of recommendation as three angles of our study triangle. We group users into advantaged and disadvantaged levels to measure user fairness based on their activity level. For item fairness, we divide items into short-head, mid-tail, and long-tail groups and study the exposure of these item groups into the top-k recommendation list of users. Experimental validation of eight different recommendation models commonly used for POI recommendation (e.g., contextual, CF) on two publicly available POI recommendation datasets, Gowalla and Yelp, indicate that most well-performing models suffer seriously from the unfairness of popularity bias (provider unfairness). Furthermore, our study shows that most recommendation models cannot satisfy both consumer and producer fairness, indicating a trade-off between these variables possibly due to natural biases in data. We choose the POI recommendation as our test scenario; however, the insights should be trivially extendable on other domains.
Metaverse Marketing for Community Development: Revitalization of Traditional Industrial Sectors in Thailand

This long abstract paper presents the study based on a concrete exploration of advanced innovation such as the metaverse needs to be done to revitalize the traditional industries in Thailand; furthermore, the consideration of community development should be followed as well as providing a concrete result based on the roadmap of promoting metaverse marketing and the implement of metaverse technology as a tool to support small local businesses in Thailand.
AdaZoom: Towards Scale-Aware Large Scene Object Detection

Detection in large scenes is a challenging issue due to small objects and extreme scale variation. It is difficult for the deep-learning-based detector to extract features of small objects with only a few pixels. Most existing methods employ image pyramid and feature pyramid for multi-scale inference to alleviate this issue. However, they lack scale awareness to adapt to objects with different scales. In this paper, we propose a novel Adaptive Zoom (AdaZoom) network for scale-aware large scene object detection. There are three main contributions. First, an Adaptive Zoom network is proposed to actively focus and adaptively zoom the focused regions for high-performance object detection in large scenes. Second, to tackle the problem of missing annotations for focused regions, we train AdaZoom with the reward which measures the quality of generated regions, based on the paradigm of deep reinforcement learning. At last, we propose collaborative training to iteratively promote the joint performance of AdaZoom and the detector. To validate the effectiveness, we conduct extensive experiments on VisDrone2019, UAVDT and DOTA datasets. The experiments show AdaZoom brings consistent and significant improvement over different detection networks, achieving state-of-the-art performance on these datasets, especially outperforming the existing methods by AP of 4.64% on VisDrone2019.
Active Inference Integrated With Imitation Learning for Autonomous Driving

Classical imitation learning methods suffer substantially from the learning hierarchical policies when the imitative agent faces an unobserved state by the expert agent. To address these drawbacks, we propose an online active learning through active inference approach that encodes the expert’s demonstrations based on observation-action to improve the learner’s future motion prediction. For this purpose, we provide a switching Dynamic Bayesian Network based on the dynamic interaction between the expert agent and another object in its surrounding as a reference model, which we exploit to initialize an incremental probabilistic learning model. This learning model grows and matures based on the free-energy formulation and message passing of active inference dynamically at discrete and continuous levels in an online active learning phase. In this scheme, generalized states of the learning world are represented as distance-vector, where it is the learner’s observation concerning its interaction with a moving object. Considering the distance vector entail intentions, it enables action prediction evaluation in a prospective sense. We illustrate these points using simulations of driving intelligent agents. The learning agent is trained by using long-term predictions from the generative learning model to reproduce the expert’s motion while learning how to select a suitable action through new experiences. Our results affirm that a Dynamic Bayesian optimal approach provides a principled framework and outperforms conventional reinforcement learning methods. Furthermore, it endorses the general formulation of action prediction as active inference.
Invertible Color-to-Grayscale Conversion Using Lossy Compression and High-Capacity Data Hiding

Invertible color-to-grayscale conversion is a research issue in grayscale image colorization, which is a complex problem resulting from information loss. This paper presents an innovative invertible color-to-grayscale conversion idea by independently compressing the chromaticity plane and hiding it into the corresponding luminance plane as a watermark. Since the chromaticity and luminance planes are orthogonal in the proposed method, they can be processed efficiently without mutual influence. By using an efficient lossy compression operation, we can save more chromatic information. By using two high-capacity data hiding techniques (reversible watermarking (RW) and least significant bit (LSB) substitution), we can embed the compressed chromatic information into the luminance plane perfectly. For the purpose of integrality authentication, the luminance plane is hashed as part of the embedded information before the embedding. Experimental results have shown that higher quality of reconstructed color images can be achieved by using RW, but the quality of the synthesized grayscale drops sharply. By using LSB substitution, we can obtain high-quality synthesized grayscale and reconstructed color images simultaneously. Furthermore, we have compared the proposed LSB-based scheme with several recently reported state-of-the-art methods to validate the superiority of the proposed approach.
Noise-Aware Quantum Circuit Simulation With Decision Diagrams

Since quantum computers can solve important problems faster than classical computers, many resources have gone into the development of this technology in recent decades. Despite the tremendous progress that has already been made toward the development of quantum computers, they are still an emerging technology, which restricts access and reliability. Thus, research on quantum algorithms still heavily relies on quantum circuit simulators that run on classical hardware. However, simulating the execution of a quantum computer on conventional hardware is exponentially difficult, which is also the reason why quantum computing is an interesting technology in the first place. Particularly complex is noise-aware simulation of quantum computers, i.e., the consideration of noise effects that are common in today’s quantum hardware during quantum circuit simulation. In this work, we investigate the use of decision diagrams for this task. To this end, we present two distinct approaches for noise-aware quantum circuit simulation, investigate how they can be realized using decision diagrams, and implement decision diagram-based solutions for each of the presented noise-aware simulation schemes. In an extensive evaluation, we unveil potential for further improvements and also demonstrate substantial speed-ups compared to the current state of the art.
A Secure Information Transmission Protocol for Healthcare Cyber Based on Quantum Image Expansion and Grover Search Algorithm

In the healthcare cyber, the transmission of information between the head office and each sub-branch has always been a very important process. However, with the continuous development of network technology in recent years, its security has also been subjected to greater and greater tests. Based on quantum image expansion technology and Grover search algorithm, this paper proposes a more secure and large-capacity secure information transmission protocol for healthcare cyber. The new protocol embeds the secret information through three layers and encrypts it at the same time. Finally, the secret information is transmitted through quantum channel in the form of quantum state, which has very high security. At the same time, the new protocol is essentially a novel quantum image steganography protocol. The imperceptibility of the steganography protocol is essentially a higher level of security, which can effectively avoid various attacks and further guarantee the security of the information transmission process.
Adversarial Steganography Embedding via Stego Generation and Selection

The recent literature has shown that adversarial embedding has promise for enhancing the security of steganography. However, existing methods achieve the final stego mainly based on a pre-trained Convolutional Neural Network (CNN)-based steganalyzer without considering any other steganalytic features. When the steganalyzer is re-trained, its performance usually drops significantly. We propose a novel adversarial embedding method via stego generation and selection. To improve the diversity of the stego images, this method first randomly generates many candidate stegos according to the amplitudes of the gradients and embedding costs of a given cover. Since the image residuals are the commonly used low-level features in many steganalyzers, the proposed method carefully designs different adaptive high-pass filters to calculate the image residuals, and then selects a final stego from among those candidate stegos which can successfully fool the pre-trained steganalyzer, according to the residual distance between stego and the cover. Extensive experimental evaluations on re-trained CNN-based and traditional steganalyzers demonstrate that the proposed method can significantly enhance the security of the modern steganographic methods in both spatial and JPEG domains, and achieve much better performance than related adversarial embedding methods.
Unsupervised Haze Removal for Aerial Imagery Based on Asymmetric Contrastive CycleGAN

Aerial image dehazing is an important preprocessing step, since haze extremely degrades the imaging quality and affects subsequent the applications of aerial imagery.Most current haze removal methods achieve encouraging performance by relying on paired synthetic data, while are limited to their generality and scalability in the practical tasks.To this end, this paper aims to learn an effective unsupervised dehazing model from an unpaired set of clear and hazy aerial images.Motivated by the great advantages of contrastive learning in unsupervised representation field, we first attempt to formulate a Asymmetric Contrastive CycleGAN dehazing framework (namely ACC-GAN) to maximize the mutual information between the hazy domain and the haze-free domain.In the latent representation space, the introduced contrastive constraint ensures that the restored image is pulled closer to the clear image and pushed away from the hazy image, so as to indirectly regularize the unsupervised dehazing process.Importantly, different from the standard CycleGAN, we develop an additional feature transfer network into the forward path to form the asymmetric structure of ACC-GAN, which can enhance encoded features from hazy domain to haze-free domain.During training, multi-dimension loss terms are jointly built into a loss committee for generating dehazed results with higher naturalness and better fidelity.Experimental results on synthesis and real-world datasets indicate that our method is superior to existing unsupervised dehazing approaches, and is also very competitive to other related supervised models.