Reviewerly/exHarmony huggingface dataset:
General Distortion Based Reversible Data Hiding for Binary Covers

The problem of general distortion model for reversible data hiding (RDH) is investigated in this letter. Unlike previous RDH schemes that regard the cover image as a memory-less sequence and assign the same distortion for each pixel, in this work, the modification distortion is adaptively defined for each pixel for visual performance enhancement. The situation is totally different compared with the traditional RDH approaches, and the classical histogram based methods can not be utilized. To deal with this new and challenging problem, we then propose a two-steps embedding framework. Considering binary image as cover, firstly, some pixels are selected and losslessly compressed as the reconstruction information for image recovery. Then, with the adaptively defined distortion and by utilizing matrix embedding, the secret message and the reconstruction information are embedded into the cover. In this way, reversible embedding is realized while the total distortion is minimized. By comparing with some previous RDH schemes for binary covers, experimental results show that better visual performance is achieved by the proposed method.
A Niching Regression Adaptive Memetic Algorithm for Multimodal Optimization of the Euclidean Traveling Salesman Problem

The traveling salesman problem (TSP) has been studied for many years. In particular, the multimodal optimization of the TSP is important for practical applications, because decision-makers can select the best candidate based on current conditions and requirements. In the Euclidean TSP, there are <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> points in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbb {R}^{d}$ </tex-math></inline-formula> space with Euclidean distance between any two points, that is, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$d(x, y) =||x-y||_{2}$ </tex-math></inline-formula> . The goal is to find a tour of minimum length visiting each point. In this article, we only focus on the case that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$d=2$ </tex-math></inline-formula> . Recently, in order to efficiently handle the multimodal optimization of the TSP, some methods have been developed to deal with it. Nevertheless, these methods usually perform poorly for large-scale cases. In this article, we propose a niching regression adaptive memetic algorithm (MA) to handle the multimodal optimization of the Euclidean TSP. We use the MA as the baseline algorithm and incorporate the neighborhood strategy to maintain the population diversity. In addition, we design a novel regression partition initialization and adaptive parameter control to enhance our algorithm, and propose the concept of the redundant individual to improve the search efficiency. To validate the performance of the proposed algorithm, we comprehensively conduct experiments on the multimodal optimization of TSP benchmark and the well-known TSPLIB library. The experimental results reveal that the proposed method outperforms other methods, especially for large-scale cases.
BLDSS: A Blockchain-Based Lightweight Searchable Data Sharing Scheme in Vehicular Social Networks

Vehicular social networks (VSNs) are likely to play an increasingly important role of the future smart cities, supporting traffic management, real-time warning, and sharing data (e.g., condition, vehicular, and warning). However, there are existing security and privacy issues with vehicular data, which directly lead to the insufficient utilization of vehicular data. In this article, a blockchain-based lightweight searchable data sharing scheme is presented. Specifically, the proposed protocol achieves efficient data matching and data sharing by uniting the feature of public-key encryption with equality test (PKE-ET) and proxy re-encryption (PRE), meanwhile ensuring reliable matching results by invoking smart contacts. Moreover, a reputation-based dynamic PBFT consensus mechanism is provided. By analyzing the reputation and behavior of the consensus nodes, we further reduce the probability of malicious nodes becoming the consensus nodes, thus decreasing the computing overhead and communication costs in the consensus process. Finally, the security of the proposed protocol is demonstrated based on a computation Diffie–Hellman assumption in the Random Oracle model. According to the performance analysis, it was found that compared to the other protocols, the proposed protocol is computationally lightweight.
Robust Tracking via Learning Model Update With Unsupervised Anomaly Detection Philosophy

Template tracking is a typical paradigm to adaptively locate arbitrary objects in the tracking literature. Although existing works present diverse template updating approaches, one of the essential problems of template updating has not been solved effectively, i.e., when and how to update a template. In this work, we treat the updating time as an abnormal moment that indicates the previous template cannot depict the target accurately any more. Thus, we introduce an effective State-Edge Awareness (SEA) module that detect such abnormal moments via unsupervised anomaly detection. To be specific, by retaining multi search frames of a video, SEA firstly analysis the correlation features that generated by the template and search images. Then, it estimates the measurement for abnormal degree that is regarded as the sign for template updating. As a result, our method can not only capture the updating time automatically, but also update the templates effectively. Furthermore, the effectiveness of the proposed method has been verified on a representative CNN-based and Transformer-based tracker, respectively. The experimental results on five popular benchmarks show that our tracker can achieve the state-of-the-art performance.
LineCap: Line Charts for Data Visualization Captioning Models

Data visualization captions help readers understand the purpose of a visualization and are crucial for individuals with visual impairments. The prevalence of poor figure captions and the successful application of deep learning approaches to image captioning motivate the use of similar techniques for automated figure captioning. However, research in this field has been stunted by the lack of suitable datasets. We introduce LineCap, a novel figure captioning dataset of 3,528 figures, and we provide insights from curating this dataset and using end-to-end deep learning models for automated figure captioning.
Efficient Hybrid Network: Inducting Scattering Features

Recent work showed that hybrid networks, which combine predefined and learnt filters within a single architecture, are more amenable to theoretical analysis and less prone to overfitting in data-limited scenarios. However, their performance has yet to prove competitive against the conventional counterparts when sufficient amounts of training data are available. In an attempt to address this core limitation of current hybrid networks, we introduce an Efficient Hybrid Network (E-HybridNet). We show that it is the first scattering based approach that consistently outperforms its conventional counterparts on a diverse range of datasets. It is achieved with a novel inductive architecture that embeds scattering features into the network flow using Hybrid Fusion Blocks. We also demonstrate that the proposed design inherits the key property of prior hybrid networks - an effective generalisation in data-limited scenarios. Our approach successfully combines the best of the two worlds: flexibility and power of learnt features and stability and predictability of scattering representations.
Document Image Watermarking Scheme with Adaptive Positioning Based on Discrete Wavelet Transform and Discrete Cosine Transform

An important technology for copyright protection is digital watermarking, which generally embeds copyright infor-mation into the carrier invisibly. In the case of various attacks, the identity of the watermark can also be extracted to prove its copyright from the carrier of the watermark. Most of the existing watermarking technologies use natural images as carriers. Unlike natural images, document images are not rich in color and texture, and thus carry less redundant information for watermarks. This paper proposes a adaptive positioning document watermarking algorithm based on Discrete Wavelet Transform(DWT) and Discrete Cosine Transform(DCT). The method integrates adaptive positioning algorithm, discrete wavelet transform, discrete cosine transform and other algorithms. In our case, we first scan the document image to adaptively embed the keypoints information, so that even if a part of the document is recorded, the complete watermark area can be accurately located. Secondly, we construct the watermark embedding regions centered at keypoints. Third, the DWT and DCT algorithms are used to embed the watermark, which achieve strong robustness to the capture process attack. Finally, the watermark can be extracted from at least one region in the extraction stage. Compared with the previous watermarking algorithms, the proposed approach not only ensures the independence of embedded content, but also is robust to various noise attacks. In addition, the method can meet the extraction accuracy requirements even when the document image is incomplete.
ISD-QA: Iterative Distillation of Commonsense Knowledge from General Language Models for Unsupervised Question Answering

Commonsense question answering has primarily been tackled through supervised transfer learning, where a language model pre-trained on large amounts of data is used as the starting point. While successful, the approach requires large amounts of labeled question-answer pairs, with increasingly larger amounts of data required as the complexity of scenarios or tasks such as commonsense QA increases. In this paper, we hypothesize that large-scale pre-training of language models encodes the necessary commonsense knowledge to answer common questions in context without labeled data. We propose a novel framework called Iterative Self Distillation for QA (ISD-QA), which extracts the "dark knowledge" encoded during largescale pre-training of language models to provide supervision for commonsense question answering. We show that the approach can be used to train common neural QA models for commonsense question answering by distilling knowledge from language models in an unsupervised manner. With no bells and whistles, we achieve an average of 68% of the performance of fully supervised QA models while requiring no labeled training data. Extensive experiments on three public benchmarks (OpenBookQA, HellaSWAG, and CommonsenseQA) show the effectiveness of the proposed approach.
Herb: Privacy-preserving Random Forest with Partially Homomorphic Encryption

Building a Machine Learning model requires the use of large amounts of data. Due to privacy and regulatory concerns, these data might be owned by multiple sites and are often not mutually shareable. Our work deals with private learning and inference for the Weighted Random Forest model when data records are vertically distributed among multiple sites. Previous privacy-preserving vertical tree-based frameworks either adapt Secure Multi-party Computation or share intermediate results and are hard to generalize or scale. In contrast, our proposal contains efficient collaborative calculation algorithms of the Gini Index and Entropy for computing the impurity of decision tree nodes while protecting all intermediate values and disclosing minimal information. We offer a learning protocol based on the Paillier Cryptosystem and Digital Envelope. Also, we provide an inference protocol found on the Look-up Table. Our experiments show that the proposed protocols do not cause predictive performance loss while still establishing and utilizing the model within a reasonable time. The results imply that practitioners can overcome the barrier of data sharing and produce random forest models for data-heavy domains with strict privacy requirements, such as Health Prediction, Fraud Detection, and Risk Evaluation.
More Efficient Adaptively Secure Lattice-Based IBE with Equality Test in the Standard Model

Identity-based encryption with equality test (IBEET) is a variant of identity-based encryption (IBE), where any users who have trapdoors can check whether two ciphertexts are encryption of the same plaintext. Although several lattice-based IBEET schemes have been proposed, they have drawbacks in either security or efficiency. Specifically, most schemes satisfy only selective security, while adaptively secure schemes in the standard model suffer from large master public keys that consist of linear numbers of matrices. In other words, known lattice-based IBEET schemes perform poorly compared to the state-of-the-art lattice-based IBE schemes (without equality test). In this paper, we propose a semi-generic construction of CCA-secure lattice-based IBEET from a certain class of lattice-based IBE schemes. As a result, we obtain the first lattice-based IBEET schemes with adaptive security and CCA security in the standard model. Furthermore, our semi-generic construction can use several state-of-the-art lattice-based IBE schemes as underlying schemes. Then, we have adaptively secure lattice-based IBEET schemes whose public keys have only poly-log matrices.
RGB-Thermal based Pedestrian Detection with Single-Modal Augmentation and ROI Pooling Multiscale Fusion

RGB-Thermal based pedestrian detection has received more extensive attention due to the provided detailed information and thermal sensitivity of pedestrians. In this paper, a single-modal feature augmentation network (SMA-Net) is proposed. Firstly, two single-modal branches are trained separately to optimize the feature extraction of each branch in addition to the training of pedestrian detection based on fused features. Secondly, a lightweight ROI pooling multiscale fusion module (PMSF) is proposed to obtain more fine-grained and abundant features, in which pooling features of different scales are integrated by adaptively weighting. Finally, a generative constraint strategy is designed to constrain fusion by minimizing the loss function between the generated fusion image and RGB-Thermal pairs. Experimental result on the challenging dataset KAIST demonstrates that the proposed SMA-Net achieves great performance in terms of accuracy and computational efficiency.
Linear Quantum State Observers

This paper considers the use of linear state observers to infer the unknown state (density matrix) of a closed quantum system using generalized quantum measurement—positive operator-valued measures (POVMs). An efficient test for observability is described in terms of two quantum observability matrices. The main contribution of this work is the development of a canonical quantum state observer, which may be utilized to estimate the state of any closed quantum system using POVM measurements. The observer ensures the state estimation error tends to the unobservable space of the system, which is exactly zero when the system is observable. The space of Hermitian matrices is shown to be invariant with respect to the dynamics of the canonical observer, which allows one to project the observer's state onto the space of valid quantum density operators while retaining convergence to the true state.
When Active Learning Meets Implicit Semantic Data Augmentation

Active learning (AL) is a label-efficient technique for training deep models when only a limited labeled set is available and the manual annotation is expensive. Implicit semantic data augmentation (ISDA) effectively extends the limited amount of labeled samples and increases the diversity of labeled sets without introducing a noticeable extra computational cost. The scarcity of labeled instances and the huge annotation cost of unlabelled samples encourage us to ponder on the combination of AL and ISDA. A nature direction is a pipelined integration, which selects the unlabeled samples via acquisition function in AL for labeling and generates virtual samples by changing the selected samples to semantic transformation directions within ISDA. However, this pipelined combination would not guarantee the diversity of virtual samples. This paper proposes diversity-aware semantic transformation active learning, or DAST-AL framework, that looks ahead the effect of ISDA in the selection of unlabeled samples. Specifically, DAST-AL exploits expected partial model change maximization (EPMCM) to consider selected samples’ potential contribution of the diversity to the labeled set by leveraging the semantic transformation within ISDA when selecting the unlabeled samples. After that, DAST-AL can confidently and efficiently augment the labeled set by implicitly generating more diverse samples. The empirical results on both image classification and semantic segmentation tasks show that the proposed DAST-AL can slightly outperform the state-of-the-art AL approaches. Under the same condition, the proposed method takes less than 3 min for the first cycle of active labeling while the existing agreement discrepancy selection incurs more than 40 min.
Emotion and Body Movement: A Comparative Study of Automatic Emotion Recognition Using Body Motions

Emotion recognition through body movement in both real and virtual worlds is an important research topic along with facial expression and voice recognition. Computational methods to recognize emotions based on body movement have been developed to utilize skeletal data and motion capture systems, and 2D and 3D pose estimation methods have recently been proposed. Although each of these methodologies involves advantages and disadvantages, they have not been compared with same data. In this study, we collected seven types of motion data associated with specified emotional states from 25 participants, including happiness, sadness, anger, disgust, fear, surprise, and a neutral emotion. We compared three methodologies, including motion capture, 2D pose estimation, and 3D pose estimation, along with human evaluations as a baseline. The results show that measurement through motion capture showed the highest performance, and the 2D and 3D pose estimation also showed relatively high performance compared to the human evaluators' results. These findings suggest that the existing methodologies can be utilized to perform emotion recognition.
Complex Python features in the wild

While Python is increasingly popular, program analysis tooling for Python is lagging. This is due, in part, to complex features of the Python language---features with difficult to understand and model semantics. Besides the "usual suspects", reflection and dynamic execution, complex Python features include context managers, decorators, and generators, among others. This paper explores how often and in what ways developers use certain complex features. We analyze over 3 million Python files mined from GitHub to address three research questions: (i) How often do developers use certain complex Python features? (ii) In what ways do developers use these features? (iii) Does use of complex features increase or decrease over time? Our findings show that usage of dynamic features that pose a threat to static analysis is infrequent. On the other hand, usage of context managers and decorators is surprisingly widespread. Our actionable result is a list of Python features that any "minimal syntax" ought to handle in order to capture developers' use of the Python language. We hope that understanding the usage of Python features will help tool-builders improve Python tools, which can in turn lead to more correct, secure, and performant Python code.
Granularity-Aware Adaptation for Image Retrieval Over Multiple Tasks

Strong image search models can be learned for a specific domain, i.e. set of labels, provided that some labeled images of that domain are available. A practical visual search model, however, should be versatile enough to solve multiple retrieval tasks simultaneously, even if those cover very different specialized domains. Additionally, it should be able to benefit from even unlabeled images from these various retrieval tasks. This is the more practical scenario that we consider in this paper. We address it with the proposed , an approach that starts from a strong pretrained model, and adapts it to tackle multiple retrieval tasks concurrently, using only unlabeled images from the different task domains. We extend the pretrained model with multiple independently trained sets of adaptors that use pseudo-label sets of different sizes, effectively mimicking different pseudo-granularities. We reconcile all adaptor sets into a single unified model suited for all retrieval tasks by learning fusion layers that we guide by propagating pseudo-granularity attentions across neighbors in the feature space. Results on a benchmark composed of six heterogeneous retrieval tasks show that the unsupervised model improves the zero-shot performance of a state-of-the-art self-supervised learning model, and in some places reaches or improves over a task label-aware oracle that selects the most fitting pseudo-granularity per task.
Disentangled OCR: A More Granular Information for “Text”-to-Image Retrieval

Most of the previous text-to-image retrieval methods were based on the semantic matching between text and image locally or globally. However, they ignore a very important element in both text and image, i.e., the OCR information. In this paper, we present a novel approach to disentangle the OCR from both text and image, and use the disentangled information from the two different modalities for matching. The matching score is consist of two parts, the traditional global semantic text-to-image representation matching and OCR matching scores. Since there is no dataset to support the training of text OCR disentangled task, we label partial useful data from TextCaps dataset, which contains scene text images and their corresponding captions. We relabel the text of captions to OCR and non-OCR words. In total, we extract 110K captions and 22K images from TextCaps, which contain OCR information. We call this dataset TextCaps-OCR. The experiments on TextCaps-OCR and another public dataset CTC (COCO-Text Captions) demonstrate the effectiveness of disentangling OCR in text and image for cross modality retrieval task.
Sensor-Based Open-Set Human Activity Recognition Using Representation Learning With Mixup Triplets

The main objective of sensor-based human activity recognition (HAR) is to classify predefined human physical activities with multi-channel signals acquired from wearable sensors.In a realworld scenario, signal data is changing over time and undefined activities may occur.Thus, an openset classifier for HAR is required to detect it as an unknown class rather than to assign it to the one of the known classes.However, open-set HAR is a challenging task because of the small variability of inter-activities and the large variability of intra-activities.To tackle this problem, we propose mixup triplet learning for Mahalanobis distance (MTMD) for open-set recognition.In MTMD, we introduce a new representation learning with stochastically linearly interpolated triplets (mixup triplets) that makes discriminative features of the known from the unknown.In the trained embedding space with the proposed representation learning, thresholds based on Mahalanobis distances from the centers of each known activity are defined for the proposed open-set action recognition.To demonstrate the effectiveness of MTMD, we conducted experiments on sensor-based HAR benchmark datasets.The experimental results show that the proposed MTMD outperforms the existing methods for various open-set experimental settings in terms of numerical results of F1, recall, precision, and accuracy.