Reviewerly/exHarmony huggingface dataset:
Histogram Matched Visible and Infrared Image Registration for Face Detection

Face detection is a common image processing task in applications such as pedestrian monitoring, crowd flow prediction, security video surveillance, and fever screening. Thermal imaging is used in conjunction with a visible light camera for improved monitoring of people. Often information from multi-modal sources such as IR/visible images is fused for efficient face detection. The accuracy of the face detection system relies on the quality of the image registration procedure. In this paper, we propose an enhanced intensity-based image registration technique for thermal and visible facial images. The method uses a histogram matching scheme to equalize the intensity levels of the IR/visible images for proper image registration. The proposed scheme when validated with facial images shows an improved registration quality when quantified using structural similarity (SSIM) and mutual information (MI) metric for facial IR and visible images.
YOLOv4-object: an Efficient Model and Method for Object Discovery

Object discovery refers to recognising all unknown objects in images, which is crucial for robotic systems to explore the unseen environment. Recently, object detection models based on deep learning have shown remarkable achievements in object classification and localisation. However, these models have difficulties handling the unseen environment because it is infeasible to exhaustively predefine all types of objects. In this paper, we propose the model YOLOv4-object to recognise all objects in images by modifying the output space of YOLOv4 and related image labels. Experiments on COCO dataset demonstrate the effectiveness of our method by achieving 67.97% recall (6.49% higher than vanilla YOLOv4). We point out that the incomplete labels (COCO only labels for 80 categories) hurt the learning process of object discovery and a higher recall can be achieved by our method if the dataset is fully labelled. Moreover, our approach is transferable, extensible, and compressible, showing broad application scenarios. Finally, we conduct extensive experiments to illustrate the factors that affect the object discovery performance of our model and some suggestions on practical implementations are elaborated.
Improved Differential-ML Distinguisher: Machine Learning Based Generic Extension for Differential Analysis

At CRYPTO 2019, Gohr first proposes a deep learning based differential analysis on round-reduced Speck32/64. Then Yadav \(et \, al.\) present a framework to construct the differential-ML (machine learning) distinguisher by combining the traditional differential distinguisher and the machine learning based differential distinguisher, which breaks the limit of the ML differential distinguisher on the number of attack rounds. However, the results obtained based on this method are not necessarily better than the results gained by traditional analysis. In this paper, we offer three novel greedy strategies (\(M_1\), \(M_2\) and \(M_3\)) to solve this problem. The strategy \(M_1\) provides better differential-ML distinguishers by considering all combinations of classical differential distinguishers and ML differential distinguishers. And the strategy \(M_2\) uses the best ML differential distinguishers to splice classical differential distinguishers forward, while the strategy \(M_3\) adopts the best classical differential distinguishers to splice ML differential distinguishers. As proof of works, we apply our methods to round-reduced Speck32/64, Speck48/72 and Speck64/96 and get some improved cryptanalysis results. For the construction of differential-ML distinguishers, we can reach 11-round Speck32/64, 14-round Speck48/72 and 18-round Speck64/96 with \(2^{27}\), \(2^{45}\), \(2^{62}\) data respectively.
Pruning Broad Learning System based on Adaptive Feature Evolution

The newly proposed Broad Learning System (BLS) offers an alternative way to deep learning which saves a time-consuming training process and powerful computing resources. However, the randomly generated feature nodes and lots of enhancement nodes in BLS may reduce the performance of the final classifier. Aiming at the problems in randomly generated feature nodes which suffer from unpredictability and need guidance, this paper proposes an adaptive feature nodes evolutionary algorithm (AFNE) to extract better features; While broad learning neural network often requires a large number of enhancement nodes parameters to ensure its performance, which easily leads to the redundancy or dependency between features, as well as the performance degradation of the final model. Therefore, this article also proposes a new criterion based on node sensitivity to prune the network enhancement layer nodes to remove redundant nodes, reduce the network scale, and increase the generalization ability. The proposed algorithm ADP-BLS can improve the accuracy and generalization performance of the final classifier through the evolution of feature nodes and the pruning of enhancement nodes. Extensive comparative experiments on real world data sets verify the effectiveness of the proposed ADP-BLS. At the same time, when verifying the module validity of the innovative algorithms added in this article, experiments also show that the AFNE and pruning method integrated into BLS can improve the model to a certain extent.
(What) Can Deep Learning Contribute to Theoretical Linguistics?

Abstract Deep learning (DL) techniques have revolutionised artificial systems’ performance on myriad tasks, from playing Go to medical diagnosis. Recent developments have extended such successes to natural language processing, an area once deemed beyond such systems’ reach. Despite their different goals (technological development vs. theoretical insight), these successes have suggested that such systems may be pertinent to theoretical linguistics. The competence/performance distinction presents a fundamental barrier to such inferences. While DL systems are trained on linguistic performance, linguistic theories are aimed at competence. Such a barrier has traditionally been sidestepped by assuming a fairly close correspondence: performance as competence plus noise. I argue this assumption is unmotivated. Competence and performance can differ arbitrarily. Thus, we should not expect DL models to illuminate linguistic theory.
A hierarchical and parallel framework for End-to-End Aspect-based Sentiment Analysis

Pipeline, joint, and collapsed models are three major approaches to solving End-to-End Aspect-based Sentiment Analysis (E2E-ABSA) task. Prior works found that joint models were consistently surpassed by the other two. To explore the potential of joint model for E2E-ABSA, we propose a hierarchical and parallel joint framework on the basis of exploiting the hierarchical nature of the pre-trained language model and performing parallel inference of the subtasks. Our framework: (1) shares the same pre-trained backbone network between two subtasks, ensuring the associations and commonalities between them; (2) considers the hierarchical feature of the deep neural network and introduces two joint approaches, namely the specific-layer joint model and multiple-layer joint model, coupling two specific layers or multiple task-related layers with subtasks; (3) carries out parallel execution in both training and inference processes, improving the inference throughput and al-leviating the target-polarity mismatch problem. The experimental results on three benchmark datasets demonstrate that our approach outperforms the state-of-the-art works.
Research and Application of Personalized Recommendation Based on Knowledge Graph

Text data resources in the power domain have become increasingly abundant in recent years with the large scale popularization of information office in the power sector, but workers are facing an increasingly severe problem of data information overload. Since the concept of knowledge graph have been proposed, researchers have used professional datasets in various fields to construct corresponding knowledge graphs and proposed various knowledge graph completion algorithms to solve the problem of missing entity and relation links. In this paper, we introduce the knowledge graph as auxiliary information into the recommendation system of power domain. Our method uses translation-based models to learn the representations of users and items and applies them to optimize the recommender system. In addition, to address users diverse interests, we also build user profiles in our method to aggregate a users history with respect to candidate items. According to the characteristics of the data and the representativeness and universality of the data, extensive experiments are conducted on the Citeulike. We apply our approach to the power domain and construct the knowledge graph of the power domain dataset. The results validate the effectiveness of our approach on recommendation.
Model-Based Transfer Reinforcement Learning Based on Graphical Model Representations

Reinforcement learning (RL) plays an essential role in the field of artificial intelligence but suffers from data inefficiency and model-shift issues. One possible solution to deal with such issues is to exploit transfer learning. However, interpretability problems and negative transfer may occur without explainable models. In this article, we define Relation Transfer as explainable and transferable learning based on graphical model representations, inferring the skeleton and relations among variables in a causal view and generalizing to the target domain. The proposed algorithm consists of the following three steps. First, we leverage a suitable casual discovery method to identify the causal graph based on the augmented source domain data. After that, we make inferences on the target model based on the prior causal knowledge. Finally, offline RL training on the target model is utilized as prior knowledge to improve the policy training in the target domain. The proposed method can answer the question of what to transfer and realize zero-shot transfer across related domains in a principled way. To demonstrate the robustness of the proposed framework, we conduct experiments on four classical control problems as well as one simulation to the real-world application. Experimental results on both continuous and discrete cases demonstrate the efficacy of the proposed method.
An Optimal Seed Scheduling Strategy Algorithm Applied to Cyberspace Mimic Defense

Cyberspace mimic defense (CMD) is an active defense theory that has emerged in recent years. By dynamically constructing and scheduling multiple executors, the CMD can not only effectively defend against security threats caused by unknown cyberspace weaknesses, but also improve the present situation of information asymmetry between attack and defense in cyberspace. However, as one of the key technologies of the CMD, the scheduling strategy algorithm still needs to be improved in real-time security, reliability, and universality, which has restricted the development and large-scale deployment of the CMD. To solve this problem, we propose an optimal seed scheduling strategy algorithm (OSSSA) in this paper. After using continuous-time Markov processes to mathematically analyze the model of the mimic defense system in cyberspace, we introduce the key factors and their evaluation methods that affecting the CMD defense performance in the OSSSA, then propose a reliable working mechanism of the OSSSA. Furthermore, we present an evaluation method that can effectively evaluate the defense performance of the scheduling algorithms. Experimental results from software simulations and real experiments both show that the OSSSA has better real-time defense performance than the current mainstream scheduling strategy algorithms, and can adapt to different practical application scenarios, which is helpful to the further development of the CMD.
Learning Geometric Transformations for Parametric Design: An Augmented Reality (AR)-Powered Approach

Despite the remarkable development of parametric modeling methods for architectural design, a significant problem still exists, which is the lack of knowledge and skill regarding the professional implementation of parametric design in architectural modeling. Considering the numerous advantages of digital/parametric modeling in rapid prototyping and simulation most instructors encourage students to use digital modeling even from the early stages of design; however, an appropriate context to learn the basics of digital design thinking is rarely provided in architectural pedagogy. This paper presents an educational tool, specifically an Augmented Reality (AR) intervention, to help students understand the fundamental concepts of parametric modeling before diving into complex parametric modeling platforms. The goal of the AR intervention is to illustrate geometric transformation and the associated math functions so that students learn the mathematical logic behind the algorithmic thinking of parametric modeling. We have developed BRICKxAR/T, an educational AR prototype, that intends to help students learn geometric transformations in an immersive spatial AR environment. A LEGO set is used within the AR intervention as a physical manipulative to support physical interaction and improve spatial skill through body gesture.
A Survey on Blockchain Data Analysis

With the development of blockchain technology, the data on the blockchain represented by Bitcoin and Ethereum is exploding. Data represents people's activities, indicating that blockchain has been paid more and more attention. However, many problems are hidden behind so much data, such as security and privacy. Analyzing the data can help to find out the problems and propose methods to improve blockchain. Therefore, in order to make the blockchain well applied to various walks of life, data analysis is quite essential. In this paper, we collected the relevant literature in the field of blockchain data analysis, summarized the current analysis methods, and analyzed the research status from four aspects: security, privacy, performance and prediction of price. Finally, we discussed those research work and forecasted the future trend and challenges in this field, providing some reference for related researchers. We thought that applying machine learning technology to blockchain data analysis will become mainstream in the future.
Embedding Heterogeneous Information Network in Hyperbolic Spaces

Heterogeneous information network (HIN) embedding, aiming to project HIN into a low-dimensional space, has attracted considerable research attention. Most of the existing HIN embedding methods focus on preserving the inherent network structure and semantic correlations in Euclidean spaces. However, one fundamental problem is whether the Euclidean spaces are the intrinsic spaces of HIN? Recent researches find the complex network with hyperbolic geometry can naturally reflect some properties, e.g., hierarchical and power-law structure. In this article, we make an effort toward embedding HIN in hyperbolic spaces. We analyze the structures of three HINs and discover some properties, e.g., the power-law distribution, also exist in HINs. Therefore, we propose a novel HIN embedding model HHNE. Specifically, to capture the structure and semantic relations between nodes, HHNE employs the meta-path guided random walk to sample the sequences for each node. Then HHNE exploits the hyperbolic distance as the proximity measurement. We also derive an effective optimization strategy to update the hyperbolic embeddings iteratively. Since HHNE optimizes different relations in a single space, we further propose the extended model HHNE++. HHNE++ models different relations in different spaces, which enables it to learn complex interactions in HINs. The optimization strategy of HHNE++ is also derived to update the parameters of HHNE++ in a principle manner. The experimental results demonstrate the effectiveness of our proposed models.
LAMP: Learning a Motion Policy to Repeatedly Navigate in an Uncertain Environment

Mobile robots are often tasked with repeatedly navigating through an environment whose traversability changes over time. These changes may exhibit some hidden structure, which can be learned. Many studies consider reactive algorithms for online planning, however, these algorithms do not take advantage of the past executions of the navigation task for future tasks. In this article, we formalize the problem of minimizing the total expected cost to perform multiple start-to-goal navigation tasks on a roadmap by introducing the learned reactive planning problem. We propose a method that captures information from past executions to learn a motion policy to handle obstacles that the robot has seen before. We propose the LAMP framework, which integrates the generated motion policy with an existing navigation stack. Finally, an extensive set of experiments in simulated and real-world environments show that the proposed method outperforms the state-of-the-art algorithms by 10% to 40% in terms of expected time to travel from start to goal. We also evaluate the robustness of the proposed method in the presence of localization and mapping errors on a real robot.
Countering Machine-Learning Classification of Applications by Equalizing Network Traffic Statistics

We propose to equalize primary statistical metrics to decrease adversarial classification accuracy of network traffic classifiers that are based on supervised machine learning (ML) algorithms. We use packet count, inter-arrival times, and packet length as primary traffic metrics and estimate their effect on classification accuracy when used for application identification. Internet traffic classification has been used for website fingerprinting and application layer protocols in the past. Here, we consider for identifying six popular but different user applications (e.g., Skype and others). We evaluate the performance of many ML algorithms in the identification of user applications on intact traffic collected from an actual production campus network and show that these ML algorithms classify the considered applications with an average precision and recall of up to 0.96. We apply the three proposed equalizing methods and show that the modified statistics decrease the average precision and recall of the considered traffic classifiers to 0.56. We discuss the efficacy of each of the proposed methods through exhaustive evaluations on actual network traffic and the effect of the implementation of these methods on other traffic properties.
Method of Analyzing and Managing Volleyball Action by Using Action Sensor of Mobile Device

With the development of electronic technology and sensor technology, more and more intelligent electronic devices integrate micro inertial sensors, which makes the research of human action recognition based on action sensing data have great application value. Data-based action recognition is a new research direction in the field of pattern recognition, which is essentially a process of action data acquisition, feature extraction, feature extraction, and recognition, the process of classification and recognition. Inertial motion information includes acceleration and angular velocity information, which is ubiquitous in daily life. Compared with motion recognition based on visual information, it can more directly reflect the meaning of action. This study mainly discusses the method of analyzing and managing volleyball action by using the action sensor of mobile device. Based on the motion recognition algorithm of support vector machine, the motion recognition process of support vector machine is constructed. When the data terminal and gateway of volleyball players are not in the same LAN, the classification algorithm classifies the samples to be tested through the characteristic data, which directly affects the recognition results. In this paper, the support vector machine algorithm is selected as the data classification algorithm, and the calculation of the classification process is reduced by designing an appropriate kernel function. For multiclass problems, the hierarchical structure of directed acyclic graph is optimized to improve the recognition rate. We need to bind motion sensors to human joints. In order to realize real-time recognition of human motion, mobile devices need to add windows to the motion capture data, that is, divide the data into a small sequence of specified length, and provide more application scenarios for the device. This method of embedding motion sensors into devices to read motion information is widely used, which provides a convenient data acquisition method for human motion pattern recognition based on motion information. The multiclassification support vector machine algorithm is used to train the classification algorithm model with action data. When the signal strength of the sensor is 90 t and the speed is 2.0 m/s and 0.5 m/s, the detection accuracy of the adaptive threshold is 93% and 95%, respectively. The results show that the SVM method based on hybrid kernel function can greatly improve the recognition accuracy of volleyball stroke, and the recognition time is short.
A Systematical Solution for Face De-identification

With the identity information in face data more closely related to personal credit and property security, people pay increasing attention to the protection of face data privacy. In different tasks, people have various requirements for face de-identification (De-ID), so we propose a systematical solution compatible for these De-ID operations. Firstly, an attribute disentanglement and generative network is constructed to encode two parts of the face, which are the identity (facial features like mouth, nose and eyes) and expression (including expression, pose and illumination). Through face swapping, we can remove the original ID completely. Secondly, we add an adversarial vector mapping network to perturb the latent code of the face image, different from previous traditional adversarial methods. Through this, we can construct unrestricted adversarial image to decrease ID similarity recognized by model. Our method can flexibly de-identify the face data in various ways and the processed images have high image quality.
SKGSUM: Abstractive Document Summarization with Semantic Knowledge Graphs

In abstractive single-document summarization task, generated summaries always suffer from fabricated and less informative content. An intuitive way to alleviate this problem is to merge external semantic knowledge into the model framework. In this paper, we incorporate explicit graphs based on semantic knowledge, including term frequency, discourse information, and entities with their relations, into neural abstractive summarization for the problem. We propose a novel model for abstractive single-document Summarization based on Semantic Knowledge Graphs (SKGSUM), which regards sentences and entities as nodes, captures the relations between units in different textual levels, and focuses on salient content in the source documents to guide the summary generation process. To the best of our knowledge, we are the first to exploit different textual-unit levels explicit graph representations in a unified framework for the neural abstractive summarization task. Results show that our model achieves significant improvements on both XSum and CNN/Daily Mail datasets over some strong baselines. Human evaluations further indicate that our model can generate informative and coherent summaries.