Reviewerly/exHarmony huggingface dataset:
Prediction of Reward Functions for Deep Reinforcement Learning via Gaussian Process Regression

Inverse reinforcement learning (IRL) is a technique for automatic reward acquisition, however, it is difficult to apply to high-dimensional problems with unknown dynamics. This article proposes an efficient way to solve the IRL problem based on the sparse Gaussian process (GP) prediction with l1-regularization only using a highly limited number of expert demonstrations. A GP model is proposed to be trained to predict a reward function using trajectory-reward pair data generated by deep reinforcement learning with different reward functions. The trained GP successfully predicts the reward functions of human experts from their collected demonstration trajectory datasets. To demonstrate our approach, the proposed approach is applied to the obstacle avoidance navigation of the mobile robot. The experimental results clearly show that the robots can clone the experts' optimality in navigation trajectories avoiding obstacles using only with a very small number of expert demonstration datasets (e.g., ≤ 6). Therefore, the proposed approach shows great potential to be applied to complex real-world applications in an expert data-efficient manner.
Bayesian Optimal Control of Greenberger-Horne-Zeilinger States in Rydberg Lattices

The ability to prepare non-classical states in a robust manner is essential for quantum sensors beyond the standard quantum limit. We demonstrate that Bayesian optimal control is capable of finding control pulses that drive trapped Rydberg atoms into highly entangled GHZ states. The control sequences have a physically intuitive functionality based on the quasi-integrability of the Ising dynamics. They can be constructed in laboratory experiments resulting in preparation times that scale very favourably with the system size.
Long-Short Term Echo State Network for Time Series Prediction

The Echo State Networks (ESNs) is an efficient recurrent neural network consisting of a randomly generated reservoir (a large number of neurons with sparse random recurrent connections) and a trainable linear layer. It has received widespread attention for its simplicity and effectiveness, especially for time series prediction tasks. However, there is no explicit mechanism in ESNs to capture the inherent multi-scale characteristics of time series. To this end, we propose a model consisting of multi-reservoir structure named long-short term echo state networks (LS-ESNs) to capture the multi-scale temporal characteristics of time series. Specifically, LS-ESNs consists of three independent reservoirs, and each reservoir has recurrent connections of a specific time-scale to model the temporal dependencies of time series. The multi-scale echo states are then collected from each reservoir and concatenated together. Finally, the concatenated echo states representations are fed to the linear regression layer to obtain the results. Experiments on two time series prediction benchmark data sets and a real-world power load data sets demonstrate the effectiveness of the proposed LS-ESNs.
The Glasgow Subgraph Solver: Using Constraint Programming to Tackle Hard Subgraph Isomorphism Problem Variants

The Glasgow Subgraph Solver provides an implementation of state of the art algorithms for subgraph isomorphism problems. It combines constraint programming concepts with a variety of strong but fast domain-specific search and inference techniques, and is suitable for use on a wide range of graphs, including many that are found to be computationally hard by other solvers. It can also be equipped with side constraints, and can easily be adapted to solve other subgraph matching problem variants. We outline its key features from the view of both users and algorithm developers, and discuss future directions.
On Sharing Models Instead of Data using Mimic learning for Smart Health Applications

Electronic health records (EHR) systems contain vast amounts of medical information about patients. These data can be used to train machine learning models that can predict health status, as well as to help prevent future diseases or disabilities. However, getting patients' medical data to obtain well-trained machine learning models is a challenging task. This is because sharing the patients' medical records is prohibited by law in most countries due to patients privacy concerns. In this paper, we tackle this problem by sharing the models instead of the original sensitive data by using the mimic learning approach. The idea is first to train a model on the original sensitive data, called the teacher model. Then, using this model, we can transfer its knowledge to another model, called the student model, without the need to learn the original data used in training the teacher model. The student model is then shared to the public and can be used to make accurate predictions. To assess the mimic learning approach, we have evaluated our scheme using different medical datasets. The results indicate that the student model mimics the teacher model performance in terms of prediction accuracy without the need to access to the patients' original data records.
An examination of social media as a platform for cyber-violence against the LGBT+ population

The popularity of social media raises concerns related to cyber-violence and the security of marginalised individuals and groups, including the lesbian, gay, bisexual and transgender (LGBT+) population. Developing worthy interventions requires exploration of the LGBT+ population’s experiences of cyber-violence in relation to gender discrimination, which was the aim of this study. A qualitative approach sourced data between 2017 and 2019 from LGBT+ Facebook groups and pages and semi-structured interviews with participants who identified themselves as among the LGBT+ population. Keywords such as LGBT+, homosexuals, and isiZulu terms such as ‘izitabane’ and ‘inkonkoni’ were used to search for content. It was found that Facebook is used as a platform by heterosexuals to make violent and hateful comments against the LBGT+ population. Comments displayed to the public (including, for example, “gays are dogs, they can never transform to being women”) reflected heteronormative behaviour. A significant finding was the infiltration of heterosexual individuals into the space created by the LGBT+ population, suggesting an increased risk of cyber-violence, and that the right to privacy and security is often compromised. Addressing cyberbullying of the LGBT+ population through education on gender diversity is recommended.
D2D-Enabled Data Sharing for Distributed Machine Learning at Wireless Network Edge

Mobile edge learning is an emerging technique that enables distributed edge devices to collaborate in training shared machine learning (ML) models by exploiting their local data samples and communication/computation resources. To deal with the “straggler's dilemma” issue faced in this technique, this letter proposes a new device-to-device (D2D)-enabled data sharing approach, in which different edge devices share their data samples among each other over D2D communication links, in order to properly adjust their computation loads for increasing the training speed. Under this setup, we optimize the radio resource allocation for both D2D-enabled data sharing and distributed training, with the objective of minimizing the total training delay under fixed numbers of local and global iterations (for training). Numerical results show that the proposed D2D-enabled data sharing design significantly reduces the training delay, and also enhances the training accuracy when the data samples are non-independent and identically distributed (non-IID) among edge devices.
PEBERS: Practical Ethereum Blockchain based Efficient Ride Hailing Service

Although the current ride hailing services have revolutionized the transportation industry in today's world, they are extremely centralized. Central authority has all the control and maintains data about the users. Such platforms raise concerns about the service policies and data reliability. In case the central server faces data tampering or ransomware attacks, all the records of data is either lost or compromised. Blockchain has gained exceptional recognition while addressing aforementioned issues because of its decentralized data auditability, anonymity and immutability. Research on blockchain technology explores the possibility of implementing decentralized application in ride hailing service. Blockchain based ride hailing architecture gives us the opportunity to deviate from centralized platforms to decentralized ones. In this paper we propose our system PEBERS: Practical Ethereum Blockchain based Efficient Ride Hailing Service, in which we demonstrate how decentralized system based on consortium blockchain can be developed to keep track of ride data. In this context we explore smart contracts to build and deploy functionalities such as create ride, auto deposit transfer, cancel and complete ride methods of our decentralized ride hailing application. Our experiments show the driver smart contracts proposed in our system consume less Gas and hence prove that its an efficient system than many other existing systems with respect to the expenses of passengers and profitability of drivers. This is an effort towards providing smarter transportation for a society moving towards smart city concept.
Structured Dictionary Learning for Image Denoising Under Mixed Gaussian and Impulse Noise

Although image denoising as a basic task of image restoration has been widely studied in the past decades, there are not many studies on mixed noise denoising. In this paper, we propose two structured dictionary learning models to recover images corrupted by mixed Gaussian and impulse noise. These two models can be merged as ℓp-norm fidelity plus ℓq-norm regularization. The fidelity term is used to fit image patches and the regularization term is employed for sparse coding. Particularly, we utilize proximal (and proximal linearized) alternating minimization methods as the main solvers to deal with these two models. We remove the Gaussian noise under the assumption that the uncorrupted image can be approximated with a linear representation under an appropriate orthogonal basis. We use different ways to remove impulse noise for these two models. The experimental results are reported to compare the existing methods and demonstrate the performance of the proposed denoising model is better than the other existing methods in terms of some quality assessment metrics.
Multi-entity sentiment analysis using self-attention based hierarchical dilated convolutional neural network

In this paper, a self-attention based hierarchical dilated convolutional neural network (SA-HDCNN) is proposed for multi-entity sentiment analysis (MESA), in which the task is directly transformed into a sequence labeling problem avoiding decomposition and is also suitable for parallel computing. Specifically, SA-HDCNN is mainly composed of encoding, feature extraction and decoding modules. The encoding module is to map the input sentence into a word embedding matrix, which contains both semantic and sentiment information. Next, the feature extraction module is to learn multi-scale local features and inter-word global dependencies of the encoded sentence through HDCNN architecture and self-attention mechanism, respectively. Afterwards, the decoding module is to output the sequence of tags, thereby completing the automatic recognition of multiple target entities and their corresponding sentiment polarities. Finally, experimental results demonstrate that, without any domain specific features or prior information, our method surpasses the existing state-of-the-art methods, implying its feasibility and wide applicability.
Enemy Within: Long-term Motivation Effects of Deep Player Behavior Models for Dynamic Difficulty Adjustment

Balancing games and producing content that remains interesting and challenging is a main cost factor in the design and maintenance of games. Dynamic difficulty adjustments (DDA) can successfully tune challenge levels to player abilities, but when implemented with classic heuristic parameter tuning (HPT) often turns out to be very noticeable, e.g. as "rubber-banding". Deep learning techniques can be employed for deep player behavior modeling (DPBM), enabling more complex adaptivity, but effects over frequent and longer-lasting game engagements, as well as how it compares to HPT has not been empirically investigated. We present a situated study of the effects of DDA via DPBM as compared to HPT on intrinsic motivation, perceived challenge and player motivation in a real-world MMORPG. The results indicate that DPBM can lead to significant improvements in intrinsic motivation and players prefer game experience episodes featuring DPBM over experience episodes with classic difficulty management.
Content-based remote sensing image retrieval using multi-scale local ternary pattern

This paper discusses a content based remote sensing image retrieval technique using multi-scale, patch-based local ternary pattern and fisher vector encoding. The technique downsamples an image in three scales, each downsampled image utilizes local ternary pattern to obtain upper and lower texture images, and divides them into dense patches to build a final histogram representation. This representation is then encoded into normalized fisher vectors. To this end, we focus on two standard remote sensing datasets namely 20-class satellite remote sensing dataset and 21-class land-cover dataset. Visual and quantitative results signify high precision for the proposed technique with an improvement of approximately 5.71% and 6.57% for 21-class land-cover and 20-class satellite remote sensing datasets respectively.
Towards More Diverse Input Representation for Neural Machine Translation

Source input information plays a very important role in the Transformer-based translation system. In practice, word embedding and positional embedding of each word are added as the input representation. Then self-attention networks are used to encode the global dependencies in the input representation to generate a source representation. However, this processing on the source representation only adopts a single source feature and excludes richer and more diverse features such as recurrence features, local features, and syntactic features, which results in tedious representation and thereby hinders the further translation performance improvement. In this paper, we introduce a simple and efficient method to encode more diverse source features into the input representation simultaneously, and thereby learning an effective source representation by self-attention networks. In particular, the proposed grouped strategy is only applied to the input representation layer, to keep the diversity of translation information and the efficiency of the self-attention networks at the same time. Experimental results show that our approach improves the translation performance over the state-of-the-art baselines of Transformer in regard to WMT14 English-to-German and NIST Chinese-to-English machine translation tasks.
Using Artificial Intelligence and Novel Polynomials to Predict Subjective Refraction

Abstract This work aimed to use artificial intelligence to predict subjective refraction from wavefront aberrometry data processed with a novel polynomial decomposition basis. Subjective refraction was converted to power vectors (M, J0, J45). Three gradient boosted trees (XGBoost) algorithms were trained to predict each power vector using data from 3729 eyes. The model was validated by predicting subjective refraction power vectors of 350 other eyes, unknown to the model. The machine learning models were significantly better than the paraxial matching method for producing a spectacle correction, resulting in a mean absolute error of 0.301 ± 0.252 Diopters (D) for the M vector, 0.120 ± 0.094 D for the J0 vector and 0.094 ± 0.084 D for the J45 vector. Our results suggest that subjective refraction can be accurately and precisely predicted from novel polynomial wavefront data using machine learning algorithms. We anticipate that the combination of machine learning and aberrometry based on this novel wavefront decomposition basis will aid the development of refined algorithms which could become a new gold standard to predict refraction objectively.
Implementing action mask in proximal policy optimization (PPO) algorithm

The proximal policy optimization (PPO) algorithm is a promising algorithm in reinforcement learning. In this paper, we propose to add an action mask in the PPO algorithm. The mask indicates whether an action is valid or invalid for each state. Simulation results show that, when compared with the original version, the proposed algorithm yields much higher return with a moderate number of training steps. Therefore, it is useful and valuable to incorporate such a mask if applicable.
Adaptive Complexity for Pixel-Value-Ordering Based Reversible Data Hiding

Pixel-value-ordering (PVO) is a widely used reversible data hiding (RDH) framework which aims to achieve the high quality of stego-image under low capacity. In this letter, we propose a general location-based adaptive complexity for PVO. Different from the block-based complexity in the previous PVO-based methods, our proposed method adaptively selects context pixels from the perspective of the relative locations of predicted pixel and prediction pixel. Consequently, different number of high correlation context pixels can be adaptively selected and the context pixels can break the limitation of the current block. Moreover, instead of sharing the same block complexity by two predicted pixels in the current block, each predicted pixel can be utilized independently according to its own corresponding complexity. Our proposed adaptive complexity can combine with any PVO-based methods and the experimental results show that our proposed method achieves a significant improvement in prediction accuracy and embedding performance.
Path Planning of Restaurant Service Robot Based on A-star Algorithms with Updated Weights

Through the real-time gridded restaurant congestion map, aiming at the shortest weighted path, and based on the degree of channel congestion to change the weight of the restaurant channel in real-time, the A Star algorithm is improved to find the optimal restaurant service robot path, and the gridded map of a restaurant is used to verify the results. The results show that, compared with the traditional gridded map of a restaurant, the proposed algorithm is effective. A star algorithm with constant path weight and improved A star algorithm can effectively avoid crowded channels and improve the mobile efficiency of service robots. It has a certain practical application value in restaurant service robot system.
Learned Data Structures

Very recently, the unexpected combination of data structures and machine learning has led to the development of a new area of research, called learned data structures. Their distinguishing trait is the ability to reveal and exploit patterns and trends in the input data for achieving more efficiency in time and space, compared to previously known data structures. The goal of this chapter is to provide the first comprehensive survey of these results and to stimulate further research in this promising area.
A Deep Multi-task Contextual Attention Framework for Multi-modal Affect Analysis

Multi-modal affect analysis (e.g., sentiment and emotion analysis) is an interdisciplinary study and has been an emerging and prominent field in Natural Language Processing and Computer Vision. The effective fusion of multiple modalities (e.g., text , acoustic, or visual frames ) is a non-trivial task, as these modalities, often, carry distinct and diverse information, and do not contribute equally. The issue further escalates when these data contain noise. In this article, we study the concept of multi-task learning for multi-modal affect analysis and explore a contextual inter-modal attention framework that aims to leverage the association among the neighboring utterances and their multi-modal information. In general, sentiments and emotions have inter-dependence on each other (e.g., anger → negative or happy → positive ). In our current work, we exploit the relatedness among the participating tasks in the multi-task framework. We define three different multi-task setups, each having two tasks, i.e., sentiment 8 emotion classification, sentiment classification 8 sentiment intensity prediction, and emotion classificati on 8 emotion intensity prediction. Our evaluation of the proposed system on the CMU-Multi-modal Opinion Sentiment and Emotion Intensity benchmark dataset suggests that, in comparison with the single-task learning framework, our multi-task framework yields better performance for the inter-related participating tasks. Further, comparative studies show that our proposed approach attains state-of-the-art performance for most of the cases.
Augmented Reality as a Resource for Improving Learning in the Physical Education Classroom

Recently, there has been a proliferation of technopedagogical practices, based on the application of active teaching and learning processes through the use of information and communication technologies (ICT). The main objective of this work is to analyse the impact of training action through the use of augmented reality in physical education for the development and acquisition of spatial orientation, as opposed to more traditional training based on the exhibition method. The methodology developed is quantitative, through a quasi-experimental design post-test in 140 high school students in the field of physical education. The results indicate that all of the dimensions show a very high, significant relationship. The greatest difference in average is observed in motivation. In contrast, the smallest difference, although significant, is observed in the grades given by the teachers themselves. It can be concluded that the method of teaching through augmented reality is effective in teaching high school students in the subject of physical education, especially for the acquisition of spatially oriented content.
Building and Validating a Scale for Secure Software Development Self-Efficacy

Security is an essential component of the software development lifecycle. Researchers and practitioners have developed educational interventions, guidelines, security analysis tools, and new APIs aimed at improving security. However, measuring any resulting improvement in secure development skill is challenging. As a proxy for skill, we propose to measure self-efficacy, which has been shown to correlate with skill in other contexts. Here, we present a validated scale measuring secure software-development self-efficacy (SSD-SES). We first reviewed popular secure-development frameworks and surveyed 22 secure-development experts to identify 58 unique tasks. Next, we asked 311 developers - over multiple rounds - to rate their skill at each task. We iteratively updated our questions to ensure they were easily understandable, showed adequate variance between participants, and demonstrated reliability. Our final 15-item scale contains two sub-scales measuring belief in ability to perform vulnerability identification and mitigation as well as security communications tasks.