Reviewerly/exHarmony huggingface dataset:
Reward Machines: Exploiting Reward Function Structure in Reinforcement Learning

Reinforcement learning (RL) methods usually treat reward functions as black boxes. As such, these methods must extensively interact with the environment in order to discover rewards and optimal policies. In most RL applications, however, users have to program the reward function and, hence, there is the opportunity to make the reward function visible – to show the reward function’s code to the RL agent so it can exploit the function’s internal structure to learn optimal policies in a more sample efficient manner. In this paper, we show how to accomplish this idea in two steps. First, we propose reward machines, a type of finite state machine that supports the specification of reward functions while exposing reward function structure. We then describe different methodologies to exploit this structure to support learning, including automated reward shaping, task decomposition, and counterfactual reasoning with off-policy learning. Experiments on tabular and continuous domains, across different tasks and RL agents, show the benefits of exploiting reward structure with respect to sample efficiency and the quality of resultant policies. Finally, by virtue of being a form of finite state machine, reward machines have the expressive power of a regular language and as such support loops, sequences and conditionals, as well as the expression of temporally extended properties typical of linear temporal logic and non-Markovian reward specification.
Multi-Scale Temporal Cues Learning for Video Person Re-Identification

Temporal cues embedded in videos provide important clues for person Re-Identification (ReID). To efficiently exploit temporal cues with a compact neural network, this work proposes a novel 3D convolution layer called Multi-scale 3D (M3D) convolution layer. The M3D layer is easy to implement and could be inserted into traditional 2D convolution networks to learn multi-scale temporal cues by end-to-end training. According to its inserted location, the M3D layer has two variants, i.e., local M3D layer and global M3D layer, respectively. The local M3D layer is inserted between 2D convolution layers to learn spatial-temporal cues among adjacent 2D feature maps. The global M3D layer is computed on adjacent frame feature vectors to learn their global temporal relations. The local and global M3D layers hence learn complementary temporal cues. Their combination introduces a fraction of parameters to traditional 2D CNN, but leads to the strong multi-scale temporal feature learning capability. The learned temporal feature is fused with a spatial feature to compose the final spatial-temporal representation for video person ReID. Evaluations on four widely used video person ReID datasets, i.e., MARS, DukeMTMC-VideoReID, PRID2011, and iLIDS-VID demonstrate the substantial advantages of our method over the state-of-the art. For example, it achieves rank1 accuracy of 88.63% on MARS without re-ranking. Our method also achieves a reasonable trade-off between ReID accuracy and model size, e.g., it saves about 40% parameters of I3D CNN.
Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Start

A standard way to address different NLP problems is by first constructing a problem-specific dataset, then building a model to fit this dataset. To build the ultimate artificial intelligence, we desire a single machine that can handle diverse new problems, for which task-specific annotations are limited. We bring up textual entailment as a unified solver for such NLP problems. However, current research of textual entailment has not spilled much ink on the following questions: (i) How well does a pretrained textual entailment system generalize across domains with only a handful of domain-specific examples? and (ii) When is it worth transforming an NLP task into textual entailment? We argue that the transforming is unnecessary if we can obtain rich annotations for this task. Textual entailment really matters particularly when the target NLP task has insufficient annotations. Universal NLP can be probably achieved through different routines. In this work, we introduce Universal Few-shot textual Entailment (UFO-Entail). We demonstrate that this framework enables a pretrained entailment model to work well on new entailment domains in a few-shot setting, and show its effectiveness as a unified solver for several downstream NLP tasks such as question answering and coreference resolution when the end-task annotations are limited.
Dynamic Context Selection for Document-level Neural Machine Translation via Reinforcement Learning

Document-level neural machine translation has yielded attractive improvements.However, majority of existing methods roughly use all context sentences in a fixed scope.They neglect the fact that different source sentences need different sizes of context.To address this problem, we propose an effective approach to select dynamic context so that the document-level translation model can utilize the more useful selected context sentences to produce better translations.Specifically, we introduce a selection module that is independent of the translation module to score each candidate context sentence.Then, we propose two strategies to explicitly select a variable number of context sentences and feed them into the translation module.We train the two modules end-to-end via reinforcement learning.A novel reward is proposed to encourage the selection and utilization of dynamic context sentences.Experiments demonstrate that our approach can select adaptive context sentences for different source sentences, and significantly improves the performance of document-level translation methods.
A robust and lightweight secure access scheme for cloud based E-healthcare services

Traditional healthcare services have transitioned into modern healthcare services where doctors remotely diagnose the patients. Cloud computing plays a significant role in this change by providing easy access to patients' medical records to all stakeholders, such as doctors, nurses, patients, life insurance agents, etc. Cloud services are scalable, cost-effective, and offer a broad range of mobile access to patients' electronic health record (EHR). Despite the cloud's enormous benefits like real-time data access, patients' EHR security and privacy are major concerns. Since the information about patients' health is highly sensitive and crucial, sharing it over the unsecured wireless medium brings many security challenges such as eavesdropping, modifications, etc. Considering the security needs of remote healthcare, this paper proposes a robust and lightweight, secure access scheme for cloud-based E-healthcare services. The proposed scheme addresses the potential threats to E-healthcare by providing a secure interface to stakeholders and prohibiting unauthorized users from accessing information stored in the cloud. The scheme makes use of multiple keys formed through the key derivation function (KDF) to ensure end-to-end ciphering of information for preventing misuse. The rights to access the cloud services are provided based on the identity and the association between stakeholders, thus ensuring privacy. Due to its simplicity and robustness, the proposed scheme is the best fit for protecting data security and privacy in cloud-based E-healthcare services.
Hardware Efficient Quantum Simulation of Non-Abelian Gauge Theories with Qudits on Rydberg Platforms

Non-abelian gauge theories underlie our understanding of fundamental forces in nature, and developing tailored quantum hardware and algorithms to simulate them is an outstanding challenge in the rapidly evolving field of quantum simulation. Here we take an approach where gauge fields, discretized in spacetime, are represented by qudits and are time-evolved in Trotter steps with multiqudit quantum gates. This maps naturally and hardware-efficiently to an architecture based on Rydberg tweezer arrays, where long-lived internal atomic states represent qudits, and the required quantum gates are performed as holonomic operations supported by a Rydberg blockade mechanism. We illustrate our proposal for a minimal digitization of SU(2) gauge fields, demonstrating a significant reduction in circuit depth and gate errors in comparison to a traditional qubit-based approach, which puts simulations of non-abelian gauge theories within reach of NISQ devices.
MFBO-SSM: Multi-Fidelity Bayesian Optimization for Fast Inference in State-Space Models

Nonlinear state-space models are ubiquitous in modeling real-world dynamical systems. Sequential Monte Carlo (SMC) techniques, also known as particle methods, are a well-known class of parameter estimation methods for this general class of state-space models. Existing SMC-based techniques rely on excessive sampling of the parameter space, which makes their computation intractable for large systems or tall data sets. Bayesian optimization techniques have been used for fast inference in state-space models with intractable likelihoods. These techniques aim to find the maximum of the likelihood function by sequential sampling of the parameter space through a single SMC approximator. Various SMC approximators with different fidelities and computational costs are often available for sample-based likelihood approximation. In this paper, we propose a multi-fidelity Bayesian optimization algorithm for the inference of general nonlinear state-space models (MFBO-SSM), which enables simultaneous sequential selection of parameters and approximators. The accuracy and speed of the algorithm are demonstrated by numerical experiments using synthetic gene expression data from a gene regulatory network model and real data from the VIX stock price index.
Private naive bayes classification of personal biomedical data: Application in cancer data analysis

Clinicians would benefit from access to predictive models for diagnosis, such as classification of tumors as malignant or benign, without compromising patients’ privacy. In addition, the medical institutions and companies who own these medical information systems wish to keep their models private when in use by outside parties. Fully homomorphic encryption (FHE) enables computation over encrypted medical data while ensuring data privacy. In this paper we use private-key fully homomorphic encryption to design a cryptographic protocol for private Naive Bayes classification. This protocol allows a data owner to privately classify his or her information without direct access to the learned model. We apply this protocol to the task of privacy-preserving classification of breast cancer data as benign or malignant. Our results show that private-key fully homomorphic encryption is able to provide fast and accurate results for privacy-preserving medical classification.
Development of a Mobile Game Application to Boost Students’ Motivation in Learning English Vocabulary

Currently, the power to master the English language has become prominent in academia, research, and business. Therefore, many non-English speaking countries, including Arab countries, are striving to improve their education systems in teaching English as a foreign language, and English vocabulary is an important factor to boost English proficiency among Arab students. Moreover, in Arab countries, there is a lack of motivation among learners of English as a foreign language which affects the learning process of students. As such, digital gaming technologies, especially mobile games, are emerging as the best way to create enthusiasm for learning new languages. Therefore, a mobile application named VocabGame was developed based on a set of persuasive guidelines, and it was launched in the Google Play Store. This paper investigated whether the developed VocabGame can motivate native Arab students learning the English language to achieve better performance. Sixty-four students were divided equally into two groups: the control group (high-performance group) and the experimental group (low-performance group). Students in the experimental group improved their motivation level significantly after the mobile learning intervention. Our findings showed that mobile game application is helpful for those students who had poor performance initially while studying English and improves their confidence. There was also an association between the pre-test and the post-test scores according to the motivation to learn based on the analysis of the covariate analysis with η_p^2 being 0.148. A mobile application game was successfully developed to motivate Arab native students to learn English as a secondary language.
<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="d1e2150" altimg="si142.svg"><mml:mrow><mml:mi>H</mml:mi><mml:mi>i</mml:mi><mml:mi>s</mml:mi></mml:mrow></mml:math>-GAN: A histogram-based GAN model to improve data generation quality

Generative Adversarial Network (GAN) has become an active research field due to its capability to generate quality simulation data. However, two consistent distributions (generated data distribution and original data distribution) produced by GAN cannot guarantee that generated data are always close to real data. Traditionally GAN is mainly applied to images, and it becomes more challenging for numeric datasets. In this paper, we propose a histogram-based GAN model (His-GAN). The purpose of our proposed model is to help GAN produce generated data with high quality. Specifically, we map generated data and original data into a histogram, then we count probability percentile on each bin and calculate dissimilarity with traditional f-divergence measures (e.g., Hellinger distance, Jensen–Shannon divergence) and Histogram Intersection Kernel. After that, we incorporate this dissimilarity score into training of the GAN model to update the generator’s parameters to improve generated data quality. This is because the parameters have an influence on the generated data quality. Moreover, we revised GAN training process by feeding GAN model with one group of samples (these samples can come from one class or one cluster that hold similar characteristics) each time, so the final generated data could contain the characteristics from a single group to overcome the challenge of figuring out complex characteristics from mixed groups/clusters of data. In this way, we can generate data that is more indistinguishable from original data. We conduct extensive experiments to validate our idea with MNIST, CIFAR-10, and a real-world numeric dataset, and the results clearly show the effectiveness of our approach.
An Investigation of Model-Free Planning

The field of reinforcement learning (RL) is facing increasingly challenging domains with combinatorial complexity. For an RL agent to address these challenges, it is essential that it can plan effectively. Prior work has typically utilized an explicit model of the environment, combined with a specific planning algorithm (such as tree search). More recently, a new family of methods have been proposed that learn how to plan, by providing the structure for planning via an inductive bias in the function approximator (such as a tree structured neural network), trained end-to-end by a model-free RL algorithm. In this paper, we go even further, and demonstrate empirically that an entirely model-free approach, without special structure beyond standard neural network components such as convolutional networks and LSTMs, can learn to exhibit many of the characteristics typically associated with a model-based planner. We measure our agent's effectiveness at planning in terms of its ability to generalize across a combinatorial and irreversible state space, its data efficiency, and its ability to utilize additional thinking time. We find that our agent has many of the characteristics that one might expect to find in a planning algorithm. Furthermore, it exceeds the state-of-the-art in challenging combinatorial domains such as Sokoban and outperforms other model-free approaches that utilize strong inductive biases toward planning.
A Real-Time Wideband Neural Vocoder at 1.6kb/s Using LPCNet

Neural speech synthesis algorithms are a promising new approach for coding speech at very low bitrate. They have so far demonstrated quality that far exceeds traditional vocoders, at the cost of very high complexity. In this work, we present a low-bitrate neural vocoder based on the LPCNet model. The use of linear prediction and sparse recurrent networks makes it possible to achieve real-time operation on general-purpose hardware. We demonstrate that LPCNet operating at 1.6 kb/s achieves significantly higher quality than MELP and that uncompressed LPCNet can exceed the quality of a waveform codec operating at low bitrate. This opens the way for new codec designs based on neural synthesis models.
The DKU Replay Detection System for the ASVspoof 2019 Challenge: On Data Augmentation, Feature Representation, Classification, and Fusion

This paper describes our DKU replay detection system for the ASVspoof 2019 challenge. The goal is to develop spoofing countermeasure for automatic speaker recognition in physical access scenario. We leverage the countermeasure system pipeline from four aspects, including the data augmentation, feature representation, classification, and fusion. First, we introduce an utterance-level deep learning framework for anti-spoofing. It receives the variable-length feature sequence and outputs the utterance-level scores directly. Based on the framework, we try out various kinds of input feature representations extracted from either the magnitude spectrum or phase spectrum. Besides, we also perform the data augmentation strategy by applying the speed perturbation on the raw waveform. Our best single system employs a residual neural network trained by the speed-perturbed group delay gram. It achieves EER of 1.04% on the development set, as well as EER of 1.08% on the evaluation set. Finally, using the simple average score from several single systems can further improve the performance. EER of 0.24% on the development set and 0.66% on the evaluation set is obtained for our primary system.
Resolution-agnostic tissue segmentation in whole-slide histopathology images with convolutional neural networks

Modern pathology diagnostics is being driven toward large scale digitization of microscopic tissue sections. A prerequisite for its safe implementation is the guarantee that all tissue present on a glass slide can also be found back in the digital image. Whole-slide scanners perform a tissue segmentation in a low resolution overview image to prevent inefficient high-resolution scanning of empty background areas. However, currently applied algorithms can fail in detecting all tissue regions. In this study, we developed convolutional neural networks to distinguish tissue from background. We collected 100 whole-slide images of 10 tissue samples—staining categories from five medical centers for development and testing. Additionally, eight more images of eight unfamiliar categories were collected for testing only. We compared our fully-convolutional neural networks to three traditional methods on a range of resolution levels using Dice score and sensitivity. We also tested whether a single neural network can perform equivalently to multiple networks, each specialized in a single resolution. Overall, our solutions outperformed the traditional methods on all the tested resolutions. The resolution-agnostic network achieved average Dice scores between 0.97 and 0.98 across the tested resolution levels, only 0.0069 less than the resolution-specific networks. Finally, its excellent generalization performance was demonstrated by achieving averages of 0.98 Dice score and 0.97 sensitivity on the eight unfamiliar images. A future study should test this network prospectively.
Quantum Long Short-Term Memory

Long short-term memory (LSTM) is a kind of recurrent neural networks (RNN) for sequence and temporal dependency data modeling and its effectiveness has been extensively established. In this work, we propose a hybrid quantum-classical model of LSTM, which we dub QLSTM. We demonstrate that the proposed model successfully learns several kinds of temporal data. In particular, we show that for certain testing cases, this quantum version of LSTM converges faster, or equivalently, reaches a better accuracy, than its classical counterpart. Due to the variational nature of our approach, the requirements on qubit counts and circuit depth are eased, and our work thus paves the way toward implementing machine learning algorithms for sequence modeling such as natural language processing, speech recognition on noisy intermediate-scale quantum (NISQ) devices.
Theoretical and Experimental Perspectives of Quantum Verification

In this perspective we discuss verification of quantum devices in the context of specific examples formulated as proposed experiments. Our first example is verification of analog quantum simulators as Hamiltonian learning, where the input Hamiltonian as the design goal is compared with the parent Hamiltonian for the quantum states prepared on the device. The second example discusses cross-device verification on the quantum level (i.e., by comparing quantum states prepared on different quantum devices). We focus in particular on protocols using randomized measurements, and we propose establishing a central data repository, where existing experimental devices and platforms can be compared. In our final example, we address verification of the output of a quantum device from a computer science perspective, addressing the question of how a user of a quantum processor can be certain of the correctness of its output, and propose minimal demonstrations on present-day devices.Received 19 January 2021DOI:https://doi.org/10.1103/PRXQuantum.2.010102Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasQuantum benchmarkingQuantum Information
A Blockchain-Based Secret-Data Sharing Framework for Personal Health Records in Emergency Condition

Blockchain technology is the most trusted all-in-one cryptosystem that provides a framework for securing transactions over networks due to its irreversibility and immutability characteristics. Blockchain network, as a decentralized infrastructure, has drawn the attention of various startups, administrators, and developers. This system preserves transactions from tampering and provides a tracking tool for tracing past network operations. A personal health record (PHR) system permits patients to control and share data concerning their health conditions by particular peoples. In the case of an emergency, the patient is unable to approve the emergency staff access to the PHR. Furthermore, a history record management system of the patient’s PHR is required, which exhibits hugely private personal data (e.g., modification date, name of user, last health condition, etc.). In this paper, we suggest a healthcare management framework that employs blockchain technology to provide a tamper protection application by considering safe policies. These policies involve identifying extensible access control, auditing, and tamper resistance in an emergency scenario. Our experiments demonstrated that the proposed framework affords superior performance compared to the state-of-the-art healthcare systems concerning accessibility, privacy, emergency access control, and data auditing.