Reviewerly/exHarmony huggingface dataset:
Learning From Irregularly-Sampled Time Series: A Missing Data Perspective

Irregularly-sampled time series occur in many domains including healthcare. They can be challenging to model because they do not naturally yield a fixed-dimensional representation as required by many standard machine learning models. In this paper, we consider irregular sampling from the perspective of missing data. We model observed irregularly-sampled time series data as a sequence of index-value pairs sampled from a continuous but unobserved function. We introduce an encoder-decoder framework for learning from such generic indexed sequences. We propose learning methods for this framework based on variational autoencoders and generative adversarial networks. For continuous irregularly-sampled time series, we introduce continuous convolutional layers that can efficiently interface with existing neural network architectures. Experiments show that our models are able to achieve competitive or better classification results on irregularly-sampled multivariate time series compared to recent RNN models while offering significantly faster training times.
Dataset Condensation with Gradient Matching

As the state-of-the-art machine learning methods in many fields rely on larger datasets, storing datasets and training models on them become significantly more expensive. This paper proposes a training set synthesis technique for data-efficient learning, called Dataset Condensation, that learns to condense large dataset into a small set of informative synthetic samples for training deep neural networks from scratch. We formulate this goal as a gradient matching problem between the gradients of deep neural network weights that are trained on the original and our synthetic data. We rigorously evaluate its performance in several computer vision benchmarks and demonstrate that it significantly outperforms the state-of-the-art methods. Finally we explore the use of our method in continual learning and neural architecture search and report promising gains when limited memory and computations are available.
Neural Clustering Processes

Probabilistic clustering models (or equivalently, mixture models) are basic building blocks in countless statistical models and involve latent random variables over discrete spaces. For these models, posterior inference methods can be inaccurate and/or very slow. In this work we introduce deep network architectures trained with labeled samples from any generative model of clustered datasets. At test time, the networks generate approximate posterior samples of cluster labels for any new dataset of arbitrary size. We develop two complementary approaches to this task, requiring either O(N) or O(K) network forward passes per dataset, where N is the dataset size and K the number of clusters. Unlike previous approaches, our methods sample the labels of all the data points from a well-defined posterior, and can learn nonparametric Bayesian posteriors since they do not limit the number of mixture components. As a scientific application, we present a novel approach to neural spike sorting for high-density multielectrode arrays.
Generating Reasonable Legal Text through the Combination of Language Modeling and Question Answering

Due to the improvement of Language Modeling, the emerging NLP assistant tools aiming for text generation greatly reduce the human workload on writing documents. However, the generation of legal text faces greater challenges than ordinary texts because of its high requirement for keeping logic reasonable, which can not be guaranteed by Language Modeling right now. To generate reasonable legal documents, we propose a novel method CoLMQA, which (1) combines Language Modeling and Question Answering, (2) generates text with slots by Language Modeling, and (3) fills the slots by our proposed Question Answering method named Transformer-based Key-Value Memory Networks. In CoLMQA, the slots represent the text part that needs to be highly constrained by logic, such as the name of the law and the number of the law article. And the Question Answering fills the slots in context with the help of Legal Knowledge Base to keep logic reasonable. The experiment verifies the quality of legal documents generated by CoLMQA, surpassing the documents generated by pure Language Modeling.
Email based Spam Detection

Nowadays, a big part of people rely on available email or messages sent by the stranger.The possibility that anybody can leave an email or a message provides a golden opportunity for spammers to write spam message about our different interests .Spam fills inbox with number of ridiculous emails .Degrades our internet speed to a great extent .Steals useful information like our details on our contact list.Identifying these spammers and also the spam content can be a hot topic of research and laborious tasks.Email spam is an operation to send messages in bulk by mail .Since the expense of the spam is borne mostly by the recipient ,it is effectively postage due advertising.Spam email is a kind of commercial advertising which is economically viable because email could be a very cost effective medium for sender .With this proposed model the specified message can be stated as spam or not using Bayes' theorem and Naive Bayes' Classifier and Also IP addresses of the sender are often detected .
Variational Neural Machine Translation with Normalizing Flows

Variational Neural Machine Translation (VNMT) is an attractive framework for modeling the generation of target translations, conditioned not only on the source sentence but also on some latent random variables. The latent variable modeling may introduce useful statistical dependencies that can improve translation accuracy. Unfortunately, learning informative latent variables is non-trivial, as the latent space can be prohibitively large, and the latent codes are prone to be ignored by many translation models at training time. Previous works impose strong assumptions on the distribution of the latent code and limit the choice of the NMT architecture. In this paper, we propose to apply the VNMT framework to the state-of-the-art Transformer and introduce a more flexible approximate posterior based on normalizing flows. We demonstrate the efficacy of our proposal under both in-domain and out-of-domain conditions, significantly outperforming strong baselines.
Argot: Generating Adversarial Readable Chinese Texts

Natural language processing (NLP) models are known vulnerable to adversarial examples, similar to image processing models. Studying adversarial texts is an essential step to improve the robustness of NLP models. However, existing studies mainly focus on analyzing English texts and generating adversarial examples for English texts. There is no work studying the possibility and effect of the transformation to another language, e.g, Chinese. In this paper, we analyze the differences between Chinese and English, and explore the methodology to transform the existing English adversarial generation method to Chinese. We propose a novel black-box adversarial Chinese texts generation solution Argot, by utilizing the method for adversarial English samples and several novel methods developed on Chinese characteristics. Argot could effectively and efficiently generate adversarial Chinese texts with good readability. Furthermore, Argot could also automatically generate targeted Chinese adversarial text, achieving a high success rate and ensuring readability of the Chinese.
Unsupervised Qualitative Scoring for Binary Item Features

Abstract Binary features, such as categories, keywords, or tags, are widely used to describe product properties. However, these features are incomplete in that they do not contain several aspects of numerical information. The qualitative score of tags is widely used to describe which product is better in terms of the given property. For example, in a restaurant navigation site, properties such as mood, dishes, and location are given in the form of numerical values, representing the goodness of each aspect. In this paper, we propose a novel approach to estimate the qualitative score from the binary features of products. Based on a natural assumption that an item with a better property is more popular among users who prefer that property, in short, “experts know best,” we introduce both discriminative and generative models with which user preferences and item qualitative scores are inferred from user--item interactions. We constrain the space of the item qualitative score by item binary features so that the score of each item and tag can only have nonzero values when the item has the corresponding tag. This approach contributes to resolving the following difficulties: (1) no supervised data for the score estimation, (2) implicit user purpose, and (3) irrelevant tag contamination. We evaluate our models by using two artificial datasets and two real-world datasets of movie and book ratings. In the experiment, we evaluate the performances of our model under sparse transaction and noisy tag settings by using two artificial datasets. We also evaluate our models’ resolution for irrelevant tags using the real-world dataset of movie ratings and observe that our models outperform a baseline model. Finally, tag rankings obtained from the real-world datasets are compared with a baseline model.
Optimal Policy for Deployment of Machine Learning Models on Energy-Bounded Systems

With the recent advances in both machine learning and embedded systems research, the demand to deploy computational models for real-time execution on edge devices has increased substantially. Without deploying computational models on edge devices, the frequent transmission of sensor data to the cloud results in rapid battery draining due to the energy consumption of wireless data transmission. This rapid power dissipation leads to a considerable reduction in the battery lifetime of the system, therefore jeopardizing the real-world utility of smart devices. It is well-established that for difficult machine learning tasks, models with higher performance often require more computation power and thus are not power-efficient choices for deployment on edge devices. However, the trade-offs between performance and power consumption are not well studied. While numerous methods (e.g., model compression) have been developed to obtain an optimal model, these methods focus on improving the efficiency of a "single" model. In an entirely new direction, we introduce an effective method to find a combination of "multiple" models that are optimal in terms of power-efficiency and performance by solving an optimization problem in which both performance and power consumption are taken into account. Experimental results demonstrate that on the ImageNet dataset, we can achieve a 20% energy reduction with only 0.3% accuracy drop compared to Squeeze-and-Excitation Networks. Compared to a pruned convolutional neural network for human activity recognition, while consuming 1.7% less energy, our proposed policy achieves 1.3% higher accuracy.
Adaptive Gradient Descent without Descent

We present a strikingly simple proof that two rules are sufficient to automate gradient descent: 1) don't increase the stepsize too fast and 2) don't overstep the local curvature. No need for functional values, no line search, no information about the function except for the gradients. By following these rules, you get a method adaptive to the local geometry, with convergence guarantees depending only on the smoothness in a neighborhood of a solution. Given that the problem is convex, our method converges even if the global smoothness constant is infinity. As an illustration, it can minimize arbitrary continuously twice-differentiable convex function. We examine its performance on a range of convex and nonconvex problems, including logistic regression and matrix factorization.
On Forgetting to Cite Older Papers: An Analysis of the ACL Anthology

The field of natural language processing is experiencing a period of unprecedented growth, and with it a surge of published papers.This represents an opportunity for us to take stock of how we cite the work of other researchers, and whether this growth comes at the expense of "forgetting" about older literature.In this paper, we address this question through bibliographic analysis.We analyze the age of outgoing citations in papers published at selected ACL venues between 2010 and 2019, finding that there is indeed a tendency for recent papers to cite more recent work, but the rate at which papers older than 15 years are cited has remained relatively stable.
ACCEPTANCE AND USER EXPERIENCE OF WEARABLE ASSISTIVE DEVICES FOR INDUSTRIAL PURPOSES

Abstract This paper presents experience-oriented aspects of the development of wearable assistive devices (exoskeletons) for industrial purposes, an area which has only begun to be explored. Our research aims to examine user acceptance criteria for assistive devices and understand the meaning of interaction with wearable assistive devices for the users. The resulting models deliver new insights about the importance of user experience for technology acceptance and should be generally considered in development processes of wearable assistive devices.
TG-GAN: Continuous-time Temporal Graph Deep Generative Models with Time-Validity Constraints

Deep generative models of graph-structured data have become popular in very recent years. Although initial research has focused on static graphs in applications such as molecular design and social networks, many challenges involve temporal graphs whose topology and attribute values evolve dynamically over time. Sophisticated and unknown network processes that affect temporal graphs cannot be captured adequately by prescribed models. Application areas include social mobility networks and catastrophic cybersecurity failures. These web-scale applications challenge current deep graph generative models with the need to capture 1) time-validity constraints, 2) time and topological distributions, and 3) joint time and graph encoding and decoding. Here, we propose the “Temporal Graph Generative Adversarial Network” (TG-GAN) for continuous-time graph generation with time-validity constraints 1. TG-GAN can jointly generate the time, node, and edge information for truncated temporal walks via a novel recurrent-based model and a valid time decoder. The generated truncated temporal walks are then assembled into time-budgeted temporal walks for temporal graphs under the learned topological and temporal dependencies. In addition, a discriminator is proposed to combine time and node encoding operations over a recurrent architecture to distinguish generated sequences from real ones sampled by a truncated temporal walk sampler. Extensive experiments on both synthetic and real-world datasets confirm that TG-GAN significantly outperforms five benchmarking methods in terms of efficiency and effectiveness.
Classification of QPSK Signals with Different Phase Noise Levels Using Deep Learning

Spectrum awareness allows the understanding of the wireless systems environment and it gives engineers and designers better control in systems design and analysis. Phase noise is one of the characteristics of the channel distortion or device distortion, which causes transmission errors. In this paper, a deep learning network is utilized to study and identify different phase noise levels for quadrature phase shift keying (QPSK) signals. Our experiment results show that the deep learning neural network is capable of classifying a wide range of phase noise levels.
Low Resource Sequence Tagging using Sentence Reconstruction

This work revisits the task of training sequence tagging models with limited resources using transfer learning.We investigate several proposed approaches introduced in recent works and suggest a new loss that relies on sentence reconstruction from normalized embeddings.Specifically, our method demonstrates how by adding a decoding layer for sentence reconstruction, we can improve the performance of various baselines.We show improved results on the CoNLL02 NER and UD 1.2 POS datasets and demonstrate the power of the method for transfer learning with low-resources achieving 0.6 F1 score in Dutch using only one sample from it.The code is publicly available at: https://github.com/tperl/Low-Resource-
Balancing a CartPole System with Reinforcement Learning -- A Tutorial

In this paper, we provide the details of implementing various reinforcement learning (RL) algorithms for controlling a Cart-Pole system. In particular, we describe various RL concepts such as Q-learning, Deep Q Networks (DQN), Double DQN, Dueling networks, (prioritized) experience replay and show their effect on the learning performance. In the process, the readers will be introduced to OpenAI/Gym and Keras utilities used for implementing the above concepts. It is observed that DQN with PER provides best performance among all other architectures being able to solve the problem within 150 episodes.
Parallelized Swarm Intelligence Approach for Solving TSP and JSSP Problems

One of the possible approaches to solving difficult optimization problems is applying population-based metaheuristics. Among such metaheuristics, there is a special class where searching for the best solution is based on the collective behavior of decentralized, self-organized agents. This study proposes an approach in which a swarm of agents tries to improve solutions from the population of solutions. The process is carried out in parallel threads. The proposed algorithm—based on the mushroom-picking metaphor—was implemented using Scala in an Apache Spark environment. An extended computational experiment shows how introducing a combination of simple optimization agents and increasing the number of threads may improve the results obtained by the model in the case of TSP and JSSP problems.
Blind quantum computation where a user only performs single-qubit gates

Blind quantum computation (BQC) allows a user who has limited quantum ability to complete a quantum computational task with the aid of a remote quantum server, such that the user’s input, output, and even the algorithm can be kept hidden from the server. Up to now, there are mainly two models of BQC. One is that the client just needs the ability to prepare single qubits initiated by Broadbent, Fitzsimons, and Kashefi and the other is that the client only needs to perform single-qubit measurements first given by Morimae. In this paper, we put forward a new model of BQC in which a user only requires implementing a few single-qubit gates. We also propose a specific BQC protocol where a user only needs to implement two kinds of single-qubit gates to show the feasibility of the presented model and thus answer positively the open question whether a verified BQC protocol that only requires single-qubit gates for the user can be designed. This circuit model is quite flexible since various users with the ability to perform different single-qubit gates may all have the chance to achieve BQC. Furthermore, compared with the other two models, it may be more suitable for practical implementation in some experimental setups such as trapped ions and superconducting systems since the single-qubit gates are the most exact operations in such systems.
Virtual image of a hand displaced in space influences action performance of the real hand

Abstract The rubber hand illusion (RHI) demonstrates that under some circumstances a fake hand can be regarded as part of one’s body; the RHI and related phenomena have been used to explore the flexibility of the body schema. Recent work has shown that a sense of embodiment may be generated by virtual reality (VR). In a series of experiments, we used VR to assess the effects of the displacement of the virtual image of subjects’ hands on action. Specifically, we tested whether spatial and temporal parameters of action change when participants perform a reaching movement towards the location of their virtual hand, the position of which was distorted on some trials. In different experiments, participants were sometimes provided with incorrect visual feedback regarding the position of the to-be-touched hand (Experiment 1), were deprived of visual feedback regarding the position of the reaching hand when acting (Experiment 2) or reached with the hand, the apparent position of which had been manipulated (Experiment 3). The effect was greatest when participants reached towards (Experiment 1) or with (Experiment 3) the displaced hand when the hand was visible during the reaching, but not when the vision of the hand was removed during the action (Experiment 2). Taken together, these data suggest that visual images of one’s hand presented in VR influence the body schema and action performance.
To Explain or Not to Explain: A Study on the Necessity of Explanations for Autonomous Vehicles

Explainable AI, in the context of autonomous systems, like self-driving cars, has drawn broad interests from researchers. Recent studies have found that providing explanations for autonomous vehicles' actions has many benefits (e.g., increased trust and acceptance), but put little emphasis on when an explanation is needed and how the content of explanation changes with driving context. In this work, we investigate which scenarios people need explanations and how the critical degree of explanation shifts with situations and driver types. Through a user experiment, we ask participants to evaluate how necessary an explanation is and measure the impact on their trust in self-driving cars in different contexts. Moreover, we present a self-driving explanation dataset with first-person explanations and associated measures of the necessity for 1103 video clips, augmenting the Berkeley Deep Drive Attention dataset. Our research reveals that driver types and driving scenarios dictate whether an explanation is necessary. In particular, people tend to agree on the necessity for near-crash events but hold different opinions on ordinary or anomalous driving situations.
Employees’ Social Graph Analysis: A Model of Detection the Most Criticality Trajectories of the Social Engineering Attack’s Spread

In this research we present the hybrid model of finding the most critical distribution trajectories of multipath Social engineering attacks, passing through which by the malefactor on a global basis has the topmost degree of probability and will bring the greatest loss to the company. The solution of search problem concerning the most critical trajectories rests upon the assumption that the estimated probabilities of the direct Social engineering attack on user, degree evaluation of documents’ criticality, the estimated probabilities of Social engineering attack’s distribution from user to user are premised on linguistic indistinct variables are already calculated. The described model finds its application at creation when constructing the estimates of information systems users’ safety against Social engineering attacks and promotes well-timed informing of decision-makers on the vulnerabilities which being available in system.
Automatic Management of Cloud Applications with Use of Proximal Policy Optimization

Reinforcement learning is a very active field of research with many practical applications. Success in many cases is driven by combining it with Deep Learning. In this paper we present the results of our attempt to use modern advancements in this area for automated management of resources used to host distributed software. We describe the use of an autonomous agent that employs a policy trained with use of Proximal Policy Optimization algorithm. The agent is managing a cloud infrastructure used to process a sample workload. We present the design and architecture of a complete autonomous management system and explain how the management policy was trained. Finally, we compare the performance to the traditional automatic management approach exploited in AWS stack and discuss feasibility to use the presented approach in other scenarios.