Reviewerly/exHarmony huggingface dataset:
The Smart$^2$ Speaker Blocker: An Open-Source Privacy Filter for Connected Home Speakers

The popularity and projected growth of in-home smart speaker assistants, such as Amazon's Echo, has raised privacy concerns among consumers and privacy advocates. Notable questions regarding the collection and storage of user data by for-profit organizations include: what data is being collected and how is it being used, who has or can obtain access to such data, and how can user privacy be maintained while providing useful services. In addition to concerns regarding what the speaker manufacturer will do with your data, there are also more fundamental concerns about the security of these devices, third-party plugins, and the servers where they store recorded data. To address these privacy and security concerns, we introduce an intermediary device to provide an additional layer of security, which we call the \textit{smart, smart speaker blocker} or Smart\textsuperscript{2} for short. By intelligently filtering sensitive conversations, and completely blocking this information from reaching a smart speaker's microphone(s), the Smart$^2$ Speaker Blocker is an open-source, network-local (offline) smart device that provides users with decisive control over what data leaves their living room.
Normalized Flat Minima: Exploring Scale Invariant Definition of Flat Minima for Neural Networks using PAC-Bayesian Analysis

The notion of flat minima has played a key role in the generalization studies of deep learning models. However, existing definitions of the flatness are known to be sensitive to the rescaling of parameters. The issue suggests that the previous definitions of the flatness might not be a good measure of generalization, because generalization is invariant to such rescalings. In this paper, from the PAC-Bayesian perspective, we scrutinize the discussion concerning the flat minima and introduce the notion of normalized flat minima, which is free from the known scale dependence issues. Additionally, we highlight the scale dependence of existing matrix-norm based generalization error bounds similar to the existing flat minima definitions. Our modified notion of the flatness does not suffer from the insufficiency, either, suggesting it might provide better hierarchy in the hypothesis class.
Generating Adversarial Perturbation with Root Mean Square Gradient

We focus our attention on the problem of generating adversarial perturbations based on the gradient in image classification domain
An investigation of model-free planning

The field of reinforcement learning (RL) is facing increasingly challenging domains with combinatorial complexity. For an RL agent to address these challenges, it is essential that it can plan effectively. Prior work has typically utilized an explicit model of the environment, combined with a specific planning algorithm (such as tree search). More recently, a new family of methods have been proposed that learn how to plan, by providing the structure for planning via an inductive bias in the function approximator (such as a tree structured neural network), trained end-to-end by a model-free RL algorithm. In this paper, we go even further, and demonstrate empirically that an entirely model-free approach, without special structure beyond standard neural network components such as convolutional networks and LSTMs, can learn to exhibit many of the characteristics typically associated with a model-based planner. We measure our agent's effectiveness at planning in terms of its ability to generalize across a combinatorial and irreversible state space, its data efficiency, and its ability to utilize additional thinking time. We find that our agent has many of the characteristics that one might expect to find in a planning algorithm. Furthermore, it exceeds the state-of-the-art in challenging combinatorial domains such as Sokoban and outperforms other model-free approaches that utilize strong inductive biases toward planning.
Impact of Data Pruning on Machine Learning Algorithm Performance

Dataset pruning is the process of removing sub-optimal tuples from a dataset to improve the learning of a machine learning model. In this paper, we compared the performance of different algorithms, first on an unpruned dataset and then on an iteratively pruned dataset. The goal was to understand whether an algorithm (say A) on an unpruned dataset performs better than another algorithm (say B), will algorithm B perform better on the pruned data or vice-versa. The dataset chosen for our analysis is a subset of the largest movie ratings database publicly available on the internet, IMDb [1]. The learning objective of the model was to predict the categorical rating of a movie among 5 bins: poor, average, good, very good, excellent. The results indicated that an algorithm that performed better on an unpruned dataset also performed better on a pruned dataset.
Dynamic Multiparty Authentication of Data Analytics Services within Cloud Environments

Business analytics processes are often composed from orchestrated, collaborating services, which are consumed by users from multiple cloud systems (in different security realms), which need to be engaged dynamically at runtime. If heterogeneous cloud systems located in different security realms do not have direct authentication relationships, then it is a considerable technical challenge to enable secure collaboration. In order to address this security challenge, a new authentication framework is required to establish trust amongst business analytics service instances and users by distributing a common session secret to all participants of a session. We address this challenge by designing and implementing a secure multiparty authentication framework for dynamic interaction, for the scenario where members of different security realms express a need to access orchestrated services. This novel framework exploits the relationship of trust between session members in different security realms, to enable a user to obtain security credentials that access cloud resources in a remote realm. The mechanism assists cloud session users to authenticate their session membership, thereby improving the performance of authentication processes within multiparty sessions. We see applicability of this framework beyond multiple cloud infrastructure, to that of any scenario where multiple security realms has the potential to exist, such as the emerging Internet of Things (IoT).
Improved and Formal Proposal for Device Independent Quantum Private Query

In this paper, we propose a novel Quantum Private Query (QPQ) scheme with full Device-Independent certification. To the best of our knowledge, this is the first time we provide such a full DI-QPQ scheme using EPR-pairs. Our proposed scheme exploits self-testing of shared EPR-pairs along with the self-testing of projective measurement operators in a setting where the client and the server do not trust each other. To certify full device independence, we exploit a strategy to self-test a particular class of POVM elements that are used in the protocol. Further, we provide formal security analysis and obtain an upper bound on the maximum cheating probabilities for both the dishonest client as well as the dishonest server.
A Scalable Model for Secure Multiparty Authentication

Distributed system architectures such as cloud computing or the emergent architectures of the Internet Of Things, present significant challenges for security and privacy. Specifically, in a complex application there is a need to securely delegate access control mechanisms to one or more parties, who in turn can govern methods that enable multiple other parties to be authenticated in relation to the services that they wish to consume. We identify shortcomings in an existing proposal by Xu et al for multiparty authentication and evaluate a novel model from Al-Aqrabi et al that has been designed specifically for complex multiple security realm environments. The adoption of a Session Authority Cloud ensures that resources for authentication requests are scalable, whilst permitting the necessary architectural abstraction for myriad hardware IoT devices such as actuators and sensor networks, etc. In addition, the ability to ensure that session credentials are confirmed with the relevant resource principles means that the essential rigour for multiparty authentication is established.
Penalized estimation of flexible hidden Markov models for time series of counts

Hidden Markov models are versatile tools for modeling sequential observations, where it is assumed that a hidden state process selects which of finitely many distributions generates any given observation. Specifically for time series of counts, the Poisson family often provides a natural choice for the state-dependent distributions, though more flexible distributions such as the negative binomial or distributions with a bounded range can also be used. However, in practice, choosing an adequate class of (parametric) distributions is often anything but straightforward, and an inadequate choice can have severe negative consequences on the model's predictive performance, on state classification, and generally on inference related to the system considered. To address this issue, we propose an effectively nonparametric approach to fitting hidden Markov models to time series of counts, where the state-dependent distributions are estimated in a completely data-driven way without the need to select a distributional family. To avoid overfitting, we add a roughness penalty based on higher-order differences between adjacent count probabilities to the likelihood, which is demonstrated to produce smooth probability mass functions of the state-dependent distributions. The feasibility of the suggested approach is assessed in a simulation experiment, and illustrated in two real-data applications, where we model the distribution of i) major earthquake counts and ii) acceleration counts of an oceanic whitetip shark (Carcharhinus longimanus) over time.
Quantized Epoch-SGD for Communication-Efficient Distributed Learning

Due to its efficiency and ease to implement, stochastic gradient descent (SGD) has been widely used in machine learning. In particular, SGD is one of the most popular optimization methods for distributed learning. Recently, quantized SGD (QSGD), which adopts quantization to reduce the communication cost in SGD-based distributed learning, has attracted much attention. Although several QSGD methods have been proposed, some of them are heuristic without theoretical guarantee, and others have high quantization variance which makes the convergence become slow. In this paper, we propose a new method, called Quantized Epoch-SGD (QESGD), for communication-efficient distributed learning. QESGD compresses (quantizes) the parameter with variance reduction, so that it can get almost the same performance as that of SGD with less communication cost. QESGD is implemented on the Parameter Server framework, and empirical results on distributed deep learning show that QESGD can outperform other state-of-the-art quantization methods to achieve the best performance.
What do Language Representations Really Represent?

A neural language model trained on a text corpus can be used to induce distributed representations of words, such that similar words end up with similar representations. If the corpus is multilingual, the same model can be used to learn distributed representations of languages, such that similar languages end up with similar representations. We show that this holds even when the multilingual corpus has been translated into English, by picking up the faint signal left by the source languages. However, just like it is a thorny problem to separate semantic from syntactic similarity in word representations, it is not obvious what type of similarity is captured by language representations. We investigate correlations and causal relationships between language representations learned from translations on one hand, and genetic, geographical, and several levels of structural similarity between languages on the other. Of these, structural similarity is found to correlate most strongly with language representation similarity, while genetic relationships---a convenient benchmark used for evaluation in previous work---appears to be a confounding factor. Apart from implications about translation effects, we see this more generally as a case where NLP and linguistic typology can interact and benefit one another.
Making AI meaningful again

Artificial intelligence (AI) research enjoyed an initial period of enthusiasm in the 1970s and 80s. But this enthusiasm was tempered by a long interlude of frustration when genuinely useful AI applications failed to be forthcoming. Today, we are experiencing once again a period of enthusiasm, fired above all by the successes of the technology of deep neural networks or deep machine learning. In this paper we draw attention to what we take to be serious problems underlying current views of artificial intelligence encouraged by these successes, especially in the domain of language processing. We then show an alternative approach to language-centric AI, in which we identify a role for philosophy.
The Effectiveness of Inside-Outside Circle and Jigsaw Techniques in Teaching Interpersonal Conversation for High and Low Self-Confident Students

This study is an experimental research with a factorial design which aimed to find out effectiveness of inside-outside circle and jigsaw techniques in teaching interpersonal conversation for high and low self-confident students. The samples of the study were tenth graders of SMA Krista Mitra Semarang the 2018/2019 academic year. There were two classes, experimental class I and experimental class II which every class consisted of 20 students. Inside-outside Circle technique was used in the experimental class I, while Jigsaw technique was used in experimental class II. To answer research questions number one up to four, T-test was used. To answer research question number five and six, studentsâ€™ mean scores in experimental class I and II were compared. While two-ways ANOVA with F-test at the 5% (0.05) level of significance was used to answer the seventh question. The result of this study showed that inside-outside circle and jigsaw techniques are effective to be used in teaching interpersonal conversation for high and low self-confident students. There were no significant differences of achievement between high and low self-confidence students using inside-outside circle and jigsaw techniques. For the interaction among the variables, it shows that there was no interaction among the techniques, studentsâ€™ self-confidence, and studentsâ€™ interpersonal conversation skill.&#x0D; &#x0D; &#x0D;
Complexity Bounds for the Controllability of Temporal Networks with Conditions, Disjunctions, and Uncertainty

In temporal planning, many different temporal network formalisms are used to model real world situations. Each of these formalisms has different features which affect how easy it is to determine whether the underlying network of temporal constraints is consistent. While many of the simpler models have been well-studied from a computational complexity perspective, the algorithms developed for advanced models which combine features have very loose complexity bounds. In this paper, we provide tight completeness bounds for strong, weak, and dynamic controllability checking of temporal networks that have conditions, disjunctions, and temporal uncertainty. Our work exposes some of the subtle differences between these different structures and, remarkably, establishes a guarantee that all of these problems are computable in PSPACE.
Team Papelo: Transformer Networks at FEVER

We develop a system for the FEVER fact extraction and verification challenge that uses a high precision entailment classifier based on transformer networks pretrained with language modeling, to classify a broad set of potential evidence. The precision of the entailment classifier allows us to enhance recall by considering every statement from several articles to decide upon each claim. We include not only the articles best matching the claim text by TFIDF score, but read additional articles whose titles match named entities and capitalized expressions occurring in the claim text. The entailment module evaluates potential evidence one statement at a time, together with the title of the page the evidence came from (providing a hint about possible pronoun antecedents). In preliminary evaluation, the system achieves .5736 FEVER score, .6108 label accuracy, and .6485 evidence F1 on the FEVER shared task test set.
Spectral Clustering via Ensemble Deep Autoencoder Learning (SC-EDAE)

Recently, a number of works have studied clustering strategies that combine classical clustering algorithms and deep learning methods. These approaches follow either a sequential way, where a deep representation is learned using a deep autoencoder before obtaining clusters with k-means, or a simultaneous way, where deep representation and clusters are learned jointly by optimizing a single objective function. Both strategies improve clustering performance, however the robustness of these approaches is impeded by several deep autoencoder setting issues, among which the weights initialization, the width and number of layers or the number of epochs. To alleviate the impact of such hyperparameters setting on the clustering performance, we propose a new model which combines the spectral clustering and deep autoencoder strengths in an ensemble learning framework. Extensive experiments on various benchmark datasets demonstrate the potential and robustness of our approach compared to state-of-the-art deep clustering methods.
FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network

This paper develops the FastRNN and FastGRNN algorithms to address the twin RNN limitations of inaccurate training and inefficient prediction. Previous approaches have improved accuracy at the expense of prediction costs making them infeasible for resource-constrained and real-time applications. Unitary RNNs have increased accuracy somewhat by restricting the range of the state transition matrix's singular values but have also increased the model size as they require a larger number of hidden units to make up for the loss in expressive power. Gated RNNs have obtained state-of-the-art accuracies by adding extra parameters thereby resulting in even larger models. FastRNN addresses these limitations by adding a residual connection that does not constrain the range of the singular values explicitly and has only two extra scalar parameters. FastGRNN then extends the residual connection to a gate by reusing the RNN matrices to match state-of-the-art gated RNN accuracies but with a 2-4x smaller model. Enforcing FastGRNN's matrices to be low-rank, sparse and quantized resulted in accurate models that could be up to 35x smaller than leading gated and unitary RNNs. This allowed FastGRNN to accurately recognize the "Hey Cortana" wakeword with a 1 KB model and to be deployed on severely resource-constrained IoT microcontrollers too tiny to store other RNN models. FastGRNN's code is available at https://github.com/Microsoft/EdgeML/.
Hypergraphic Degree Sequences are Hard

We show that deciding if a given vector is the degree sequence of a 3-hypergraph is NP-complete.
Comments on "Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?"

In a recently published paper [1], it is shown that deep neural networks (DNNs) with random Gaussian weights preserve the metric structure of the data, with the property that the distance shrinks more when the angle between the two data points is smaller. We agree that the random projection setup considered in [1] preserves distances with a high probability. But as far as we are concerned, the relation between the angle of the data points and the output distances is quite the opposite, i.e., smaller angles result in a weaker distance shrinkage. This leads us to conclude that Theorem 3 and Figure 5 in [1] are not accurate. Hence the usage of random Gaussian weights in DNNs cannot provide an ability of universal classification or treating in-class and out-of-class data separately. Consequently, the behavior of networks consisting of random Gaussian weights only is not useful to explain how DNNs achieve state-of-art results in a large variety of problems.
A New Perspective on Machine Learning: How to do Perfect Supervised Learning

In this work, we introduce the concept of bandlimiting into the theory of machine learning because all physical processes are bandlimited by nature, including real-world machine learning tasks. After the bandlimiting constraint is taken into account, our theoretical analysis has shown that all practical machine learning tasks are asymptotically solvable in a perfect sense. Furthermore, the key towards this solvability almost solely relies on two factors: i) a sufficiently large amount of training samples beyond a threshold determined by a difficulty measurement of the underlying task; ii) a sufficiently complex and bandlimited model. Moreover, for some special cases, we have derived new error bounds for perfect learning, which can quantify the difficulty of learning. These generalization bounds are not only asymptotically convergent but also irrelevant to model complexity. Our new results on generalization have provided a new perspective to explain the recent successes of large-scale supervised learning using complex models like neural networks.
Vector representations of text data in deep learning

In this dissertation we report results of our research on dense distributed representations of text data. We propose two novel neural models for learning such representations. The first model learns representations at the document level, while the second model learns word-level representations. For document-level representations we propose Binary Paragraph Vector: a neural network models for learning binary representations of text documents, which can be used for fast document retrieval. We provide a thorough evaluation of these models and demonstrate that they outperform the seminal method in the field in the information retrieval task. We also report strong results in transfer learning settings, where our models are trained on a generic text corpus and then used to infer codes for documents from a domain-specific dataset. In contrast to previously proposed approaches, Binary Paragraph Vector models learn embeddings directly from raw text data. For word-level representations we propose Disambiguated Skip-gram: a neural network model for learning multi-sense word embeddings. Representations learned by this model can be used in downstream tasks, like part-of-speech tagging or identification of semantic relations. In the word sense induction task Disambiguated Skip-gram outperforms state-of-the-art models on three out of four benchmarks datasets. Our model has an elegant probabilistic interpretation. Furthermore, unlike previous models of this kind, it is differentiable with respect to all its parameters and can be trained with backpropagation. In addition to quantitative results, we present qualitative evaluation of Disambiguated Skip-gram, including two-dimensional visualisations of selected word-sense embeddings.