Reviewerly/exHarmony huggingface dataset:
Global Optimization with Ensemble Machine Learning Models

Gradient boosted trees and other regression tree models are known to perform well in a wide range of real-world, industrial applications. These tree models (i) offer insight into important prediction features, (ii) effectively manage sparse data, and (iii) have excellent prediction capabilities. We consider holistic decision-making problems where pre-trained tree models are part of larger optimization tasks. Our contributions include: (i) explicitly integrating model uncertainty considerations, (ii) solving the larger optimization problems that incorporate these uncertain tree models, (iii) proving that the resulting solutions are globally optimal, i.e., no better solution exists.
BaFFLe: Backdoor detection via Feedback-based Federated Learning.

Recent studies have shown that federated learning (FL) is vulnerable to poisoning attacks that inject a backdoor into the global model. These attacks are effective even when performed by a single client, and undetectable by most existing defensive techniques. In this paper, we propose Backdoor detection via Feedback-based Federated Learning (BAFFLE), a novel defense to secure FL against backdoor attacks. The core idea behind BAFFLE is to leverage data of multiple clients not only for training but also for uncovering model poisoning. We exploit the availability of diverse datasets at the various clients by incorporating a feedback loop into the FL process, to integrate the views of those clients when deciding whether a given model update is genuine or not. We show that this powerful construct can achieve very high detection rates against state-of-the-art backdoor attacks, even when relying on straightforward methods to validate the model. Through empirical evaluation using the CIFAR-10 and FEMNIST datasets, we show that by combining the feedback loop with a method that suspects poisoning attempts by assessing the per-class classification performance of the updated model, BAFFLE reliably detects state-of-the-art backdoor attacks with a detection accuracy of 100% and a false-positive rate below 5%. Moreover, we show that our solution can detect adaptive attacks aimed at bypassing the defense.
A Scalable Mixture Model Based Defense Against Data Poisoning Attacks on Classifiers

Classifiers, e.g., those based on Naive Bayes, a support vector machine, or even a neural network, are highly susceptible to a data-poisoning attack. The attack objective is to degrade classification accuracy by covertly embedding malicious (labeled) samples into the training set. Such attacks can be mounted by an insider, through an outsourcing process (for data acquisition or training), or conceivably during active learning. In some cases, a very small amount of poisoning can result in dramatic reduction in classification accuracy. Data poisoning attacks are successful mainly because the malicious injected samples significantly skew the data distribution of the corrupted class. Such attack samples are generally data outliers and in principle separable from the clean samples. We propose a generalized, scalable, and dynamic data driven defense system that: 1) uses a mixture model both to well-fit the (potentially multi-modal) data and to give potential to isolate attack samples in a small subset of the mixture components; 2) performs hypothesis testing to decide both which components and which samples within those components are poisoned, with the identified poisoned ones purged from the training set. Our approaches addresses the attack scenario where adversarial samples are an unknown subset embedded in the initial training set, and can be used to perform data sanitization as a precursor to the training of any type of classifier. The promising results for experiments on the TREC05 spam corpus and Amazon reviews polarity dataset demonstrate the effectiveness of our defense strategy.
Lipschitz Bounds and Provably Robust Training by Laplacian Smoothing

In this work we propose a graph-based learning framework to train models with provable robustness to adversarial perturbations. In contrast to regularization-based approaches, we formulate the adversarially robust learning problem as one of loss minimization with a Lipschitz constraint, and show that the saddle point of the associated Lagrangian is characterized by a Poisson equation with weighted Laplace operator. Further, the weighting for the Laplace operator is given by the Lagrange multiplier for the Lipschitz constraint, which modulates the sensitivity of the minimizer to perturbations. We then design a provably robust training scheme using graph-based discretization of the input space and a primal-dual algorithm to converge to the Lagrangian's saddle point. Our analysis establishes a novel connection between elliptic operators with constraint-enforced weighting and adversarial learning. We also study the complementary problem of improving the robustness of minimizers with a margin on their loss, formulated as a loss-constrained minimization problem of the Lipschitz constant. We propose a technique to obtain robustified minimizers, and evaluate fundamental Lipschitz lower bounds by approaching Lipschitz constant minimization via a sequence of gradient $p$-norm minimization problems. Ultimately, our results show that, for a desired nominal performance, there exists a fundamental lower bound on the sensitivity to adversarial perturbations that depends only on the loss function and the data distribution, and that improvements in robustness beyond this bound can only be made at the expense of nominal performance. Our training schemes provably achieve these bounds both under constraints on performance and~robustness.
Combining Deep Reinforcement Learning and Search for Imperfect-Information Games

The combination of deep reinforcement learning and search at both training and test time is a powerful paradigm that has led to a number of successes in single-agent settings and perfect-information games, best exemplified by AlphaZero. However, prior algorithms of this form cannot cope with imperfect-information games. This paper presents ReBeL, a general framework for self-play reinforcement learning and search that provably converges to a Nash equilibrium in any two-player zero-sum game. In the simpler setting of perfect-information games, ReBeL reduces to an algorithm similar to AlphaZero. Results in two different imperfect-information games show ReBeL converges to an approximate Nash equilibrium. We also show ReBeL achieves superhuman performance in heads-up no-limit Texas hold'em poker, while using far less domain knowledge than any prior poker AI.
Automated Detection of Drift in Deep Learning Based Classifiers Performance Using Network Embeddings

In supervised learning, neural network generalize, or learn, based on various statistical properties of the training data. Typically, an unseen randomly sampled test set is used to estimate the performance (e.g., accuracy) of the neural network during deployment time. The performance on the test set is used to project the performance of the neural network at deployment time under the implicit assumption that the data distribution of the test set represents the data distribution at deployment time. This assumption often breaks in real world deployment scenarios. Statistic characteristics of the data may change between training time and deployment time. For example, new unforeseen objects may emerge in the data, the data may become noisy due to external conditions (e.g., rain), or characteristic of objects may change over time (e.g., subjects with a new group age). All of these changes in the data, may degrade the performance of the neural network below an anticipated and desired threshold. However, if the production data is not labeled, as is typically the case in deployment time, detection of such a drift in network performance is not straightforward. We develop a technique that identifies drift in neural network performance in time windows for which the data is not labeled. We take advantage of the network structure, specifically the vector of activation level at the layer before the last for a given data point x, to define the statistic used to identify drift. We use cosine similarity as statistics defined on the vector of activation level at the layer before the last. We then apply non-parametric statistical tests on a sliding window of the time series of data arrivals to identify drift. The approach efficiently detects drift and is only sensitive to the choice of the sliding window which needs to have sufficient overlap with the window in which the drift is gradually introduced.
Low-Resource Multi-Domain Machine Translation for Spanish-Farsi: Neural or Statistical?

An important prerequisite for data-driven Machine Translation (MT) systems is the availability of high-quality training data. Corpus-based MT systems requiring domain specificity additionally require the selection of a large training dataset and application of proper domain adaptation techniques. This paper demonstrates the portability of MT systems to multiple domains under low-resource conditions for the target language, where target-language domain data are not available. We refer to this case as a bilingually low-resource language pair. We compare the performance of an attention-based Neural MT (NMT) system to that of a phrase-based Statistical MT (SMT) system under this condition, training each on parallel corpora consisting of different domains. Experimental results on Spanish-Farsi as a bilingually low-resource language pair show that the SMT paradigm still outperforms that of NMT.
Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization

The problem of inverse reinforcement learning (IRL) is relevant to a variety of tasks including value alignment and robot learning from demonstration. Despite significant algorithmic contributions in recent years, IRL remains an ill-posed problem at its core; multiple reward functions coincide with the observed behavior and the actual reward function is not identifiable without prior knowledge or supplementary information. This paper presents an IRL framework called Bayesian optimization-IRL (BO-IRL) which identifies multiple solutions that are consistent with the expert demonstrations by efficiently exploring the reward function space. BO-IRL achieves this by utilizing Bayesian Optimization along with our newly proposed kernel that (a) projects the parameters of policy invariant reward functions to a single point in a latent space and (b) ensures nearby points in the latent space correspond to reward functions yielding similar likelihoods. This projection allows the use of standard stationary kernels in the latent space to capture the correlations present across the reward function space. Empirical results on synthetic and real-world environments (model-free and model-based) show that BO-IRL discovers multiple reward functions while minimizing the number of expensive exact policy optimizations.
Meta-Gradient Reinforcement Learning with an Objective Discovered Online

Deep reinforcement learning includes a broad family of algorithms that parameterise an internal representation, such as a value function or policy, by a deep neural network. Each algorithm optimises its parameters with respect to an objective, such as Q-learning or policy gradient, that defines its semantics. In this work, we propose an algorithm based on meta-gradient descent that discovers its own objective, flexibly parameterised by a deep neural network, solely from interactive experience with its environment. Over time, this allows the agent to learn how to learn increasingly effectively. Furthermore, because the objective is discovered online, it can adapt to changes over time. We demonstrate that the algorithm discovers how to address several important issues in RL, such as bootstrapping, non-stationarity, and off-policy learning. On the Atari Learning Environment, the meta-gradient algorithm adapts over time to learn with greater efficiency, eventually outperforming the median score of a strong actor-critic baseline.
Butterfly Optimization Algorithm

Butterfly Optimization Algorithm (BOA) is a unique algorithm inspired by nature. The algorithm replicates the behavior of the natural butterfly which can be described to perform a cooperative movement while navigating towards its food source and position. The butterflies are well known for having the power to analyze and receive smell in the air thereby tracing and discovering the direction of their partner and food source. BOA model was implemented in MATLAB to illustrate the approach, 30 search agents were considered (n = 30) and the maximum number of iterations was set to 1000. The best solution obtained by BOA is [3.0175 1.9708] and the best optimal value of the objective function found by BOA is 0.015431. The parameter space and the convergence test are presented. BOA is considered very useful for solving complex optimization problems.
Learning discrete distributions with infinite support

We present a novel approach to estimating discrete distributions with (potentially) infinite support in the total variation metric. In a departure from the established paradigm, we make no structural assumptions whatsoever on the sampling distribution. In such a setting, distribution-free risk bounds are impossible, and the best one could hope for is a fully empirical data-dependent bound. We derive precisely such bounds, and demonstrate that these are, in a well-defined sense, the best possible. Our main discovery is that the half-norm of the empirical distribution provides tight upper and lower estimates on the empirical risk. Furthermore, this quantity decays at a nearly optimal rate as a function of the true distribution. The optimality follows from a minimax result, of possible independent interest. Additional structural results are provided, including an exact Rademacher complexity calculation and apparently a first connection between the total variation risk and the missing mass.
Reading In-Between the Lines: An Analysis of Dissenter

Efforts by content creators and social networks to enforce legal and policy-based norms, e.g. blocking hate speech and users, has driven the rise of unrestricted communication platforms. One such recent effort is Dissenter, a browser and web application that provides a conversational overlay for any web page. These conversations hide in plain sight -- users of Dissenter can see and participate in this conversation, whereas visitors using other browsers are oblivious to their existence. Further, the website and content owners have no power over the conversation as it resides in an overlay outside their control.In this work, we obtain a history of Dissenter comments, users, and the websites being discussed, from the initial release of Dissenter in Feb. 2019 through Apr. 2020 (14 months). Our corpus consists of approximately 1.68M comments made by 101k users commenting on 588k distinct URLs. We first analyze macro characteristics of the network, including the user-base, comment distribution, and growth. We then use toxicity dictionaries, Perspective API, and a Natural Language Processing model to understand the nature of the comments and measure the propensity of particular websites and content to elicit hateful and offensive Dissenter comments. Using curated rankings of media bias, we examine the conditional probability of hateful comments given left and right-leaning content. Finally, we study Dissenter as a social network, and identify a core group of users with high comment toxicity.
Bayesian Robust Optimization for Imitation Learning

One of the main challenges in imitation learning is determining what action an agent should take when outside the state distribution of the demonstrations. Inverse reinforcement learning (IRL) can enable generalization to new states by learning a parameterized reward function, but these approaches still face uncertainty over the true reward function and corresponding optimal policy. Existing safe imitation learning approaches based on IRL deal with this uncertainty using a maxmin framework that optimizes a policy under the assumption of an adversarial reward function, whereas risk-neutral IRL approaches either optimize a policy for the mean or MAP reward function. While completely ignoring risk can lead to overly aggressive and unsafe policies, optimizing in a fully adversarial sense is also problematic as it can lead to overly conservative policies that perform poorly in practice. To provide a bridge between these two extremes, we propose Bayesian Robust Optimization for Imitation Learning (BROIL). BROIL leverages Bayesian reward function inference and a user specific risk tolerance to efficiently optimize a robust policy that balances expected return and conditional value at risk. Our empirical results show that BROIL provides a natural way to interpolate between return-maximizing and risk-minimizing behaviors and outperforms existing risk-sensitive and risk-neutral inverse reinforcement learning algorithms.
Model-Based Systems Engineering methods to support the reuse of knowledge within the development of validation environments

Systemic knowledge is an important factor within the development of successful products. Thus, validation is the central activity of product engineering [1]. Only through the analysis of a product and its subsystems in the context of its relevant environment and usage, properties, feasibility and customer benefits can be determined. This constitutes the need for validation environments (VE).
Bayes Consistency vs. H -Consistency: The Interplay between Surrogate Loss Functions and the Scoring Function Class.

A fundamental question in multiclass classification concerns understanding the consistency properties of surrogate risk minimization algorithms, which minimize a (often convex) surrogate to the multiclass 0-1 loss. In particular, the framework of calibrated surrogates has played an important role in analyzing Bayes consistency of such algorithms, i.e. in studying convergence to a Bayes optimal classifier (Zhang, 2004; Tewari and Bartlett, 2007). However, follow-up work has suggested this framework can be of limited value when studying H-consistency; in particular, concerns have been raised that even when the data comes from an underlying linear model, minimizing certain convex calibrated surrogates over linear scoring functions fails to recover the true model (Long and Servedio, 2013). In this paper, we investigate this apparent conundrum. We find that while some calibrated surrogates can indeed fail to provide H -consistency when minimized over a naturallooking but naively chosen scoring function class F , the situation can potentially be remedied by minimizing them over a more carefully chosen class of scoring functions F . In particular, for the popular one-vs-all hinge and logistic surrogates, both of which are calibrated (and therefore provide Bayes consistency) under realizable models, but were previously shown to pose problems for realizable H -consistency, we derive a form of scoring function class F that enables Hconsistency. When H is the class of linear models, the class F consists of certain piecewise linear scoring functions that are characterized by the same number of parameters as in the linear case, and minimization over which can be performed using an adaptation of the min-pooling idea from neural network training. Our experiments confirm that the one-vs-all surrogates, when trained over this class of nonlinear scoring functions F , yield better linear multiclass classifiers than when trained over standard linear scoring functions.
Smoothed Analysis of Online and Differentially Private Learning

Practical and pervasive needs for robustness and privacy in algorithms have inspired the design of online adversarial and differentially private learning algorithms. The primary quantity that characterizes learnability in these settings is the Littlestone dimension of the class of hypotheses [Ben-David et al., 2009, Alon et al., 2019]. This characterization is often interpreted as an impossibility result because classes such as linear thresholds and neural networks have infinite Littlestone dimension. In this paper, we apply the framework of smoothed analysis [Spielman and Teng, 2004], in which adversarially chosen inputs are perturbed slightly by nature. We show that fundamentally stronger regret and error guarantees are possible with smoothed adversaries than with worst-case adversaries. In particular, we obtain regret and privacy error bounds that depend only on the VC dimension and the bracketing number of a hypothesis class, and on the magnitudes of the perturbations.
Research on Text Classification Algorithm Based on Machine Learning

Abstract In the era of big data, the data we analyze is no longer limited to numbers, while even more data analysis is based on text. Facing massive and complex data, traditional analysis methods cannot effectively process text data. Manual processing of text data not only consumes a lot of time and manpower, but also it is difficult to ensure the fairness and objectivity of the assessment. The development of machine learning provides us with an effective method of text mining. This article will introduce three classic algorithms in machine learning for elaboration, and apply them into a specific experiment to show the wisdom of machine learning in the field of text classification.
Semi-Supervised Learning for Video Captioning

Deep neural networks have made great success on video captioning in supervised learning setting. However, annotating videos with descriptions is very expensive and time-consuming. If the video captioning algorithm can benefit from a large number of unlabeled videos, the cost of annotation can be reduced. In the proposed study, we make the first attempt to train the video captioning model on labeled data and unlabeled data jointly, in a semi-supervised learning manner. For labeled data, we train them with the traditional cross-entropy loss. For unlabeled data, we leverage a self-critical policy gradient method with the difference between the scores obtained by Monte-Carlo sampling and greedy decoding as the reward function, while the scores are the negative K-L divergence between output distributions of original video data and augmented video data. The final loss is the weighted sum of losses obtained by labeled data and unlabeled data. Experiments conducted on VATEX, MSR-VTT and MSVD dataset demonstrate that the introduction of unlabeled data can improve the performance of the video captioning model. The proposed semi-supervised learning algorithm also outperforms several state-of-the-art semi-supervised learning approaches.
Digital Duty to Deal, Data Portability, and Interoperability

In this chapter, we discuss the development of the duty to deal doctrine in antitrust law, its application to the digital economy, and proposals for specific duties to deal, such as data portability and interoperability.Part I outlines the development of the duty to deal doctrine in antitrust law. The development of the doctrine in the United States will be compared to that in the European Union. Popular economic justifications for the doctrine and key cases will be explored. Part II then situates this doctrine within the digital economy, focusing on the importance of getting the contours of the doctrine right in that economy. As we shall see, the law and economics of the duty to deal caution against its application to dynamic, digital markets. This will be illustrated by looking at cases where it has been applied. Part III focuses on two specific categories of duties to deal: data portability and interoperability.