huashen218/convxai-cia-dataset huggingface dataset:
Stochastic AUC Maximization with Deep Neural Networks
Keywords: Stochastic AUC Maximization, Deep Neural Networks
TL;DR: The paper designs two algorithms for the stochastic AUC maximization problem with state-of-the-art complexities when using deep neural network as predictive model, which are also verified by empirical studies.

Stochastic AUC maximization has garnered an increasing interest due to better fit to imbalanced data classification. However, existing works are limited to stochastic AUC maximization with a linear predictive model, which restricts its predictive power when dealing with extremely complex data. In this paper, we consider stochastic AUC maximization problem with a deep neural network as the predictive model. Building on the saddle point reformulation of a surrogated loss of AUC, the problem can be cast into a {\it non-convex concave} min-max problem. The main contribution made in this paper is to make stochastic AUC maximization more practical for deep neural networks and big data with theoretical insights as well. In particular, we propose to explore Polyak-\L{}ojasiewicz (PL) condition that has been proved and observed in deep learning, which enables us to develop new stochastic algorithms with even faster convergence rate and more practical step size scheme. An AdaGrad-style algorithm is also analyzed under the PL condition with adaptive convergence rate. Our experimental results demonstrate the effectiveness of the proposed algorithms.
Difference-Seeking Generative Adversarial Network--Unseen Sample Generation
Keywords: generative adversarial network, semi-supervised learning, novelty detection
TL;DR: We proposed a novel GAN framework to generate unseen data.

Unseen data, which are not samples from the distribution of training data and are difficult to collect, have exhibited importance in numerous applications, ({\em e.g.,} novelty detection, semi-supervised learning, and adversarial training).  In this paper, we introduce a general framework called  \textbf{d}ifference-\textbf{s}eeking \textbf{g}enerative \textbf{a}dversarial \textbf{n}etwork (DSGAN), to generate various types of unseen data. Its novelty is the consideration of the probability density of the unseen data distribution as the difference between two distributions pd¯ and pd whose samples are relatively easy to collect.
        
        The DSGAN can learn the target distribution, pt, (or the unseen data distribution)  from only the samples from the two distributions, pd and pd¯. In our scenario, pd is the distribution of the seen data, and pd¯ can be obtained from pd via simple operations, so that  we only need the samples of pd during the training. 
        Two key applications, semi-supervised learning and novelty detection, are taken as case studies to illustrate that the DSGAN enables the production of various unseen data. We also provide theoretical analyses about the convergence of the DSGAN.
FasterSeg: Searching for Faster Real-time Semantic Segmentation
Keywords: neural architecture search, real-time, segmentation
TL;DR: We present a real-time segmentation model automatically discovered by a multi-scale NAS framework, achieving 30% faster than state-of-the-art models.

We present FasterSeg, an automatically designed semantic segmentation network with not only state-of-the-art performance but also faster speed than current methods. Utilizing neural architecture search (NAS), FasterSeg is discovered from a novel and broader search space integrating multi-resolution branches, that has been recently found to be vital in manually designed segmentation models. To better calibrate the balance between the goals of high accuracy and low latency, we propose a decoupled and fine-grained latency regularization, that effectively overcomes our observed phenomenons that the searched networks are prone to "collapsing" to low-latency yet poor-accuracy models. Moreover, we seamlessly extend FasterSeg to a new collaborative search (co-searching) framework, simultaneously searching for a teacher and a student network in the same single run. The teacher-student distillation further boosts the student model’s accuracy. Experiments on popular segmentation benchmarks demonstrate the competency of FasterSeg. For example, FasterSeg can run over 30% faster than the closest manually designed competitor on Cityscapes, while maintaining comparable accuracy.
LEARNING EXECUTION THROUGH NEURAL CODE FUSION
Keywords: code understanding, graph neural networks, learning program execution, execution traces, program performance

As the performance of computer systems stagnates due to the end of Moore’s Law,
        there is a need for new models that can understand and optimize the execution
        of general purpose code. While there is a growing body of work on using Graph
        Neural Networks (GNNs) to learn static representations of source code, these
        representations do not understand how code executes at runtime. In this work, we
        propose a new approach using GNNs to learn fused representations of general
        source code and its execution. Our approach defines a multi-task GNN over
        low-level representations of source code and program state (i.e., assembly code
        and dynamic memory states), converting complex source code constructs and data
        structures into a simpler, more uniform format. We show that this leads to improved
        performance over similar methods that do not use execution and it opens the door
        to applying GNN models to new tasks that would not be feasible from static code
        alone. As an illustration of this, we apply the new model to challenging dynamic
        tasks (branch prediction and prefetching) from the SPEC CPU benchmark suite,
        outperforming the state-of-the-art by 26% and 45% respectively. Moreover, we
        use the learned fused graph embeddings to demonstrate transfer learning with high
        performance on an indirectly related algorithm classification task.
Editable Neural Networks
Keywords: editing, editable, meta-learning, maml
TL;DR: Training neural networks so you can efficiently patch them later.

These days deep neural networks are ubiquitously used in a wide range of tasks, from image classification and machine translation to face identification and self-driving cars. In many applications, a single model error can lead to devastating financial, reputational and even life-threatening consequences. Therefore, it is crucially important to correct model mistakes quickly as they appear. In this work, we investigate the problem of neural network editing - how one can efficiently patch a mistake of the model on a particular sample, without influencing the model behavior on other samples. Namely, we propose Editable Training, a model-agnostic training technique that encourages fast editing of the trained model. We empirically demonstrate the effectiveness of this method on large-scale image classification and machine translation tasks.
Can gradient clipping mitigate label noise?
TL;DR: Gradient clipping doesn't endow robustness to label noise, but a simple loss-based variant does.

Gradient clipping is a widely-used technique in the training of deep networks, and is generally motivated from an optimisation lens: informally, it controls the dynamics of iterates, thus enhancing the rate of convergence to a local minimum. This intuition has been made precise in a line of recent works, which show that suitable clipping  can yield significantly faster convergence than vanilla gradient descent. In this paper, we propose a new lens for studying gradient clipping, namely, robustness: informally, one expects clipping to provide robustness to noise, since one does not overly trust any single sample. Surprisingly, we prove that  for the common problem of label noise in classification, standard gradient clipping does not in general provide robustness. On the other hand, we show that  a simple variant of gradient clipping is provably robust, and corresponds to suitably modifying the underlying loss function. This yields a simple, noise-robust alternative to the standard cross-entropy loss which performs well empirically.
Pruned Graph Scattering Transforms
Keywords: Graph scattering transforms, pruning, graph convolutional networks, stability, deep learning

Graph convolutional networks (GCNs) have achieved remarkable performance in a variety of network science learning tasks. However, theoretical analysis of such approaches is still at its infancy. Graph scattering transforms (GSTs) are non-trainable deep GCN models that are amenable to generalization and stability analyses. The present work addresses some limitations of GSTs by introducing a novel so-termed pruned (p)GST approach. The resultant pruning algorithm is guided by a graph-spectrum-inspired criterion, and retains informative scattering features on-the-fly while bypassing the exponential complexity associated with GSTs. It is further established that pGSTs are stable to perturbations of the input graph signals with bounded energy. Experiments showcase that i) pGST performs comparably to the baseline GST that uses all scattering features, while achieving significant computational savings; ii) pGST achieves comparable performance to state-of-the-art GCNs; and iii) Graph data from various domains lead to different scattering patterns, suggesting domain-adaptive pGST network architectures.
GLAD: Learning Sparse Graph Recovery
Keywords: Meta learning, automated algorithm design, learning structure recovery, Gaussian graphical models
TL;DR: A data-driven learning algorithm based on unrolling the Alternating Minimization optimization for sparse graph recovery.

Recovering sparse conditional independence graphs from data is a fundamental problem in machine learning with wide applications. A popular formulation of the problem is an ℓ1 regularized maximum likelihood estimation. Many convex optimization algorithms have been designed to solve this formulation to recover the graph structure. Recently, there is a surge of interest to learn algorithms directly based on data, and in this case, learn to map empirical covariance to the sparse precision matrix. However, it is a challenging task in this case, since the symmetric positive definiteness (SPD) and sparsity of the matrix are not easy to enforce in learned algorithms, and a direct mapping from data to precision matrix may contain many parameters. We propose a deep learning architecture, GLAD, which uses an Alternating Minimization (AM) algorithm as our model inductive bias, and learns the model parameters via supervised learning. We show that GLAD learns a very compact and effective model for recovering sparse graphs from data.
VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generation
Keywords: Video generation, flow-based generative models, stochastic video prediction
TL;DR: We demonstrate that flow-based generative models offer a viable and competitive approach to generative modeling of video.

Generative models that can model and predict sequences of future events can, in principle, learn to capture complex real-world phenomena, such as physical interactions. However, a central challenge in video prediction is that the future is highly uncertain: a sequence of past observations of events can imply many possible futures. Although a number of recent works have studied probabilistic models that can represent uncertain futures, such models are either extremely expensive computationally as in the case of pixel-level autoregressive models, or do not directly optimize the likelihood of the data. To our knowledge, our work is the first to propose multi-frame video prediction with normalizing flows, which allows for direct optimization of the data likelihood, and produces high-quality stochastic predictions. We describe an approach for modeling the latent space dynamics, and demonstrate that flow-based generative models offer a viable and competitive approach to generative modeling of video.
Adversarial Policies: Attacking Deep Reinforcement Learning
Keywords: deep RL, adversarial examples, security, multi-agent
TL;DR: Deep RL policies can be attacked by other agents taking actions so as to create natural observations that are adversarial.

Deep reinforcement learning (RL) policies are known to be vulnerable to adversarial perturbations to their observations, similar to adversarial examples for classifiers. However, an attacker is not usually able to directly modify another agent's observations. This might lead one to wonder: is it possible to attack an RL agent simply by choosing an adversarial policy acting in a multi-agent environment so as to create natural observations that are adversarial? We demonstrate the existence of adversarial policies in zero-sum games between simulated humanoid robots with proprioceptive observations, against state-of-the-art victims trained via self-play to be robust to opponents. The adversarial policies reliably win against the victims but generate seemingly random and uncoordinated behavior. We find that these policies are more successful in high-dimensional environments, and induce substantially different activations in the victim policy network than when the victim plays against a normal opponent. Videos are available at https://adversarialpolicies.github.io/.
Escaping Saddle Points Faster with Stochastic Momentum
Keywords: SGD, momentum, escaping saddle point
TL;DR: Higher momentum parameter β helps for escaping saddle points faster

Stochastic gradient descent (SGD) with stochastic momentum is popular in nonconvex stochastic optimization and particularly for the training of deep neural networks. In standard SGD, parameters are updated by improving along the path of the gradient at the current iterate on a batch of examples, where the addition of a ``momentum'' term biases the update in the direction of the previous change in parameters. In non-stochastic convex optimization one can show that a momentum adjustment provably reduces convergence time in many settings, yet such results have been elusive in the stochastic and non-convex settings. At the same time, a widely-observed empirical phenomenon is that in training deep networks stochastic momentum appears to significantly improve convergence time, variants of it have flourished in the development of other popular update methods, e.g. ADAM, AMSGrad, etc. Yet theoretical justification for the use of stochastic momentum has remained a significant open question. In this paper we propose an answer: stochastic momentum improves deep network training because it modifies SGD to escape saddle points faster and, consequently, to more quickly find a second order stationary point. Our theoretical results also shed light on the related question of how to choose the ideal momentum parameter--our analysis suggests that β∈[0,1) should be large (close to 1), which comports with empirical findings. We also provide experimental findings that further validate these conclusions.
Few-shot Text Classification with Distributional Signatures
Keywords: text classification, meta learning, few shot learning
TL;DR: Meta-learning methods used for vision, directly applied to NLP, perform worse than nearest neighbors on new classes; we can do better with distributional signatures.

In this paper, we explore meta-learning for few-shot text classification. Meta-learning has shown strong performance in computer vision, where low-level patterns are transferable across learning tasks. However, directly applying this approach to text is challenging--lexical features highly informative for one task may be insignificant for another. Thus, rather than learning solely from words, our model also leverages their distributional signatures, which encode pertinent word occurrence patterns. Our model is trained within a meta-learning framework to map these signatures into attention scores, which are then used to weight the lexical representations of words. We demonstrate that our model consistently outperforms prototypical networks learned on lexical knowledge (Snell et al., 2017) in both few-shot text classification and relation classification by a significant margin across six benchmark datasets (20.0% on average in 1-shot classification).
Geometric Insights into the Convergence of Nonlinear TD Learning
Keywords: TD, nonlinear, convergence, value estimation, reinforcement learning

While there are convergence guarantees for temporal difference (TD) learning when using linear function approximators, the situation for nonlinear models is far less understood, and divergent examples are known. Here we take a first step towards extending theoretical convergence guarantees to TD learning with nonlinear function approximation. More precisely, we consider the expected learning dynamics of the TD(0) algorithm for value estimation. As the step-size converges to zero, these dynamics are defined by a nonlinear ODE which depends on the geometry of the space of function approximators, the structure of the underlying Markov chain, and their interaction. We find a set of function approximators that includes ReLU networks and has geometry amenable to TD learning regardless of environment, so that the solution performs about as well as linear TD in the worst case. Then, we show how environments that are more reversible induce dynamics that are better for TD learning and prove global convergence to the true value function for well-conditioned function approximators. Finally, we generalize a divergent counterexample to a family of divergent problems to demonstrate how the interaction between approximator and environment can go wrong and to motivate the assumptions needed to prove convergence.
Learning Self-Correctable Policies and Value Functions from Demonstrations with Negative Sampling
Keywords: imitation learning, model-based imitation learning, model-based RL, behavior cloning, covariate shift
TL;DR: We introduce a notion of conservatively-extrapolated value functions, which provably lead to policies that can self-correct to stay close to the demonstration states, and learn them with a novel negative sampling technique.

Imitation learning, followed by reinforcement learning algorithms, is a promising paradigm to solve complex control tasks sample-efficiently. However, learning from demonstrations often suffers from the covariate shift problem, which results
        in cascading errors of the learned policy. We introduce a notion of conservatively extrapolated value functions, which provably lead to policies with self-correction. We design an algorithm Value Iteration with Negative Sampling (VINS) that practically learns such value functions with conservative extrapolation. We show that VINS can correct mistakes of the behavioral cloning policy on simulated robotics benchmark tasks. We also propose the algorithm of using VINS to initialize a reinforcement learning algorithm, which is shown to outperform prior works in sample efficiency.
Exploring Model-based Planning with Policy Networks
Keywords: reinforcement learning, model-based reinforcement learning, planning
TL;DR: how to achieve state-of-the-art performance by combining policy network in model-based planning

Model-based reinforcement learning (MBRL) with model-predictive control or
        online planning has shown great potential for locomotion control tasks in both
        sample efficiency and asymptotic performance. Despite the successes, the existing
        planning methods search from candidate sequences randomly generated in the
        action space, which is inefficient in complex high-dimensional environments. In
        this paper, we propose a novel MBRL algorithm, model-based policy planning
        (POPLIN), that combines policy networks with online planning. More specifically,
        we formulate action planning at each time-step as an optimization problem using
        neural networks. We experiment with both optimization w.r.t. the action sequences
        initialized from the policy network, and also online optimization directly w.r.t. the
        parameters of the policy network. We show that POPLIN obtains state-of-the-art
        performance in the MuJoCo benchmarking environments, being about 3x more
        sample efficient than the state-of-the-art algorithms, such as PETS, TD3 and SAC.
        To explain the effectiveness of our algorithm, we show that the optimization surface
        in parameter space is smoother than in action space. Further more, we found the
        distilled policy network can be effectively applied without the expansive model
        predictive control during test time for some environments such as Cheetah. Code
        is released.
On Identifiability in Transformers
Keywords: Self-attention, interpretability, identifiability, BERT, Transformer, NLP, explanation, gradient attribution
TL;DR: We investigate the identifiability and interpretability of attention distributions and tokens within contextual embeddings in the self-attention based BERT model.

In this paper we delve deep in the Transformer architecture by investigating two of its core components: self-attention and contextual embeddings. In particular, we study the identifiability of attention weights and token embeddings, and the aggregation of context into hidden tokens. We show that, for sequences longer than the attention head dimension, attention weights are not identifiable. We propose effective attention as a complementary tool for improving explanatory interpretations based on attention. Furthermore, we show that input tokens retain to a large degree their identity across the model. We also find evidence suggesting that identity information is mainly encoded in the angle of the embeddings and gradually decreases with depth. Finally, we demonstrate strong mixing of input information in the generation of contextual embeddings by means of a novel quantification method based on gradient attribution. Overall, we show that self-attention distributions are not directly interpretable and present tools to better understand and further investigate Transformer models.
Automated curriculum generation through setter-solver interactions
Keywords: Deep Reinforcement Learning, Automatic Curriculum
TL;DR: We investigate automatic curriculum generation and identify a number of losses useful to learn to generate a curriculum of tasks.

Reinforcement learning algorithms use correlations between policies and rewards to improve agent performance.   But in dynamic or sparsely rewarding environments these correlations are often too small,  or rewarding events are too infrequent to make learning feasible. Human education instead relies on curricula –the breakdown of tasks into simpler, static challenges with dense rewards– to build up to complex behaviors.  While curricula are also useful for artificial agents, hand-crafting them is time consuming.  This has lead researchers to explore automatic curriculum generation. Here we explore automatic curriculum generation in rich,dynamic environments.  Using a setter-solver paradigm we show the importance of considering goal validity, goal feasibility, and goal coverage to construct useful curricula.  We demonstrate the success of our approach in rich but sparsely rewarding 2D and 3D environments, where an agent is tasked to achieve a single goal selected from a set of possible goals that varies between episodes, and identify challenges for future work.  Finally, we demonstrate the value of a novel technique that guides agents towards a desired goal distribution. Altogether, these results represent a substantial step towards applying automatic task curricula to learn complex,  otherwise unlearnable goals,  and to our knowledge are the first to demonstrate automated curriculum generation for goal-conditioned agents in environments where the possible goals vary between episodes.