In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.

[Q]: Stochastic gradient descent (SGD), which dates back to the 1950s, is one of the most popular and effective approaches for performing stochastic optimization. Research on SGD resurged recently in machine learning for optimizing convex loss functions and training nonconvex deep neural networks. The theory assumes that one can easily compute an unbiased gradient estimator, which is usually the case due to the sample average nature of empirical risk minimization. There exist, however, many scenarios (e.g., graphs) where an unbiased estimator may be as expensive to compute as the full gradient because training examples are interconnected. Recently, Chen et al. (2018) proposed using a consistent gradient estimator as an economic alternative. Encouraged by empirical success, we show, in a general setting, that consistent estimators result in the same convergence behavior as do unbiased ones. Our analysis covers strongly convex, convex, and nonconvex objectives. We verify the results with illustrative experiments on synthetic and real-world data. This work opens several new research directions, including the development of more efficient SGD updates with consistent estimators and the design of efficient training algorithms for large-scale graphs.

[A]: Convergence theory for biased (but consistent) gradient estimators in stochastic optimization and application to graph convolutional networks

[Q]: In deep learning, performance is strongly affected by the choice of architecture
and hyperparameters. While there has been extensive work on automatic hyperpa-
rameter optimization for simple spaces, complex spaces such as the space of deep
architectures remain largely unexplored. As a result, the choice of architecture is
done manually by the human expert through a slow trial and error process guided
mainly by intuition. In this paper we describe a framework for automatically
designing and training deep models. We propose an extensible and modular lan-
guage that allows the human expert to compactly represent complex search spaces
over architectures and their hyperparameters. The resulting search spaces are tree-
structured and therefore easy to traverse. Models can be automatically compiled to
computational graphs once values for all hyperparameters have been chosen. We
can leverage the structure of the search space to introduce different model search
algorithms, such as random search, Monte Carlo tree search (MCTS), and sequen-
tial model-based optimization (SMBO). We present experiments comparing the
different algorithms on CIFAR-10 and show that MCTS and SMBO outperform
random search. We also present experiments on MNIST, showing that the same
search space achieves near state-of-the-art performance with a few samples. These
experiments show that our framework can be used effectively for model discov-
ery, as it is possible to describe expressive search spaces and discover competitive
models without much effort from the human expert. Code for our framework and
experiments has been made publicly available
[A]: We describe a modular and composable language for describing expressive search spaces over architectures and simple model search algorithms applied to these search spaces.

[Q]: The information bottleneck method provides an information-theoretic method for representation learning, by training an encoder to retain all information which is relevant for predicting the label, while minimizing the amount of other, superfluous information in the representation. The original formulation, however, requires labeled data in order to identify which information is superfluous. In this work, we extend this ability to the multi-view unsupervised setting, in which two views of the same underlying entity are provided but the label is unknown. This enables us to identify superfluous information as that which is not shared by both views. A theoretical analysis leads to the definition of a new multi-view model that produces state-of-the-art results on the Sketchy dataset and on label-limited versions of the MIR-Flickr dataset. We also extend our theory to the single-view setting by taking advantage of standard data augmentation techniques, empirically showing better generalization capabilities when compared to traditional unsupervised approaches for representation learning.
[A]:
We extend the information bottleneck method to the unsupervised multiview setting and show state of the art results on standard datasets