text: 419,5,1213889108751241217,2800204849,Andrew Gordon Wilson,"“What I cannot create, I do not understand”. We develop normalizing flows for end-to-end fully generative semi-supervised classification (with code)! Our new paper, with @Pavel_Izmailov, @polkirichenko, @m_finzi: <LINK> (1/8) <LINK> The discriminative approach to classification models the probability of a class label given an input p(y|x) directly. The generative, approach, by contrast, models the class conditional density p(x|y), then finds p(y|x) with Bayes rule. 2/8 Nearly all classifiers are discriminative. Even approaches that use a generator typically involve a discriminator in the pipeline. For example, sometimes one learns a generator on unlabelled data, then recycles the representation as part of a discriminative classifier. 3/8 Generative models are compelling because we are trying to create an object of interest. The challenge in generative modelling is that standard approaches to density estimation are poor descriptions of high-dimensional natural signals. 4/8 For example, a Gaussian mixture directly over images, while highly flexible for density estimation, would specify similarities between images as related to Euclidean distances between pixel intensities, which is a poor inductive bias for translation and other invariances. 5/8 Normalizing flows provide a pleasingly simple approach to generative modelling. By transforming a latent distribution through an invertible network, we have both an exact likelihood for the data, and useful inductive biases from a convolutional neural network. 6/8 FlowGMM models the latent space as a Gaussian mixture, where each mixture component is associated with a class label. This approach specifies an exact joint likelihood over both labelled and unlabelled data for end-to-end training. 7/8 FlowGMM has broad applicability. We consider text, tabular, and image data. FlowGMM can also discover interpretable structure, provide real-time optimization-free feature visualizations, and specify well calibrated predictive distributions. 8/8 @matvil @Pavel_Izmailov @polkirichenko @m_finzi Normalizing flows require an invertible NN, which imposes some constraints on speed and architectural design. However, invertible NNs are rapidly improving, such that approaches based on normalizing flows are becoming increasingly compelling.",http://arxiv.org/abs/1912.13025,"Normalizing flows transform a latent distribution through an invertible neural network for a flexible and pleasingly simple approach to generative modelling, while preserving an exact likelihood. We propose FlowGMM, an end-to-end approach to generative semi supervised learning with normalizing flows, using a latent Gaussian mixture model. FlowGMM is distinct in its simplicity, unified treatment of labelled and unlabelled data with an exact likelihood, interpretability, and broad applicability beyond image data. We show promising results on a wide range of applications, including AG-News and Yahoo Answers text data, tabular data, and semi-supervised image classification. We also show that FlowGMM can discover interpretable structure, provide real-time optimization-free feature visualizations, and specify well calibrated predictive distributions. ",Semi-Supervised Learning with Normalizing Flows,9,"['“What I cannot create, I do not understand”. We develop normalizing flows for end-to-end fully generative semi-supervised classification (with code)! Our new paper, with @Pavel_Izmailov, @polkirichenko, @m_finzi: <LINK> (1/8) <LINK>', 'The discriminative approach to classification models the probability of a class label given an input p(y|x) directly. The generative, approach, by contrast, models the class conditional density p(x|y), then finds p(y|x) with Bayes rule. 2/8', 'Nearly all classifiers are discriminative. Even approaches that use a generator typically involve a discriminator in the pipeline. For example, sometimes one learns a generator on unlabelled data, then recycles the representation as part of a discriminative classifier. 3/8', 'Generative models are compelling because we are trying to create an object of interest. The challenge in generative modelling is that standard approaches to density estimation are poor descriptions of high-dimensional natural signals. 4/8', 'For example, a Gaussian mixture directly over images, while highly flexible for density estimation, would specify similarities between images as related to Euclidean distances between pixel intensities, which is a poor inductive bias for translation and other invariances. 5/8', 'Normalizing flows provide a pleasingly simple approach to generative modelling. By transforming a latent distribution through an invertible network, we have both an exact likelihood for the data, and useful inductive biases from a convolutional neural network. 6/8', 'FlowGMM models the latent space as a Gaussian mixture, where each mixture component is associated with a class label. This approach specifies an exact joint likelihood over both labelled and unlabelled data for end-to-end training. 7/8', 'FlowGMM has broad applicability. We consider text, tabular, and image data. FlowGMM can also discover interpretable structure, provide real-time optimization-free feature visualizations, and specify well calibrated predictive distributions. 8/8', '@matvil @Pavel_Izmailov @polkirichenko @m_finzi Normalizing flows require an invertible NN, which imposes some constraints on speed and architectural design. However, invertible NNs are rapidly improving, such that approaches based on normalizing flows are becoming increasingly compelling.']",19,12,2300