In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.

Q: Deep ensembles have been empirically shown to be a promising approach for improving accuracy, uncertainty  and out-of-distribution robustness of deep learning models. While deep ensembles were theoretically motivated by the bootstrap, non-bootstrap ensembles trained with just random initialization also perform well in practice, which suggests that there could be other explanations for why deep ensembles work well. Bayesian neural networks, which learn distributions over the parameters of the network, are theoretically well-motivated by Bayesian principles, but do not perform as well as deep ensembles in practice, particularly under dataset shift. One possible explanation for this gap between theory and practice is that popular scalable approximate Bayesian methods tend to focus on a single mode, whereas deep ensembles tend to explore diverse modes in function space. We investigate this hypothesis by building on recent work on understanding the loss landscape of neural networks and adding our own exploration to measure the similarity of functions in the space of predictions. Our results show that random initializations explore entirely different modes, while functions along an optimization trajectory or sampled from the subspace thereof cluster within a single mode predictions-wise, while often deviating significantly in the weight space. We demonstrate that while low-loss connectors between modes exist, they are not connected in the space of predictions. Developing the concept of the diversity--accuracy plane, we show that the decorrelation power of random initializations is unmatched by popular subspace sampling methods.

A: We study deep ensembles through the lens of loss landscape and the space of predictions, demonstrating that the decorrelation power of random initializations is unmatched by subspace sampling that only explores a single mode.
****
Q: Deep neural networks have been shown to perform well in many classical machine learning problems, especially in image classification tasks. However, researchers have found that neural networks can be easily fooled, and they are surprisingly sensitive to small perturbations imperceptible to humans.   Carefully crafted input images (adversarial examples) can force a well-trained neural network to provide arbitrary outputs.   Including adversarial examples during training is a popular defense mechanism against adversarial attacks. In this paper we propose a new defensive mechanism under the generative adversarial network~(GAN) framework. We model the adversarial noise using a generative network, trained jointly with a classification discriminative network as a minimax game. We show empirically that our adversarial network approach works well against black box attacks, with performance on par with state-of-art methods such as ensemble adversarial training and adversarial training with projected gradient descent.


A: Jointly train an adversarial noise generating network with a classification network to provide better robustness to adversarial attacks.
****
Q: Determining the number of latent dimensions is a ubiquitous problem in machine
 learning. In this study, we introduce a novel method that relies on SVD to discover
 the number of latent dimensions. The general principle behind the method is to
 compare the curve of singular values of the SVD decomposition of a data set with
 the randomized data set curve. The inferred number of latent dimensions corresponds
 to the crossing point of the two curves. To evaluate our methodology, we
 compare it with competing methods such as Kaisers eigenvalue-greater-than-one
 rule (K1), Parallel Analysis (PA), Velicers MAP test (Minimum Average Partial).
 We also compare our method with the Silhouette Width (SW) technique which is
 used in different clustering methods to determine the optimal number of clusters.
 The result on synthetic data shows that the Parallel Analysis and our method have
 similar results and more accurate than the other methods, and that our methods is
 slightly better result than the Parallel Analysis method for the sparse data sets.

A:
In this study, we introduce a novel method that relies on SVD to discover the number of latent dimensions.
****