  Nested networks or slimmable networks are neural networks whose architectures
can be adjusted instantly during testing time, e.g., based on computational
constraints. Recent studies have focused on a "nested dropout" layer, which is
able to order the nodes of a layer by importance during training, thus
generating a nested set of sub-networks that are optimal for different
configurations of resources. However, the dropout rate is fixed as a
hyper-parameter over different layers during the whole training process.
Therefore, when nodes are removed, the performance decays in a human-specified
trajectory rather than in a trajectory learned from data. Another drawback is
the generated sub-networks are deterministic networks without well-calibrated
uncertainty. To address these two problems, we develop a Bayesian approach to
nested neural networks. We propose a variational ordering unit that draws
samples for nested dropout at a low cost, from a proposed Downhill
distribution, which provides useful gradients to the parameters of nested
dropout. Based on this approach, we design a Bayesian nested neural network
that learns the order knowledge of the node distributions. In experiments, we
show that the proposed approach outperforms the nested network in terms of
accuracy, calibration, and out-of-domain detection in classification tasks. It
also outperforms the related approach on uncertainty-critical tasks in computer
vision.
