Before revision:  We evaluate our approach on a diverse set of tasks , including image classification, bidirectional image-sentence retrieval, and phrase grounding, creating high performing models even when using as little as 1\% of the parameters . We also apply SSNs to knowledge distillation, where we obtain state-of-the-art results when combined with traditional distillation methods .
After revision:  We evaluate our approach on a diverse set of tasks , including image classification, bidirectional image-sentence retrieval, and phrase grounding, creating high performing models even when using as little as 1\% of the parameters .