B. Mirzasoleiman. Big Data Summarization Using Submodular Functions. PhD thesis, ETH Zurich, 2017.
B. Mirzasoleiman, A. Badanidiyuru, A. Karbasi, J. Vondrák, and A. Krause. Lazier than lazy greedy. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015.
W. Park, D. Kim, Y. Lu, and M. Cho. Relational knowledge distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3967-3976, 2019.
A. Parulekar, L. Collins, K. Shanmugam, A. Mokhtari, and S. Shakkottai. Infonce loss provably learns cluster-preserving representations. arXiv preprint arXiv:2302.07920, 2023.
H. Phan. huyvnphan/pytorch_cifar10, jan 2021.
M. Phuong and C. Lampert. Towards understanding knowledge distillation. In International Conference on Machine Learning, pages 5142-5151. PMLR, 2019.
Q. Qian, H. Li, and J. Hu. Improved knowledge distillation via full kernel matrix transfer. In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), pages 612-620. SIAM, 2022.
A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550, 2014.
J. Schreiber, J. Bilmes, and W. S. Noble. apricot: Submodular selection for data summarization in python. Journal of Machine Learning Research, 21(161):1-6, 2020.
R. Shen, S. Bubeck, and S. Gunasekar. Data augmentation as feature manipulation. In International Conference on Machine Learning, pages 19773-19808. PMLR, 2022.
P. Y. Simard, Y. A. LeCun, J. S. Denker, and B. Victorri. Transformation invariance in pattern recognition—tangent distance and tangent propagation. In Neural networks: tricks of the trade, pages 239–274. Springer, 2002.
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
K. Sohn, D. Berthelot, N. Carlini, Z. Zhang, H. Zhang, C. A. Raffel, E. D. Cubuk, A. Kurakin, and C.-L. Li. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in neural information processing systems, 33:596-608, 2020.
U. von Luxburg. A tutorial on spectral clustering. Statistics and Computing, 17(4):395-416, Dec. 2007. ISSN 1573-1375.
M. J. Wainwright. High-Dimensional Statistics: A Non-Asymptotic Viewpoint. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2019.
H. Wang, S. Lohit, M. J. Jones, and Y. Fu. What makes a" good" data augmentation in knowledge distillation-a statistical perspective. In Advances in Neural Information Processing Systems, 2022.
L. Wang and K.-J. Yoon. Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
C. Wei and T. Ma. Data-dependent sample complexity of deep neural networks via lipschitz augmentation. Advances in Neural Information Processing Systems, 32, 2019.
C. Wei, K. Shen, Y. Chen, and T. Ma. Theoretical analysis of self-training with deep networks on unlabeled data. In International Conference on Learning Representations, 2021.
M. Welling and T. N. Kipf. Semi-supervised classification with graph convolutional networks. In J. International Conference on Learning Representations (ICLR 2017), 2016.
S. Yang, Y. Dong, R. Ward, I. S. Dhillon, S. Sanghavi, and Q. Lei. Sample efficiency of data augmentation consistency regularization. In International Conference on Artificial Intelligence and Statistics, pages 3825-3853. PMLR, 2023.