[1] Maram Assi and Ramzi A. Haraty. A survey of the knapsack problem. In Proceedings of ACIT, pages 1-6, 2018. https://ieeexplore.ieee.org/document/8672677.
[2] Olivier Beaumont, Lionel Eyraud-Dubois, and Alena Shilova. Optimalgpu-cpu offloading strategies for deep neural network training. In Proceedings of Euro-Par, pages 151-166, 2020. https://link.springer.com/chapter/10.1007/978-3-030-57675-2_10.
[3] Olivier Beaumont, Lionel Eyraud-Dubois, and Alena Shilova. Efficient combination of rematerialization and offloading for training dnns. In Proceedings of NeurIPS, volume 34, pages 23844-23857, 2021. https://papers.nips.cc/paper_files/paper/2021/file/c8461bf13fca8a2b9912ab2eb1668e4b-Paper.pdf.
[4] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In Proceedings of NeurIPS, volume 33, pages 1877-1901, 2020. https://arxiv.org/pdf/2005.14165.pdf.
[5] Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174, 2016. https://arxiv.org/pdf/1604.06174.pdf.
[6] Xiaoming Chen, Danny Z. Chen, and Xiaobo Sharon Hu. modnn: Memory optimal dnn training on gpus. In Proceedings of DATE, pages 13-18, 2018. https://ieeexplore.ieee.org/document/8341972.
[7] Tobias Dantzig and Joseph Mazur. Number: The language of science. 2007. https:// philpapers.org/rec/DANNTL.
[8] Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Marc'aurlio Ranzato, Andrew Senior, Paul Tucker, Ke Yang, et al. Large scale distributed deep networks. In Proceedings of NeurIPS, volume 25, 2012. https://proceedings.neurips.cc/paper/2012/file/6aca97005c68f1206823815f66102863-Paper.pdf.
[9] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL, pages 4171–4186, 2019. https://www.aclweb.org/anthology/N19-1423.pdf.
[10] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. https://arxiv.org/pdf/2010.11929.pdf.
[11] Priya Goyal, Piotr Dólar, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch sgd: TrainingImagenet in 1 hour. arXiv preprint arXiv:1706.02677, 2017. https://arxiv.org/pdf/1706.02677.pdf.
[12] Andreas Griewank and Andrea Walther. Algorithm 799: Revolve: An implementation of checkpointing for the reverse or adjoint mode of computational differentiation. ACM Transactions on Mathematical Software (TOMS), 26(1):19-45, 2000. https://dl.acm.org/doi/pdf/10.1145/347837.347846.
[13] José Grimm, Loïc Pottier, and Nicole Rostaing-Schmidt. Optimal time and minimum space-time product for reversing a certain class of programs. PhD thesis, INRIA, 1996. https://inria.halscience/inria-00073896/file/RR-2794.pdf.
[14] Julien Herrmann, Olivier Beaumont, Lionel Eyraud-Dubois, Julien Hermann, Alexis Joly, and Alena Shilova. Optimal checkpointing for heterogeneous chains: How to train deep neural networks with limited memory. arXiv preprint arXiv:1911.13214, 2019. https://arxiv.org/pdf/1911.13214.pdf.