Task‐aware swapping for efficient DNN inference on DRAM‐constrained edge systems

International Journal of Intelligent SystemsEarly View RESEARCH ARTICLE Task-aware swapping for efficient DNN inference on DRAM-constrained edge systems Cheng Ji, Cheng Ji orcid.org/0000-0002-2525-8070 School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China Cheng Ji and Zongwei Zhu contribute equally.Search for more papers by this authorZongwei Zhu, Corresponding Author Zongwei Zhu zzw1988@ustc.edu.cn zwjsec@mail.ustc.edu.cn orcid.org/0000-0003-3607-2631 Suzhou Institute for Advanced Study, University of Science and Technology of China, Suzhou, China Cheng Ji and Zongwei Zhu contribute equally. Correspondence Zongwei Zhu, Suzhou Institute for Advanced Study, University of Science and Technology of China, 215123 Suzhou, China. Email: zzw1988@ustc.edu.cn and zwjsec@mail.ustc.edu.cn Xianmin Wang, Institute of Artificial Intelligence and Blockchain, Guangzhou University, 510700 Guangzhou, China. Email: xianmin@gzhu.edu.cnSearch for more papers by this authorXianmin Wang, Corresponding Author Xianmin Wang xianmin@gzhu.edu.cn orcid.org/0000-0003-3480-8780 Institute of Artificial Intelligence and Blockchain, Guangzhou University, Guangzhou, China Correspondence Zongwei Zhu, Suzhou Institute for Advanced Study, University of Science and Technology of China, 215123 Suzhou, China. Email: zzw1988@ustc.edu.cn and zwjsec@mail.ustc.edu.cn Xianmin Wang, Institute of Artificial Intelligence and Blockchain, Guangzhou University, 510700 Guangzhou, China. Email: xianmin@gzhu.edu.cnSearch for more papers by this authorWenjie Zhai, Wenjie Zhai orcid.org/0000-0002-8335-0932 Suzhou Institute for Advanced Study, University of Science and Technology of China, Suzhou, ChinaSearch for more papers by this authorXuemei Zong, Xuemei Zong orcid.org/0000-0002-9655-4256 Jiangsu XCMG Construction Machinery Research Institue Ltd., Xuzhou, ChinaSearch for more papers by this authorAnqi Chen, Anqi Chen orcid.org/0000-0002-0504-4892 School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSearch for more papers by this authorMingliang Zhou, Mingliang Zhou orcid.org/0000-0002-1874-3641 College of Computer Science, Chongqing University, Chongqing, ChinaSearch for more papers by this author Cheng Ji, Cheng Ji orcid.org/0000-0002-2525-8070 School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China Cheng Ji and Zongwei Zhu contribute equally.Search for more papers by this authorZongwei Zhu, Corresponding Author Zongwei Zhu zzw1988@ustc.edu.cn zwjsec@mail.ustc.edu.cn orcid.org/0000-0003-3607-2631 Suzhou Institute for Advanced Study, University of Science and Technology of China, Suzhou, China Cheng Ji and Zongwei Zhu contribute equally. Correspondence Zongwei Zhu, Suzhou Institute for Advanced Study, University of Science and Technology of China, 215123 Suzhou, China. Email: zzw1988@ustc.edu.cn and zwjsec@mail.ustc.edu.cn Xianmin Wang, Institute of Artificial Intelligence and Blockchain, Guangzhou University, 510700 Guangzhou, China. Email: xianmin@gzhu.edu.cnSearch for more papers by this authorXianmin Wang, Corresponding Author Xianmin Wang xianmin@gzhu.edu.cn orcid.org/0000-0003-3480-8780 Institute of Artificial Intelligence and Blockchain, Guangzhou University, Guangzhou, China Correspondence Zongwei Zhu, Suzhou Institute for Advanced Study, University of Science and Technology of China, 215123 Suzhou, China. Email: zzw1988@ustc.edu.cn and zwjsec@mail.ustc.edu.cn Xianmin Wang, Institute of Artificial Intelligence and Blockchain, Guangzhou University, 510700 Guangzhou, China. Email: xianmin@gzhu.edu.cnSearch for more papers by this authorWenjie Zhai, Wenjie Zhai orcid.org/0000-0002-8335-0932 Suzhou Institute for Advanced Study, University of Science and Technology of China, Suzhou, ChinaSearch for more papers by this authorXuemei Zong, Xuemei Zong orcid.org/0000-0002-9655-4256 Jiangsu XCMG Construction Machinery Research Institue Ltd., Xuzhou, ChinaSearch for more papers by this authorAnqi Chen, Anqi Chen orcid.org/0000-0002-0504-4892 School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSearch for more papers by this authorMingliang Zhou, Mingliang Zhou orcid.org/0000-0002-1874-3641 College of Computer Science, Chongqing University, Chongqing, ChinaSearch for more papers by this author First published: 01 June 2022 https://doi.org/10.1002/int.22933 Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Object detection at the edge side is a common task in various environments. The deployment of convolutional neural networks in intelligent edge systems is very challenging because of the highly constrained main-memory space. This study aims at operating neural networks with a reduced memory requirement. The basic idea is that tasks of the same type would involve the same critical subnetwork. We propose identifying the critical network connections by considering the importance of channels. During runtime, the proposed method detects the task types and timely swaps the model parameters of the critical subnetworks from the external storage into dynamic random access memory (DRAM). Compared with conventional network pruning, the proposed approach further reduced the DRAM requirement by 34.6% while maintaining a high inference accuracy. Open Research DATA AVAILABILITY STATEMENT Research data are not shared. Early ViewOnline Version of Record before inclusion in an issue RelatedInformation