# Image-Captioning Model Compression
###### Abstract  
Image captioning is a very important task, which is on the edge between natural language processing (NLP) and computer vision (CV). The current quality of the captioning models allows them to be used for practical tasks, but they require both large computational power and considerable storage space. Despite the practical importance of the image-captioning problem, only a few papers have investigated model size compression in order to prepare them for use on mobile devices. Furthermore, these works usually only investigate decoder compression in a typical encoder-decoder architecture, while the encoder traditionally occupies most of the space. We applied the most efficient model-compression techniques such as architectural changes, pruning and quantization to several state-of-the-art image-captioning architectures. As a result, all of these models were compressed by no less than 91% in terms of memory (including encoder), but lost no more than 2% and 4.5% in metrics such as CIDEr and SPICE, respectively. At the same time, the best model showed results of 127.4 CIDEr and 21.4 SPICE, with a size equal to only 34.8 MB, which sets a strong baseline for compression problems for image-captioning models, and could be used for practical applications. image captioning; model compression; pruning; quantization +
Footnote †: journal: Applied Sciences  
1
Footnote 1: email: {[[https://www.mdpi.com/journal/appl](https://www.mdpi.com/journal/appl)]([https://www.mdpi.com/journal/appl](https://www.mdpi.com/journal/appl))  
as they usually weigh 500 MB and more. The problem of model compression for image captioning is very poorly studied. Even presented papers [20; 21; 22] usually only research decoder compression, although the encoder is an integral part of the model used when applying it to new data. At the same time, model-compression techniques in other related areas of CV and NLP are well studied. So, for example, research on the various architectures of the object detector in the image (which is often used as an encoder for the image-captioning task) [23; 24; 25] have been conducted for a long time, with the aim of significantly reducing their size and increasing the operating speed without losing quality (or even increasing it). On the other hand, methods of compressing NLP models are investigated for various tasks, starting from text classification [26], and ending with text translation [27], which is similar in meaning to the image-captioning problem. In this article, we have decided to fill this gap and undertake a comprehensive use and analysis of deep neural networks (DNN) model-compression techniques [28] applied to image-captioning models. We investigated various encoder architectures specifically tailored to the task of image detection in a low-resource environment. We also applied methods of compressing models to the decoder, reducing the architecture of the model itself, as well as applying various pruning [29] and quantization [30] methods. As a result, we were able to significantly reduce the occupied space of two state-of-the-art models named Up-Down [9] and AoANet [10], while not losing much in terms of important metrics. For example, using our approach, we were able to reduce the size of the classic Up-Down model from 661.4 MB to 58.5 MB (including the encoder), that is, by 91.2%. At the same time, the main image-captioning metrics such as CIDEr [31] and SPICE [32] decreased from 120.1 and 21.4 to 118.4 and 20.8, respectively (by 1.4% and 2.8%, respectively). For the AoANet model, the size reduction was 95.6% from 791.8 MB to 34.8 MB, the CIDEr and SPICE metrics fell by 1.7% and 4%, respectively, from 129.8 to 127.6 and from 22.4 to 21.5, respectively. Thus, the best-resulting model reaches 127.6 CIDEr and 21.5 SPICE at a size of only 34.8 MB.