# Enhanced Image Captioning with Color Recognition Using Deep Learning Methods
## 2 Related Works  
There exist numerous research works related to image captioning. Initially, image captioning was performed under constrained conditions. For example, [PERSON] et al. [2] used hierarchical actions to describe human activities from a video image, while [PERSON] et al. [3] presented an image captioning method using a dictionary of objects and language templates. However, such constrained methods of image captioning are not applicable to daily life [4]. There are two other common types of image captioning methods: retrieval-based methods [5, 6, 7, 8, 9, 10], and template-based methods [11, 12, 13, 14, 15]. In retrieval-based methods, the visually similar images are retrieved with their captions from the training dataset. Template-based methods require a predefined sentence template for each category ofimages during training processes. Neither method is good enough, as the obtained image captions contain lack descriptive details. Today, encoder-decoder models are widely used for image captioning, as they can better produce the image captions accurately as compared with template- and retrieval-based models. In general, the encoder uses the convolutional neural network for feature extraction, while the decoder uses the recurrent neural network for caption generation. Moreover, an attention mechanism has been adopted to filter out the unnecessary information and, thus, improve the accuracy of image descriptions. [PERSON] et al. [16] proposed an encoder-decoder-based image captioning model, where an adaptive attention mechanism and a long short-term memory network (LSTM) were used. [PERSON] et al. [17] proposed a multilayer dense attention mechanism for image caption generation, using faster-RCNN as an encoder and LSTM-attend as a decoder. [PERSON] et al. [18] also presented an encoder-decoder model for remote sensing image captioning based on a convolutional neural network (CNN) and LSTM combined with a novel attention model called a visual aligning attention model. [PERSON] et al. [19] proposed a hierarchical LSTM with an adaptive attention-based encoder-decoder model for visual captioning. [PERSON] et al. [20] proposed an end-to-end deep learning approach for image captioning; essentially, a CNN and LSTM with attention were considered in the previous encoder-decoder structure for feature extraction and image captioning. None of the aforementioned papers have discussed the color information of the objects. The object color is an important factor in object recognition and, thus, in image captioning. In the proposed method, color analysis is also addressed to make the image captioning more descriptive and accurate. Some of the research works based on color detection are discussed as follows: [PERSON] et al. [21] used image segmentation and color analysis to distinguish fruits from leaves and branches. [PERSON] et al. [22] proposed an algorithm to detect the color signal light image of a UAV. [PERSON] et al. [23] proposed an automatic grabbing robot using the OpenCV image processing library for object color recognition. [PERSON] et al. [24] proposed a license plate recognition method based on color features, where the location of license plates can be recognized from their hue histogram and shape. This paper proposes an ROS-based image captioning model, where the VGG16 convolutional network is employed as an encoder to extract the image features, and an LSTM neural network with attention is used as a decoder for semantic language processing. Moreover, Mask R-CNN with OpenCV is used for object detection and color recognition. In this paper, an enhanced image captioning model is proposed to produce captions with more descriptive information of images. The contributions of this paper are given as follows:  
1. An enhanced image captioning algorithm is proposed that can successfully generate the textual description of an image;
2.