Positional Selfattention Based Hierarchical Image Captioning Intelligent robotic systems process images captured by camera devices to get highlevel semantic concepts. Current image captioning approaches use convolutional neural networks as encoder and recurrent neural networks as decoder. Various modalities are treated differently and parallel computation is not fully allowed due to the chain architecture. To address these issues, we present a parallel hierarchical neural network based on encoderdecoder architecture to generate descriptions for images. Only convolutional neural networks are adopted here, and the hierarchical framework takes less steps to capture distant dependencies. Masked positional selfattention mechanism is utilized in the decoder to improve the performance. The fixed sized windows make it possible for parallel computation. This generative model uses unified architecture for different modalities, reaching a BLEU1 score (the higher the better) over 0.7 with a higher training speed.

Attention Analysis in Caption Generation Caption Generation is one of the fundamental tasks combining computer vision and natural language processing. To achieve this goal, neural networks are employed to implement a caption generation system. In this paper, we proposed a caption generation system combining a CNNbased object detection system and a language model with a recurrent neural network. Especially, a vector which is sent from the object detection system to the language model is generated using an attention mechanism. Attention visualization can help us to understand the system focuses on a part of the input image in generating a caption. In the experiments, we evaluate the performance of the proposed system and discuss the effects of the attention mechanism in the image caption. Especially, the attention contributes to the improvement of caption generation but the attention is uncorrelated to system interpretation.