Video Captioning Based on Joint Imagexe2x80x93Audio Deep Learning Techniques With the advancement in technology, deep learning has been widely used for various multimedia applications. Herein, we utilized this technology to video captioning. The proposed system uses different neural networks to extract features from image, audio, and semantic signals. Image and audio features are concatenated before being fed into a long shortterm memory (LSTM) for initialization. The joint audioimage features help the entire semantics to form a network with better performance.A bilingual evaluation understudy algorithm (BLEU)xe2x80x94an automatic speech scoring mechanismxe2x80x94was used to score sentences. We considered the length of the word group (one word to four words); with the increase of all BLEU scores by more than 1%, the CIDErD score increased by 2.27%, and the METEOR and ROUGEL scores increased by 0.2% and 0.7%, respectively. The improvement is highly significant.

LSTM language model adaptation with images and titles for multimedia automatic speech recognition Transcription of multimedia data sources is often a challenging automatic speech recognition (ASR) task. The incorporation of visual features as additional contextual  a :22,means to improve ASR for this data has recently  :33,from researchers. Our investigation extends existing ASR methods by using images and video titles to adapt  :50,neural network (RNN) language model with a longshort term memory (LSTM) network. Our language  :65,tested on transcription of an existing corpus of  and on a :74,new corpus consisting of lecture videos. Consistent reduction in perplexity by 510 is observed on  :93,When the nonadapted model is combined with  :101,adaptation and video title :101,adaptation models for  :109,hypotheses reranking, additionally the word error  is :116,decreased by around 0.5% on both datasets.  the :125,output word probabilities of the model, it  :134,that both image :101,adaptation and video title  the model :142,more confidence in the choice of  :151,informative words