  Natural Language Video Description (NLVD) has recently received strong
interest in the Computer Vision, Natural Language Processing (NLP), Multimedia,
and Autonomous Robotics communities. The State-of-the-Art (SotA) approaches
obtained remarkable results when tested on the benchmark datasets. However,
those approaches poorly generalize to new datasets. In addition, none of the
existing works focus on the processing of the input to the NLVD systems, which
is both visual and textual. In this work, it is presented an extensive study
dealing with the role of the visual input, evaluated with respect to the
overall NLP performance. This is achieved performing data augmentation of the
visual component, applying common transformations to model camera distortions,
noise, lighting, and camera positioning, that are typical in real-world
operative scenarios. A t-SNE based analysis is proposed to evaluate the effects
of the considered transformations on the overall visual data distribution. For
this study, it is considered the English subset of Microsoft Research Video
Description (MSVD) dataset, which is used commonly for NLVD. It was observed
that this dataset contains a relevant amount of syntactic and semantic errors.
These errors have been amended manually, and the new version of the dataset
(called MSVD-v2) is used in the experimentation. The MSVD-v2 dataset is
released to help to gain insight into the NLVD problem.
