AUTOMATIC ENHANCEMENT OF IMAGE MEMORABILITY
Creators: name: Majtanovic, Cveta
Publisher: name: Universit� degli studi di Trento
Year: 2021

description: Every picture tells a story. Images are one of the most dominant types of media, uploaded an average in billions every single day and in hundreds of billions on an annual basis. Artifacts depicting visual perception like photographs and other two-dimensional pictures are distributed through the growing number of image-sharing websites. Consequently, a thriving interest in understanding the whole image or objects depicted in it, its style or the emotions a picture might evoke, together with all the other image properties, became increasingly represented in research practice. This research focuses on the problem of automatically enhancing memorability of an image. Recent works in Computer Vision and Multimedia have shown that intrinsic image properties like memorability can be automatically inferred by exploiting powerful deep learning models. This research advances the state of the art in this area by addressing a novel and more challenging issue: ?Can we transform an arbitrary input image and make it more memorable??. To state this question properly one requires the existence of memorability measures. Methods for automatically increasing image memorability would have an impact in many application fields, such as education, gaming or advertising. To tackle the problem, we introduce an approach inspired by editing-by-applying-filters paradigm, adopted in photo editing applications like Instagram and Prisma. Users of the two apps generally have to go through the available filters before finding the desired solution which is turning the editing process into a resource- and time-consuming task. In the work conducted for the purpose of this thesis, we reverse the process: given an input image, we propose to automatically retrieve a set of ?style seeds?, i.e., a set of style images which, applied to the input image through a neural style transfer algorithm, provide the highest increase in memorability. As a result, we demonstrate that it is possible to automatically retrieve the best style seeds for a given image, thus, remarkably reducing the number of human attempts needed to find a good match. Furthermore, we show the effectiveness of the proposed approach with experiments on the publicly available LaMem dataset, performing both a quantitative evaluation and a user study. To demonstrate the flexibility of the proposed framework, we also analyze the impact of different implementation choices, such as using different state of the art neural style transfer methods. Finally, we show several qualitative results to provide additional insights on the link between image style and memorability. This approach arises from recent advances in the field of image synthesis and adopts a deep architecture for generating a memorable picture from a given input image and a style seed. Importantly, to automatically select the best style, also relying on deep models, a novel learning-based solution is proposed. The experimental evaluation, conducted on publicly available benchmarks, demonstrates the effectiveness of the proposed approach for generating memorable images through automatic style seed selection.