An exciting outcome of research at the intersection of language and vision is that of zeroshot learning (ZSL). ZSL promises to scale visual recognition by borrowing distributed semantic models learned from linguistic corpora and turning them into visual recognition models. However the popular word-vector DSM embeddings are relatively impoverished in their expressivity as they model each word as a single vector point. In this paper we explore word-distribution embeddings for ZSL. We present a visual-linguistic mapping for ZSL in the case where words and visual categories are both represented by distributions. Experiments show improved results on ZSL benchmarks due to this better exploiting of intra-concept variability in each modality
Recently, Mukherjee REF further verified that a visuallinguistic mapping where words and visual categories are both represented by distribution can improve result at the intersection of language and vision, due to the better exploiting of intra-concept variability in each modality.
Gaussian Visual-Linguistic Embedding for Zero-Shot Recognition
EMNLP
Computer Science