  Nowadays, as cameras are rapidly adopted in our daily routine, images of
documents are becoming both abundant and prevalent. Unlike natural images that
capture physical objects, document-images contain a significant amount of text
with critical semantics and complicated layouts. In this work, we devise a
generic unsupervised technique to learn multimodal affinities between textual
entities in a document-image, considering their visual style, the content of
their underlying text and their geometric context within the image. We then use
these learned affinities to automatically cluster the textual entities in the
image into different semantic groups. The core of our approach is a deep
optimization scheme dedicated for an image provided by the user that detects
and leverages reliable pairwise connections in the multimodal representation of
the textual elements in order to properly learn the affinities. We show that
our technique can operate on highly varying images spanning a wide range of
documents and demonstrate its applicability for various editing operations
manipulating the content, appearance and geometry of the image.
