  In this article, we bring together theories of multimodal communication and
computational methods to study how primary school science diagrams combine
multiple expressive resources. We position our work within the field of digital
humanities, and show how annotations informed by multimodality research, which
target expressive resources and discourse structure, allow imposing structure
on the output of computational methods. We illustrate our approach by analysing
two multimodal diagram corpora: the first corpus is intended to support
research on automatic diagram processing, whereas the second is oriented
towards studying diagrams as a mode of communication. Our results show that
multimodally-informed annotations can bring out structural patterns in the
diagrams, which also extend across diagrams that deal with different topics.
