  Representing the semantics of GUI screens and components is crucial to
data-driven computational methods for modeling user-GUI interactions and mining
GUI designs. Existing GUI semantic representations are limited to encoding
either the textual content, the visual design and layout patterns, or the app
contexts. Many representation techniques also require significant manual data
annotation efforts. This paper presents Screen2Vec, a new self-supervised
technique for generating representations in embedding vectors of GUI screens
and components that encode all of the above GUI features without requiring
manual annotation using the context of user interaction traces. Screen2Vec is
inspired by the word embedding method Word2Vec, but uses a new two-layer
pipeline informed by the structure of GUIs and interaction traces and
incorporates screen- and app-specific metadata. Through several sample
downstream tasks, we demonstrate Screen2Vec's key useful properties:
representing between-screen similarity through nearest neighbors,
composability, and capability to represent user tasks.
