Title: Learning Representations Through Contrastive Neural Model Checking
Primary area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords: Contrastive Learning, CLIP, Neurosymbolic, Formal Methods, Verification, Representation Learning

Abstract: Model checking is a key technique for verifying safety-critical systems against formal specifications, where recent applications of deep learning have shown promise. However, while ubiquitous for vision and language domains, representation learning remains underexplored in formal verification. We introduce Contrastive Neural Model Checking (CNML), a novel method that leverages the model checking task as a guiding signal for learning aligned representations. CNML jointly embeds logical specifications and systems into a shared latent space through a self-supervised contrastive objective. On industry-inspired retrieval tasks, CNML considerably outperforms both algorithmic and neural baselines in cross-modal and intra-modal settings. We further show that the learned representations effectively transfer to downstream tasks and generalize to more complex formulas.  These findings demonstrate that model checking can serve as an objective for learning  representations for formal languages.

TL;DR: Contrastive learning on systems and specifications leads to learning aligned representations that can be transferred to a wide array of tasks and used for cross- or intra modal search