ActInf GuestStream #032.1 ~ Adam Pease "A Neuro-Symbolic Approach to Language Understanding"
Channel: Active Inference Institute
Date: 11/01/2023

A Neuro-Symbolic Approach to Language Understanding: Using an Ontology to Create A Language Model
Adam Pease
https://www.adampease.org/
https://www.ontologyportal.org/
Adam's YouTube channel: https://www.youtube.com/channel/UCVyCle22zz5gZjWvNWCBq9Q 

Formal linguistic semantics has a long tradition of using logic as a way
to capture the objective semantics of language.  We know an
interpretation is wrong if apparently consistent sets of sentences yield
a contradiction when a deterministic method is applied.  But creating
those models has been largely limited to academic works in which a
theory is embodied in rules on paper.  Computational models, in
contrast, have had very shallow or approximate interpretations of
semantics.

In this current work, I describe how to generate a very large set
(1TB or larger) set of language-logic pairs that is then used to train a
neural-network based language model.  Because the sentences are created
synthetically, we can use a representation that can also generate a
consistent logical interpretation of the sentence automatically.
Because this model uses the Suggested Upper Merged Ontology (SUMO), tens
of thousands of concepts are used.  Because SUMO uses a higher order
logic, we can generate sentences far more interesting than can be
captured in simple triples.  We can use quantifiers, modals, epistemic
and teleological operators and arbitrary levels of embedded clauses.
Because SUMO has been mapped to all of the WordNet lexicon, we can also
freely include synonyms for all concepts, if they exist.  This leads to
a data set that can cover a diverse and interesting subset of English.

Active Inference Institute information: 
Website: https://activeinference.org/ 
Twitter: https://twitter.com/InferenceActive 
Discord: https://discord.gg/8VNKNp4jtx 
YouTube: https://www.youtube.com/c/ActiveInference/ 
Active Inference Livestreams: https://coda.io/@active-inference-institute/livestreams