A Formal Model of the Mechanismof Semantic Analysis in the Brain
Yuuji Ichisugi and Naoto Takahashi
National Institute of Advanced Industrial Science and Technology (AIST),
Central 1,Tsukuba,Ibaraki 305-8560, Japan{y-ichisugi,naoto.takahashi}@aist.go.jp

Abstract. We propose a formal model of the mechanism of semantic
analysis in the language areas of the cerebral cortex. The frameworkof Combinatory Categorial Grammar, a framework of grammar description in theoretical linguistics, is modiﬁed so that it does not use lambda
calculus to represent semantic rules. This model uses a novel form of
semantic representation named hierarchical address representation, anduses only ﬁxed-length data structures. The knowledge of syntax and theknowledge of semantics are clearly separated in this model. Therefore, it
is possible to reproduce disorders speciﬁc to syntax (utterance similar to
Broca’s aphasia) and disorders speciﬁc to semantics (utterance similar
to Wernicke’s aphasia) by disabling diﬀerent modules in the model. We
estimate that the model can be implemented using the Bayesian network model of the cerebral cortex that we have proposed earlier. We
believe that this research will connect computational neuroscience andtheoretical linguistics, and greatly evolve both of them.

1

Introduction

The language areas, which are considered to be centers of human language activities, are parts of the cerebral cortex. There is a hypothesis[3] that claims“the cerebral cortex is a kind of Bayesian network.” If so, we must be able tobuild a system that reproduces the behavior of the human language areas usinga Bayesian network. Thus we aim at constructing a system that processes Combinatorial Categorial Grammar(CCG)[2], a framework of grammar description,using a Bayesian network[9]. As the ﬁrst step towards the aim, we propose aformal model of the mechanism of semantic analysis in the brain based on CCG.
Theoretical linguistics is one ﬁeld of linguistics that analyzes the characteristics of natural languages by mathematical methods. The relation between linguistics and theoretical linguistics resembles the relation between neuroscienceand computational neuroscience. One purpose of theoretical linguistics is to ﬁnd
out some characteristics shared by all existing natural languages. Such characteristics can be considered as the characteristics of the information processingof the language areas in the brain.
Lambda calculus is usually used as a tool to describe semantic rules of CCG;
however, it is diﬃcult to handle variable-length data structures like lambda terms

This is a pre-print of a contribution published in Samsonovich A. (eds) Biologically Inspired Cognitive Architectures 2018
(BICA 2018), published by Springer.
The final authenticated version is available online at: https://doi.org/10.1007/978-3-319-99316-4_17

2

Yuuji Ichisugi and Naoto Takahashi

by Bayesian networks. The proposed model uses a novel form of semantic representation named hierarchical address representation, which does not use lambda
terms. The model uses only ﬁxed-length data structures. We estimate that themodel can be implemented within the framework that we have proposed[5][9],which is based on Bayesian networks.
Although it is unknown whether the proposed method is applicable to thesyntactic rules and the semantic rules of all the existing natural languages, we
believe that this research will connect computational neuroscience and theoretical linguistics and greatly evolve both of them.
Sec.2 brieﬂy explains CCG. Sec.3 describes the proposed model and examples
of analysis of some sentences. Sec.4 describes the correspondence between themodules of the model and some areas of the cerebral cortex, then we demonstrate
reproduction of utterance of aphasia by disabling several modules.

2

Combinatory Categorial Grammar (CCG)

CCG is one of the most successful frameworks of grammar description. Its expressive power of grammar description is “mildly context-sensitive”, which locates in between context-sensitive and context-free in the Chomsky hierarchy.
Although the framework is very simple, grammars deﬁned in CCG successfully
explain many language phenomena (even though it is not complete). Therefore,
we consider that CCG is the theory of information processing of the language
areas in the brain.
In theoretical linguistics, some frameworks use unification as the core operation of analysis, and they are called unification grammars. CCG is one of
uniﬁcation grammars.
In CCG, general syntactic categories have structures that consist of groundcategories (e.g., S for sentence, N P for noun phrase) combined by the operators“/” and “\”. Theoretically, the length of a syntactic category is not restricted.
A ground category may have syntactic features. A syntactic feature may be
a discrete variable whose value will be determined by the uniﬁcation operations
during the process of syntactic analysis. A ground category G with a syntactic
feature F is denoted as GF .
Production rules are deﬁned by the form of inference rules in CCG. Syntactic analysis (i.e. parsing) is formalized as proof search showing a given word
sequence being a sentence. A parse tree obtained as a result of syntactic analysis
corresponds to a proof diagram.
The inference rules in CCG are accompanied by semantic rules that compose
meaning of phrases. For example, the function which is the meaning of the word
“black” : λx.black(x) is applied to the term that means “cats” : cats to get themeaning of the phrase “black cats” : black(cats). The semantic representationof the whole sentence is obtained by performing function applications (i.e. beta
reductions) or function compositions sequentially from the leaves to the root,
along with the parse tree.

A Formal Model of the Mechanism of Semantic Analysis in the Brain

3

In theoretical linguistics, including CCG, lambda calculus has been used asa tool for describing semantics; however, it is hard to imagine that the actual
neural networks in the language areas have realized the complicated lambda
calculus. In the following section, we propose a model of semantic analysis that
does not use lambda calculus.
Lexicon

Ϯ

ϭ

Words

Parserϰ
Addresses

ϯ
Syntactic Categories

ϱ

Ϯ

Ϯ

ϲ
Gates

ϳ

Semantic Representations

Fig. 1. The architecture of the model and typical information ﬂow among the modules
during the parsing process.

3

The proposed model

3.1

Scope of the model

We aim at constructing a model of the mechanism for unconscious and instantinterpretation of the superﬁcial and literal meaning of comparatively simple sentences. Neither conscious interpretation of sentences with complicated structures
nor presumption of unexpressed intention is a target of this model.
The length of word sequences, the length of syntactic categories, and thelength of generated semantic representations are all limited. Although human
can interpret a long sentence incrementally from the head, this model assumes
that all words are given at once.
We adopt a simpliﬁed English grammar for a straightforward explanationof the behavior of the proposed model. Although the model accepts complex
sentences consisting of two clauses connected with a subordinating conjunction(“if”, “after”, etc.) and sentences containing relative pronouns (“which”, etc.),
it is assumed that a subordinating conjunction or a relative pronoun appears at
most once in a sentence.
3.2

The outline of the process of semantic analysis

In this model, syntactic analysis and semantic analysis are conducted simultaneously. Fig.1 shows a typical information ﬂow among the modules during parsing.
(1) First, a word sequence is given. (2) By referring to the lexicon, three data

4

Yuuji Ichisugi and Naoto Takahashi
SR
Address(sconj, −, −)ifbig(c1, agent, size)*(c1, agent, color)(c1, agent, entity) dogs
(c1, modality, −) *chase(c1, action, −)(c1, patient, size) small(c1, patient, color) *(c1, patient, entity) mice*(c2, agent, size)
black(c2, agent, color)(c2, agent, entity) cats(c2, modality, −) may
eat(c2, action, −)(c2, patient, size) *(c2, patient, color) *(c2, patient, entity) mice

>
>B

X/Y

Y
X

X/Y

>T

Y/Z
X/Z

X
T/(T\X)

<
<B

Y

X\Y
X

Y\Z

<T

X\Y
X\Z

X
T\(T/X)

Fig. 3. The inference rules in the proposedmodel. These are identical to the inference
rules of the usual CCG except that there
are no semantic rules using lambda calculus.

Fig. 2. All addresses in the prototype system and semantic representations obtained
as a result of the semantic analysis of thesentence “if small mice areChasedBy big
dogs black cats may eat mice”. Undetermined values are denoted by “∗”.

structures, a syntactic category, an address and a semantic representation, areobtained for each word. Variables may be unbound in syntactic features andin addresses at the time. (3) The parser merges syntactic categories and ﬁnds
a parse tree in which the whole word sequence forms a sentence. During thisprocess, unbound syntactic features receive a value through uniﬁcation operations. (4) If a complete parse tree is generated, all values of syntactic features
are determined. (5) All values of variables contained in the addresses are also
determined. (6) A set of pairs of addresses and semantic representations that
represents the meaning of the whole sentence is obtained. (7) Each semantic
representation is written into the “memory” at the corresponding address.
Generally speaking, the information ﬂow is not limited in the direction above.
For example, when a prior knowledge for the meaning is given, the information
will ﬂow backwards from the semantic representation module to the parser module and the parser can use it to resolve lexical or syntactic ambiguities. Moreover,
it is also possible to infer appropriate word sequences when a semantic representation is given. We show such an example in Sec. 4.

A Formal Model of the Mechanism of Semantic Analysis in the Brain

3.3

5

Hierarchical address representation, inference rules and lexicon

In this model, the meaning of a sentence is represented as a set of pairs ofaddresses and semantic representations. We call this form “hierarchical address
representation”.
Each address is a tuple of the three values (C, R, F). Each semantic representation is written into an address. All possible addresses in the prototype
system are listed in Fig.2.
The topmost hierarchy of an address, C, represents the index of a clause in
a complex sentence, which uses either a subordinate conjunction (e.g. “if”) or a
relative pronoun (e.g. “which”). The value of C is either c1 (for the ﬁrst clause)or c2 (for the second). The type of the subordinate conjunction is written into
the special address: (sconj, −, −). For example, when the sentence starts with an“if”, the semantic representation of “if” is written into the said special address.
The second hierarchy of an address, R, primarily indicates the semantic roleof a word in the sentence. In the current prototype system, its value is limited
to either agent or patient, but it can be extended for other semantic roles (e.g.
instrument, location, time) easily. Also, R can take the value action (for thesemantic representation of a verb) or modality (for the semantic representationof an auxiliary verb).
When the value of R is agent or patient, i.e. when the word is a part ofa noun phrase, the third hierarchy, F, indicates a feature of the word. In the
prototype system, the value of F is limited to color, size, and entity. F being
entity means that the word is the head noun of a noun phrase.
The hierarchical address representation is inspired by a neuroscientiﬁc ﬁnding about the encoding of sentence meaning in the brain[8], which suggests thateach semantic role (agent, patient) has its own representing place in the brain,
regardless of the superﬁcial voice (active or passive) of the sentence. We suppose that ﬁxing positions for all elements of meaning facilitates learning andprocessing of language for neural networks and Bayesian networks in the brain.
Fig.3 shows the inference rules in the proposed model. There are no semantic rules using lambda calculus. The semantic analysis is performed by merely
uniﬁcation operations of syntactic features as explained in the next subsection.
The lexicon is a set of lexical items. A lexical item in this model is represented
as a tuple of four data structures (a word, a syntactic category, an address, anda semantic representation), as shown in Fig.4.
3.4

Examples of analysis

Fig.5 shows the parse tree of the sentence “black cats eat mice”.
Let us explain the process of analyzing the phrase “black cats” in detail. First,
by referring to the lexicon for these words, the corresponding syntactic categories
N PC1 ,R1 /N PC1 ,R1 and N PC2 ,R2 are obtained. Next, the inference rule “>” ischosen because this rule has the premises that unify with the obtained syntactic
categories. Then, the inference rule is applied to the syntactic categories and themerged syntactic category N PC1 ,R1 is derived as a result. This derived syntactic

6

Yuuji Ichisugi and Naoto Takahashi

Syntactic Category
Address
SR
Word
’if’
(Sc1 /Sc2 )/Sc1(sconj, −, −)if
N PC,R /N PC,R
(C, R, color)
black’black’N PC,R /N PC,R
(C, R, size)
big
’big’
N PC,R(C, R, entity)
cats’cats’(SC \N PC,agent )/N PC,patient(C, action, −)
eat
’eat’
(C, action, −)eat’areEatenBy’ (SC \N PC,patient )/N PC,agent(SC \N PC,R )/(SC \N PC,R )(C, modality, −) may’may’(N Pc1,R1 \N Pc1,R1 )/(Sc2 \N Pc2,R2 ) (c2, R2 , entity) (c1, R1 , entity)’which’(N Pc1,R1 \N Pc1,R1 )/(Sc2 /N Pc2,R2 ) (c2, R2 , entity) (c1, R1 , entity)’which’Fig. 4. Examples of lexical items contained in the lexicon. Each bold letter denotes anunbound variable. The same variables within a lexical item must have the same value
at the end of the parsing. The ﬁrst lexical item of “which” is used for the nominative
case, and the second one is for the objective case.

category itself is merged with other syntactic categories as the analysis goes
further.
The analysis progresses in such a way until the syntactic category SC (sentence) is ﬁnally derived. At that time, the proof of the whole word sequence
being a sentence is completed. By the uniﬁcations performed during the processof analysis, the semantic roles R2 (for “cats”) and R4 (for “mice”) are determined as agent and patient, respectively. Then, the address where the semantic
representation of each word should be written is determined. The addresses andthe semantic representations ﬁnally obtained are shown in Fig. 7.
Fig.6 shows a part of the parse tree of the sentence “mice which dogs chaseareEatenBy cats” that uses a relative pronoun of the objective case. The analysisof this sentence requires the type raising rule “> T ”. The semantic representationof the relative pronoun “which” is the address of its antecedent (Fig.8).

black
cats
N PC1 ,R1 /N PC1 ,R1 N PC2 ,R2>
N PC1 ,R1
<

eat
mice
(SC3 \N PC3 ,agent )/N PC3 ,patient N PC4 ,R4>SC3 \N PC3 ,agent
SC3

Fig. 5. A parse tree (proof diagram) of the sentence “black cats eat mice”. Although thevariables Ri , semantic roles, are unbound at the beginning, their values are determinedas R1 = R2 = agent and R4 = patient by uniﬁcation operations during parsing. A
parse tree of the sentence “black cats areEatenBy mice” results in the same form
but the semantic roles of agent and patient are exchanged. The variables Ci are left
unbound after parsing; however, they are restricted to have the same value, i.e. C1 =C2 = C3 = C4 .

A Formal Model of the Mechanism of Semantic Analysis in the Brain

7

dogs
N PC3 ,R3

>

chase
>T
T3 /(T3 \N PC3 ,R3 ) (SC4 \N PC4 ,agent )/N PC4 ,patientwhich>B(N Pc1,R21 \N Pc1,R21 )/(Sc2 /N Pc2,R22 )SC4 /N PC4 ,patient
N Pc1,R21 \N Pc1,R21

Fig. 6. A parse tree of the word sequence “which dogs chase”. In this relative clause,
whose index is C3 = C4 = c2, the semantic roles of “dogs” and “which” become R3 =agent and R22 = patient, respectively. The index of the main clause is determined as
c1, but the semantic role of the antecedent R21 , is determined only when the wholesentence has been analyzed.

SR
Address(c1, agent, entity) catseat(c1, action, −)(c1, patient, entity) mice(c2, agent, entity) dogs
chase(c2, action, −)(c2, patient, entity) (c1, patient, entity)

AddressSR(c1, agent, color)black(c1, agent, entity) catseat(c1, action, −)(c1, patient, entity) mice
Fig. 7. The pairs of addresses andsemantic representations obtainedby analyzing the sentence “blackcats eat mice”. Because the valueof the address C is arbitrary, we
set it as c1.

Fig. 8. The pairs of addresses and semantic representations obtained by analyzing the sentence “mice which dogs
chase areEatenBy cats”.

BA44,45
Addresses,
Categories
ParserBA41,42

Semantic
Representations

Words
Lexicon

BA22

Fig. 9. A possible correspondence between the modules in the model (Fig. 1) andcortical areas.

8

4

Yuuji Ichisugi and Naoto Takahashi

Correspondence to cortical areas

A possible correspondence between the modules in the model (Fig. 1) and cortical areas is shown in Fig. 9. Broca’s area (Brodmann Areas 44 and 45) participates in grammar processing and Wernicke’s area (BA22; close to the primary
auditory area (BA41,42) and to the angular gyrus (BA39)) participates in association between speech sounds and concepts[7]. The parser in the proposedmodel is the module of grammar processing, thus it is matched with Broca’sarea. Because it is suggested that agents and patients are represented at theleft mid-superior temporal gyrus[8], we suppose that the human’s module forsemantic representations is located there, around BA22.
By “disabling” a part of the model, utterance that is similar to Broca’s
aphasia or to Wernicke’s aphasia can be reproduced. Although the symptoms
of aphasia[7] is complicated and largely vary from patient to patient, we give
simple explanation below. Broca’s aphasia arises from damage to Broca’s area.
Its utterance consists of scattering words that do not constitute sentences. The
Wernicke’s aphasia arises from damage to the Wernicke’s area. Its utterance isﬂuent but does not make sense because of mistakenly selected words.
The proposed model has been implemented in the Prolog language. First, weshow an example of a normal behavior of sentence generation. For this example,
the semantic representation shown in Fig. 7, which means “black cats eat mice”,
is given. If the model infers all possible sentences that consist of four words, thefollowing two sentences are obtained as solutions.
black cats eat mice
mice areEatenBy black cats

In the same condition but without the parser module, the obtained solutions
include syntactically incorrect word sequences; however, only those words thatare semantically suitable are chosen (Fig.10(a)). This phenomenon is essentiallythe same one seen in utterance of Broca’s aphasia.
If the model infers all possible sentences that consist of four words without
giving concrete semantic representation, all syntactically correct sentences will
be obtained as solutions(Fig.10(b)). This phenomenon is essentially the same
one seen in utterance of Wernicke’s aphasia.
In the proposed model, the parser module processes only the addresses where
meanings are written, but does not process the semantic representations themselves. Because the knowledge of syntax and the knowledge of semantics areclearly separated in this architecture, we can reproduce syntactic disorder andsemantic disorder, like actual aphasia.

5

Related work

We aim at the model of the language processing that can be realized easily inthe form of Bayesian networks or neural networks like the cerebral cortex. The
essential diﬀerence from the conventional frameworks of the formal semantics
is a complete exclusion of variable-length data structures. Although uniﬁcation

A Formal Model of the Mechanism of Semantic Analysis in the Brain(a)black black black black
black black black cats
black black black mice
black black black eat
black black black areEatenBy
black black cats black
black black cats cats
black black cats mice
...

9

(b)
white dogs eat dogs
white dogs eat cats
white dogs eat mice
white dogs chase dogs
white dogs chase cats
white dogs chase mice
white dogs areEatenBy dogs
white dogs areEatenBy cats...

Fig. 10. Inferred word sequences similar to (a)Broca’s aphasia and (b)Wernicke’s aphasia.

operations are occasionally used to express semantic rules, tree structures havebeen used to express semantic representations. MRS (Minimal Recursion Semantics)[4] expresses meanings not with a tree structure but with a ﬂat structure;
however, it needs to handle variable-length data structures.
There are some systems that parse natural language eﬃciently using loopy
belief propagation. For example, the system in [6] is an eﬃcient CCG parser;
however, it does not include semantic analysis.

6Conclusion

We proposed a model of the mechanism of the semantic analysis that does notuse variable-length data structure (e.g. lambda terms) but uses a novel form of
semantic representation named hierarchical address representation. The modulesof the model have correspondence to cortical areas in the brain.
We can reproduce utterance of aphasia by “disabling” some modules in themodel.
A chart parser for context-free grammar can be realized as a Bayesian network[9]. It should be possible to apply this method to CCG. Moreover, themechanism of the uniﬁcation that the proposed model uses is also easily realizable as Bayesian networks. If the whole model is realized as a Bayesian network,
it is possible that lexical items and inference rules can be learned from pairsof word sequences and semantic representations. We believe that this research
will connect computational neuroscience and theoretical linguistics, and greatly
evolve both of them.
Acknowledgments. We greatly thank to Dr. Daisuke Bekki, who gave usefulcomments to this research.

References
1. J. Pearl , Probabilistic Reasoning in Intelligent Systems: Networks of Plausible
Inference, Morgan Kaufmann, 1988.
2. M. Steedman, The Syntactic Process. The MIT Press, 2000.

10

Yuuji Ichisugi and Naoto Takahashi

3. T.S. Lee, D. Mumford, Hierarchical Bayesian inference in the visual cortex. Journalof Optical Society of America, A 20(7): pp.1434–1448, 2003.
4. Ann Copestake, Daniel Flickinger, Carl Pollard, and Ivan A. Sag. Minimal Recursion Semantics. An introduction. Research on Language and Computation, 2005.
5. Yuuji ICHISUGI, The Cerebral Cortex Model that Self-Organizes Conditional
Probability Tables and Executes Belief Propagation, In Proc. of IJCNN 2007,
pp.1065–1070, 2007.
6. Auli, M. and Lopez, A., A Comparison of Loopy Belief Propagation and Dual
Decomposition for Integrated CCG Supertagging and Parsing, In Proc. of ACL,
pp.470-480, 2011.
7. Eric R. Kandel et.al ed., Principles of Neural Science, Fifth Edition, McGraw-Hill
Companies, 2012.
8. Frankland S. M., Greene J. D., An architecture for encoding sentence meaning inleft mid-superior temporal cortex., Proc. Natl. Acad. Sci. U.S.A. 112, 11732-11737,
2015.
9. Naoto Takahashi and Yuuji Ichisugi, Restricted Quasi Bayesian Networks as a Prototyping Tool for Computational Models of Individual Cortical Areas, Proceedingsof Machine Learning Research (AMBN 2017), Vol .73, pp.188199, 2017.