# Ontology as manifold: towards symbolic and numerical artificial embedding

Chloé Mercier, Frédéric Alexandre, Thierry Viéville

► To cite this version:

Chloé Mercier, Frédéric Alexandre, Thierry Viéville. Ontology as manifold: towards symbolic and numerical artificial embedding. Workshop on VSA and hyperdimensional computing - Winter 2022 session, Luleå University of Technology, Sweden and UC Berkeley, USA, Jan 2022, Virtual, Sweden. hal-03550354

HAL Id: hal-03550354

https://inria.hal.science/hal-03550354

Submitted on 1 Feb 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Ontology as manifold:
towards symbolic and numeric artificial embedding

Chloé Mercier, Frédéric Alexandre, Thierry Viéville
## How to model human cognition with regard to...

Psychologically inspired
symbolic reasoning

Biologically plausible
spiking neural networks

We propose a geometric mapping of **ontologies** onto **neuronal manifolds**

<figure><img src="image_2.png" /></figure>

RDF(S): Resource Description Framework (Schema)
OWL: Web Ontology Language

<figure><img src="image_3.png" /></figure>

VSA: Vector Symbolic Architecture
SPA: Semantic Pointer Architecture
NEF: Neural Engineering Framework
# Vector Symbolic Architecture

* Each symbol encoded as a vector sampled from the unit hypersphere

* High-dimensional vector space => random vectors approximately orthogonal

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

The neural manifold hypothesis (Gallego et al, 2017)
# Vector Symbolic Architecture

* Each symbol encoded as a vector sampled from the unit hypersphere

* High-dimensional vector space => random vectors approximately orthogonal

* **Dot product** as a **similarity measure**:
compares the semantic meaning of 2 vectors.
# Vector Symbolic Architecture

* Each symbol encoded as a vector sampled from the unit hypersphere

* High-dimensional vector space => random vectors approximately orthogonal

* Dot product as a similarity measure:
compares the semantic meaning of 2 vectors.

<table><thead><tr><th>Closed-world reasoning</th><th>Open-world reasoning</th></tr></thead><tbody><tr><td>Anything that cannot be stated true is false</td><td>Anything might be true, unless proven false</td></tr><tr><td>τ ∈ {0,1}</td><td>τ ∈ {-1, 0, 1}</td></tr></tbody></table>
# RDF/RDFS: Resource Description Framework (Schema)

<figure><img src="image_11.png" /></figure>

* **RDF: Resource Description Framework**
  → syntax: (subject, predicate, object) triples

* **RDFS: RDF Schema**
  → metamodel of RDF, written in RDFS
  → allows to write (lightweight) **ontologies** and draw **inferences**
A triplestore using
associative memories
A Semantic Pointer triplestore

(subject, predicate, object), τ

vectors

S

p

o

scalar

<table><thead><tr><th>key</th><th>value</th></tr></thead><tbody><tr><td>S</td><td>τ B(o,p)</td></tr></tbody></table>
# A Semantic Pointer triplestore

(`subject<sub>1</sub>`, `predicate<sub>1</sub>`, `object<sub>1</sub>`), τ<sub>1</sub>

<table><thead><tr><th>key</th><th>value</th></tr></thead><tbody><tr><td>s<sub>1</sub></td><td>τ<sub>1</sub> B(o<sub>1</sub>, p<sub>1</sub>)</td></tr></tbody></table>
A Semantic Pointer triplestore

(subject<sub>1</sub>, predicate<sub>1</sub>, object<sub>1</sub>), τ<sub>1</sub>

(subject<sub>2</sub>, predicate<sub>2</sub>, object<sub>2</sub>), τ<sub>2</sub>

<table><thead><tr><th>key</th><th>value</th></tr></thead><tbody><tr><td>s<sub>1</sub></td><td>τ<sub>1</sub> B(o<sub>1</sub>,p<sub>1</sub>)</td></tr><tr><td>s<sub>2</sub></td><td>τ<sub>2</sub> B(o<sub>2</sub>,p<sub>2</sub>)</td></tr></tbody></table>
A Semantic Pointer triplestore

(subject<sub>1</sub>, predicate<sub>1</sub>, object<sub>1</sub>), τ<sub>1</sub>

(subject<sub>2</sub>, predicate<sub>2</sub>, object<sub>2</sub>), τ<sub>2</sub>

(subject<sub>2</sub>, predicate<sub>3</sub>, object<sub>3</sub>), τ<sub>3</sub>

<table><thead><tr><th>key</th><th>value</th></tr></thead><tbody><tr><td>s<sub>1</sub></td><td>→ τ<sub>1</sub> B(o<sub>1</sub>,p<sub>1</sub>)</td></tr><tr><td>s<sub>2</sub></td><td>→ τ<sub>2</sub> B(o<sub>2</sub>,p<sub>2</sub>) + τ<sub>3</sub> B(o<sub>3</sub>,p<sub>3</sub>)</td></tr></tbody></table>
# A Semantic Pointer triplestore

(subject<sub>1</sub>, predicate<sub>1</sub>, object<sub>1</sub>), τ<sub>1</sub>

(subject<sub>2</sub>, predicate<sub>2</sub>, object<sub>2</sub>), τ<sub>2</sub>

(subject<sub>2</sub>, predicate<sub>3</sub>, object<sub>3</sub>), τ<sub>3</sub>

Need to introduce:

- scalar multiplication a = τ a

- vector superposition c = a + b

- vector binding c = B(a,b)

## Semantic Pointer Architecture

(Eliasmith et al, 2013)

<table><thead><tr><th>key</th><th>value</th></tr></thead><tbody><tr><td>s<sub>1</sub></td><td>→ τ<sub>1</sub> B(o<sub>1</sub>, p<sub>1</sub>)</td></tr><tr><td>s<sub>2</sub></td><td>→ τ<sub>2</sub> B(o<sub>2</sub>, p<sub>2</sub>) + τ<sub>3</sub> B(o<sub>3</sub>, p<sub>3</sub>)</td></tr></tbody></table>

## Vector-derived Transformation Binding

(Gosmann and Eliasmith, 2019)

- Square dimensionality d with d'² = d

- Binding operation defined as:

$$ \mathcal{B}(\mathbf{x}, \mathbf{y}) = \mathbf{B}_{\mathbf{y}}\mathbf{x} = \begin{bmatrix} \mathbf{B}'_{\mathbf{y}} & 0 & 0 \\ 0 & \mathbf{B}'_{\mathbf{y}} & 0 \\ 0 & 0 & \mathbf{B}'_{\mathbf{y}} \end{bmatrix} \mathbf{x} \quad \text{where} \quad \mathbf{B}'_{\mathbf{y}} = d^{\frac{1}{4}} \begin{bmatrix} y_1 & y_2 & \cdots & y_{d'} \\ y_{d'+1} & y_{d'+2} & \cdots & y_{2d'} \\ \vdots & \vdots & \ddots & \vdots \\ y_{d-d'+1} & y_{d-d'+2} & \cdots & y_d \end{bmatrix} $$

- Non commutative, non associative

- Distributive and bilinear

- Admits right identity and right inverse
# A Semantic Pointer triplestore

(subject<sub>1</sub>, predicate<sub>1</sub>, object<sub>1</sub>), τ<sub>1</sub>

(subject<sub>2</sub>, predicate<sub>2</sub>, object<sub>2</sub>), τ<sub>2</sub>

(subject<sub>2</sub>, predicate<sub>3</sub>, object<sub>3</sub>), τ<sub>3</sub>

<table><thead><tr><th>key</th><th>value</th></tr></thead><tbody><tr><td>s<sub>1</sub></td><td>→ τ<sub>1</sub> B(o<sub>1</sub>,p<sub>1</sub>)</td></tr><tr><td>s<sub>2</sub></td><td>→ τ<sub>2</sub> B(o<sub>2</sub>,p<sub>2</sub>) + τ<sub>3</sub> B(o<sub>3</sub>,p<sub>3</sub>)</td></tr></tbody></table>

Need to introduce:

- scalar multiplication a = τ a

- vector superposition c = a + b

- vector binding c = B(a,b)

Semantic Pointer Architecture
(Eliasmith et al, 2013)

<figure><img src="image_12.png" /></figure>

NEF Associative Memory
(Voelker et al, 2014)

yellow circles: passthrough nodes

blue circles: neuron ensembles

orange connections: inhibitory

green connections: modulatory
# A Semantic Pointer triplestore

(subject<sub>1</sub>, predicate<sub>1</sub>, object<sub>1</sub>)

(subject<sub>2</sub>, predicate<sub>2</sub>, object<sub>2</sub>)

(subject<sub>2</sub>, predicate<sub>3</sub>, object<sub>3</sub>)

<table><thead><tr><th>key</th><th>value</th></tr></thead><tbody><tr><td>s<sub>1</sub></td><td>→ B(o<sub>1</sub>,p<sub>1</sub>)</td></tr><tr><td>s<sub>2</sub></td><td>→ B(o<sub>2</sub>,p<sub>2</sub>) + B(o<sub>3</sub>,p<sub>3</sub>)</td></tr></tbody></table>

Need to introduce:

- scalar multiplication $a = \tau a$

- vector superposition $c = a + b$

- vector binding $c = B(a,b)$

Semantic Pointer Architecture
(Eliasmith et al, 2013)

<figure><img src="image_13.png" /></figure>

NEF Associative Memory
(Voelker et al, 2014)

yellow circles: passthrough nodes

blue circles: neuron ensembles

orange connections: inhibitory

green connections: modulatory
# Example of inference

Inheritance inference rule:

X rdf:type C & C rdfs:subClassOf C' ⇒ X rdf:type C'

inspired by

https://protege.stanford.edu/ontologies/pizza/pizza.owl

<figure><img src="image_14.png" /></figure>
# Example of inference

Inheritance inference rule:

$$x \text{ rdf:type } C \ \& \ C \text{ rdfs:subClassOf } C' \Rightarrow x \text{ rdf:type } C'$$

Sufficient condition:

$$B_{type} \ B_{subclass} \rightarrow B_{type}$$

“inference rule” stored into a 2nd AM:

$$\text{type} \not\rightarrow \text{subclass} \rightarrow \text{type}$$

where x ≠ y is such that $B_{x \neq y} = B_x B_y$

inspired by
https://protege.stanford.edu/ontologies/pizza/pizza.owl

<figure><img src="image_15.png" /></figure>
CUE 1:
subject of interest

<figure><img src="image_16.png" /></figure>
CUE 1:
subject of interest

<figure><img src="image_17.png" /></figure>

CUE 2:
predicate of interest

<figure><img src="image_18.png" /></figure>
### CUE 1: subject of interest

<figure><img src="image_19.png" /></figure>

### CUE 2: predicate of interest

<figure><img src="image_20.png" /></figure>

### CUE 3: rule to apply

<figure><img src="image_21.png" /></figure>
### CUE 1: subject of interest

<figure><img src="image_22.png" /></figure>

### CUE 2: predicate of interest

<figure><img src="image_23.png" /></figure>

### CUE 3: rule to apply

<figure><img src="image_24.png" /></figure>
Simulation on Nengo

<figure><img src="image_25.png" /></figure>

<figure><img src="image_26.png" /></figure>

<figure><img src="image_27.png" /></figure>

Plotted here are the input
cue signals: accounting
respectively for the
subject, the predicate and
the rule to apply.
Simulation on Nengo

<figure><img src="image_28.png" /></figure>

Plotted here are the
outputs given by the
model: the object
associated to the given
subject and predicate, the
simplified predicate
obtained from the rule
(subClass * type -> type,
from the 2nd AM) and the
final statement inferred.

<figure><img src="image_29.png" /></figure>
Underlying mathematics
# VT unbinding

(Gosmann and Eliasmith, 2019)

* VTB admits right approximate inverses:

* the right approximate inverse $y$~ for $y$ is such that:

$$ \forall \mathbf{x}, \mathcal{B}(\mathcal{B}(\mathbf{x}, \mathbf{y}), \mathbf{y}^{\sim})) = \mathbf{B}_{\mathbf{y}^{\sim}} \mathbf{B}_{\mathbf{y}} \mathbf{x} = \mathbf{x} $$

* $\mathbf{B}_{\mathbf{y}}$ almost orthogonal: $\mathbf{B}_{\mathbf{y}^{\sim}} = \mathbf{B}_{\mathbf{y}}^{\top}$

=> just permute the elements of $y$ to get $y$~

* Right identity vector $\mathbf{i}_B$ such that $\forall \mathbf{x}, \mathcal{B}(\mathbf{x}, \mathbf{i}_B) = \mathbf{x}$ ie $\mathbf{B}_{\mathbf{i}_B} = \mathbf{I}$
# VT unbinding

What if we want to check if two resources are linked to each other by a predicate and, if so, retrieve the corresponding property (left unbinding)?

We can flip the order of the operands using a well-chosen matrix $\mathbf{B} \leftrightarrow$ : $\mathcal{B}(\mathbf{y}, \mathbf{p}) = \mathbf{B} \leftrightarrow \mathcal{B}(\mathbf{p}, \mathbf{y})$

=> Check if two resources are linked to each other by a predicate and, if so, retrieve the corresponding property:

- Flip the operands (multiply by $\mathbf{B} \leftrightarrow$ )

- Unbind y (multiply by the transpose of By)

- Check if the result is semantically close to a word from the vocabulary (compute the dot product)
Towards more reasoning power
# Beyond the type inheritance inference

* Covering all RDFS entailments reduced to the rule schema developed here, e.g.:

* property inference (e.g., if Bob is the son of Ali then Bob is of the same family as Ali)

* range inference (e.g., if Bob is the son of Ali, then Bob is a male)

* domain inference (e.g., if Bob is the son of Ali, then Ali is a human)

etc...

* To what extent could it be applicable to description logics and OWL entailment rules?

* SWRL rules (i.e., inference rules expressed directly as rewriting rules on variables) stored in associative memories?
# All RDFS entailments

* Everything is a resource

* IF $x$ $p$ $y$ THEN

* $x$ $rdf:type$ $rdfs:Resource$

* $y$ $rdf:type$ $rdfs:Resource$

* Any class is a subclass of $rdfs:Resource$

* IF $c$ $rdf:type$ $rdfs:Class$ THEN

* $c$ $rdfs:subClassOf$ $rdfs:Resource$

* Type propagation

* IF $c2$ $rdfs:subClassOf$ $c1$

* AND $x$ $rdf:type$ $c2$

* THEN $x$ $rdf:type$ $c1$

* Subsumption reflexivity

* IF $c$ $rdf:type$ $rdfs:Class$

* THEN $c$ $rdfs:subClassOf$ $c$

* Subsumption transitivity

* IF $c2$ $rdfs:subClassOf$ $c1$

* AND $c3$ $rdfs:subClassOf$ $c3$

* THEN $c3$ $rdfs:subClassOf$ $c1$

* Property propagation

* IF $p2$ $rdfs:subPropertyOf$ $p1$

* AND $x$ $p2$ $y$

* THEN $x$ $p1$ $y$

* Property subsumption reflexivity

* IF $p$ $rdf:type$ $rdf:Property$

* THEN $p$ $rdfs:subPropertyOf$ $p$

* Property subsumption transitivity

* IF $p2$ $rdfs:subPropertyOf$ $p1$

* AND $p3$ $rdfs:subPropertyOf$ $p2$

* THEN $p3$ $rdfs:subPropertyOf$ $p1$

* Type inference (domain)

* IF $p$ $rdfs:domain$ $d$ AND $x$ $p$ $y$

* THEN $x$ $rdf:type$ $d$

* Type inference (range)

* IF $p$ $rdfs:range$ $r$ AND $x$ $p$ $y$

* THEN $y$ $rdf:type$ $r$
# Type inheritance

inspired by
https://protege.stanford.edu/ontologies/pizza/pizza.owl

$$x rdf:type C & C rdfs:subClassOf C' \Rightarrow x rdf:type C'$$

Sufficient condition:

$$B_{type} B_{subclass} \rightarrow B_{type}$$

“inference rule” stored into a 2nd AM:

$$type \not= subclass \rightarrow type$$

where x $\not= y$ is such that $B_{x \not= y} = B_x B_y$

<figure><img src="image_30.png" /></figure>
## Subsumption transitivity

$$ 
\begin{array}{l}
C \ rdfs:subClassOf \ C' \ \& \ C' \ rdfs:subClassOf \ C'' \\
\Rightarrow C \ rdfs:subClassOf \ C''
\end{array}
$$

Stored in 2^nd^ AM as:

subClassOf Æ subClassOf → subClassOf

<figure><img src="image_31.png" /></figure>
Domain and range inferences

p rdfs:domain C & x p y

⇒ X rdf:type C

p rdfs:range C & x p y

⇒ y rdf:type C

<figure><img src="image_32.png" /></figure>
Domain and range inferences

p rdfs:domain C & x p y
⇒ X rdf:type C

p rdfs:range C & x p y
⇒ y rdf:type C

We introduce an UNKNOWN vector Υ
(accounting for OWA) such that:

B<sub>Bdomain</sub>c Υ ⇒ B<sub>type</sub>c

AM1 p → B<sub>domain</sub>c

x → B<sub>p</sub> Υ

AM2 x → B<sub>type</sub>c

<figure><img src="image_33.png" /></figure>

(same with range inference)
## Property propagation

$$ 
\begin{array}{l}
p \ rdfs:subPropertyOf \ q \quad & \text{&} \quad \text{x} \ \text{p} \ \text{y} \\
\Rightarrow \quad \text{x} \ \text{q} \ \text{y}
\end{array}
$$

Using our previous architecture, this kind of inference is only possible when

$$ \mathbf{subPropertyOf} = i_B $$

Is this consistent?

Why not do the same for all "is_a" relationships (subClassOf, type) ?...
=> ongoing work

<figure><img src="image_34.png" /></figure>

<figure><img src="image_35.png" /></figure>
# Property composition

Rules such as

`eats Ø hasTopping → eats`

to infer relational compositions

(not part of RDFS entailments, but
similar to OWL *property chains*)

<figure><img src="image_36.png" /></figure>
# OWL: another representation for knowledge

## Web Ontology Language

semantics based on description logics
=> fragment of 1st-order logic

more reasoning capabilities,
computability and decidability
under constraints

<figure><img src="image_37.png" /></figure>

<figure><img src="image_38.png" /></figure>
# OWL + SWRL

## SWRL: a Semantic Web Rule Language combining OWL and RuleML

Completes OWL with implication rules (written with variables, e.g. ?x):

hasParent(?x1,?x2) ∧ hasBrother(?x2,?x3)

⇒ hasUncle(?x1,?x3)

<figure><img src="image_39.png" /></figure>
Alternate implementation:
a compressed triplestore vector
Note that for an efficient retrieval, when storing triples, we can index them either by subject, object or property:

$$^{po}_s \mathcal{B}(p, y) = \mathcal{B}(p, y) = \mathcal{B}_y p \quad \text{(indexing triples by subject)}$$

$$^{so}_p \mathcal{B}(x, y) = \mathcal{B}_y^T x = \mathcal{B}_y \sim x \quad \text{(indexing triples by property)}$$

$$^{sp}_o \mathcal{B}(x, p) = \mathcal{B}_p^T x \sim = \mathcal{B}_p \sim x \quad \text{(indexing triples by object)}$$

$$^{op}_s \mathcal{B}(y, p) = \mathcal{B}(p, y) = \mathcal{B}_y p$$

$$^{os}_p \mathcal{B}(y, x) = \mathcal{B}_y^T x = \mathcal{B}_y \sim x$$

$$^{ps}_o \mathcal{B}(p, x) = \mathcal{B}_p^T x \sim = \mathcal{B}_p \sim x \sim$$

B is a bilinear form.
The class inheritance example:

($s rdf:type $c1.) and ($c1 rdfs:subClassOf $c2.) $s rdf:type $c2.)

Open (new) triple box

Enumerate the
matched triples

Iterate on new triples

Closed (known) triple box

input A new triple ($s_0 $p_0 $o_0.) and a closed set of triples (($s_i $p_i $o_i.)...).

let (($s_0 $p_0 $o_0.)) an “open” triple set, initialized with the new triple inside.

repeat

pull a triple ($s_0 $p_0 $o_0.) from the open triple set.

if $p_0 = rdf:type then

for all ($s_i $p_i $o_i.), $p_i = rdfs:subClassOf & $s_i = $o_0, in the closed triple set do

add ($s_0 rdf:type $o_i.) to the open triple set.

end for

else if $p_0 = rdf:subClassOf then

for all ($s_i $p_i $o_i.), $p_i = rdf:type & $o_i = $s_0, in the closed triple set do

add ($s_i rdf:type $o_0.) to the open triple set.

end for

end if

add the triple ($s_0 $p_0 $o_0.) to the closed triple set.

until the open triple set is empty
The class inheritance example:

- The rule corresponds to a hardwiring of a 2 input 1 output operator

- Approximate equality (i.e. similarity) is intrinsic to the calculus

- Approximately true property is intrinsic to the calculus

- The ‘&’ operator can be implemented as a product (but not only).

<figure><img src="image_40.png" /></figure>
The class inheritance example:

- The rule corresponds to a hardwiring
of a 2 input 1 output operator

- Approximate equality (i.e. similarity)
is intrinsic to the calculus

- Approximately true property
is intrinsic to the calculus

- The ‘&’ operator can be implemented
as a product (but not only).

<figure><img src="image_41.png" /></figure>

- For “negative” inferences (i.e. $\lambda < 0$) a different (symmetric) rule is required :

($s \neg rdf:type $c_1$) and ($c_2 rdfs:subClassOf $c_2$) $\Rightarrow$ ($s \neg rdf:type $c_2$.)
Accounting for uncertainty:
possibility, necessity
Linking with possibility π / necessity v theory

$$0 \leq \nu(E) \leq p(E) \leq \pi(E) \leq 1$$

$$\tau = \pi + \nu - 1 \in [-1, 1]$$

$$\rho = \nu - \pi + 1 \in [0, 1]$$

A new interpretation of
**necessity** and **possibility**
as probability bounds

<figure><img src="image_42.png" /></figure>
Linking with possibility $\pi$ / necessity v theory

Given a set of a-priori knowledge $\mathcal{K}$ and observation $O$, $(\pi, \nu)$ is properly defined as:

$$ \nu(E) = \min_{\kappa \in \mathcal{K}} P_{\kappa}(E|O) $$

$$ \pi(E) = \max_{\kappa \in \mathcal{K}} P_{\kappa}(E|O) $$

Given a possibility/necessity distribution $(\pi, \nu)$ we can define

* a probability distribution of maximal entropy

* for “minimally corrected” compatible distributions $\tilde{\nu} \le \nu, \pi \le \tilde{\pi}$
# Linking with possibility π / necessity v theory

Compatibility with Boolean and three-value logic calculus can be derived

Partially obtained by distribution monotonicity

$$E \subseteq E' \Rightarrow \nu(E) \leq \nu(E') \text{ and } \pi(E) \leq \pi(E')$$

Fully compatible in the “deterministic case” (vanilla theory)

Requires additional constraints in the 2D case for 0 or ± 1 values

In an open world (i.e. if we do not consider an exhaustive universe),
logical derivation related to the the complement of an event are not possible.

The correct model underneath is modal logic.
# Linking with possibility $\pi /$ necessity v theory

In VSA ($\pi$, $\nu$) could be implemented either as a **pair of real numbers**, or as a **complex number**.

* Similarity can be defined as the dot product $Re(xy^*)$ in the hermitian space $C^d$ with the same interpretation

* VTB algebra and related operators seem generalizable

However there is no real “gain” to consider complex number apart from using the complex product instead of 2d matrix product.
# Conclusion/Perspectives

* Distributive, tightly coupled approach, drawing inspiration from biology and psychology

* 2 associative memories:

* AM1 to store the ontological "facts"

* AM2 to store the inference rules => used to disclose implicit relationships/memberships

* Alternate implementation: compressed triplestore (computing the transitive closure)

* Achievable inferences:

* RDFS entailments:

* concept inclusion an inheritance

* property domains and ranges

* extension to OWL ontologies: class complement, intersection, etc

* Scalability, completeness, soundness on larger databases?

* How to manage uncertainty? (possibility-necessity approach?)

* How to update the associative memory to add the inferred relationships?

* How to learn the inference rules from patterns discovered in AM1 ?

Nengo associative memory is read-only (though some work is aiming to implement learning within AMs, cf Voelker et al 2014)