NLU and Natural Language Interfaces to Databases
By: Walid Saba
Tags: Naturallanguageprocessing, Nlu, Deep Learning, Querying, Machine Learning

The Plot

What is involved in getting (from some database) the correct answer to this natural language (NL) query (imagine a C-level executive querying their database on their mobile):

Do we have a retired BBC reporter that was based in an East European country during the Cold War?

Clearly, there is one correct answer to this question — this is not a retrieval problem where similarity of word embeddings might make few matches on attributes and relations in the database (or a KnowledgeGraph). The NL query above must be converted into an unambiguous SQL query (or, SPARQL, or whatever…) Of course, theoretically, there is an infinite set of queries that are syntactically different, but ‘semantically’ similar, but the semantics (meaning) is what matters and in the end we are talking about a single query that should retrieve the one and only one correct answer. This means that we cannot misinterpret anything, not even the function words, and we cannot, of course, not pay attention to the structure and how the various constituents are composed.

Let us see the details of what needs to be done to get the right answer.

Quantifier Scope

Although we have the phrase ‘a retired BBC reporter’ we are seeking all/any ‘retired BBC reporters’ and not a specific one. Similarly for ‘an’ in ‘an East European country’. So (1) should be interpreted as saying:

Do we have any retired BBC reporter that was based in any East European country during the Cold War?

This decision is made by relying on some knowledge of the relationship between reporters and countries they might be based in.

Nominal Compounds

The meaning of the nominal ‘retired BBC reporter’ refers to, in the end, a set of reporters that worked for BBC and are retired. Roughly, a set like this:

In this case computing the meaning of the nominal was relatively easy. But suppose we had the nominal ‘former BBC reporter’. In this case, the adjective ‘retired’ was replaced by an intensional adjective that has no meaning on its own — unlike retired, which could be approximated by the set of all those that are retired (for subsequent intersection with the set of reporters, etc.) ‘former’ does not have a meaning on its own (there is no set of ‘former’ things!). Instead, ‘former’ is an intensional adjective and thus is contextual — it is a function that requires a nominal (a concept), to be computed. In other words, ‘former’ alone does not refer, but ‘former reporter’ does and clearly this function has some temporal aspect. Note also that unlike ‘retired’, ‘former’ does not imply not working for someone other than BBC. Roughly, then, ‘former BBC reporter’ can be approximated by

Basically, ‘former BBC reporters’ refers to the set of reporters, that also worked for BBC for a certain period, and that period is before ‘now’.

Prepositional Phrase Attachments

The (prepositional) phrase ‘in an East European country’ attaches to (read ‘modifies’) the nominal ‘that was based in’ and not ‘a retired BBC reporter’. To appreciate this, just contemplate the situation if ‘that was based in’ was replaced by ‘that reported on’ where in this case the prepositional phrase modifies the reporter! Thus, the correct decision as to what a prepositional phrase attaches to requires knowing how the world of reporters and countries they ‘report on’ or are ‘based in’ works. The phrase ‘during the Cold War’ attaches to ‘that was based in’ and not to ‘an East European country’ . Furthermore, this additional modification ends-up being another ‘filter’ on the set of reporters that were based in an East European country (think WHERE clause in SQL). In the end, and going up the tree, we get this for ‘retired reporter that was based in an East European country during the Cold War’:

So we now have the set of all those reporters (in our database ) that

- are retired

- reported for BBC

- were based in an East European country

- (were based in an East European country) during the cold war

After all the compositional semantics are done, either we get a set that if empty, the answer is no, otherwise all the elements in the set are valid answers.

Regardless of the Clumsy Semantics …

The very trivial compositional semantics I suggested above is not the issue — actually the above was a rough sketch of what needs to be done, not how. But regardless of the (toy) semantics, the point I was making is that the NL query must result in an unambiguous formal (logical) query and so all the components must be properly assembled (composed) from the ground up. No room for a mistake, since, again, there is one and only one correct answer. If you can do this in a non-symbolic paradigm, and without variables, scope and variable binding, and without compositional semantics that is driven by the syntactic structure, then by all means. But understanding a query and fetching the right answer is not a retrieval problem where no answer is provided but a set of answers that are ‘probably approximately correct’ (PAC) (which, ironically also stands for ‘probably approximately correct’ in machine learning theory)

Not in NLU. The answer that we require in NLU is not a set of ‘probably approximately correct’ answers but the NL query must be interpreted and translated into the one logical query that will fetch the correct answer.

I have not covered in the above exposition many other complex scenarios but I think the example we walked through demonstrates why there is, after 60 years of research in NLU, no system that can translate NL queries into the correct database query. When such an engine is built, it will turn the world of information technology upside down. It will be the new operating system (how’d you like to tell your machine ‘please download and install the latest Chrome version for Windows 10’ or ‘where is the PowerPoint presentation that I edited last week and has references to the PreSearch website?).

Some Concluding Remarks

I’m always puzzled when I hear AI researchers (who, for all intents and purposes are brilliant scientists/engineers) make statements that not only are unscientific and irrational, but outright silly. For example, one of the idols of Deep Learning (DL) enthusiasts recently proclaimed that “Deep Learning is going to be able to do everything.” Does that include learning a compiler like Python by just giving the networks sample programs that are labeled 1 and 0 — depending on whether they are valid programs or not? I mention this because that is another case where no approximations can be tolerated and the answer is not fuzzy — no softmax and no probabilities. Either the program is a valid program or it is not (we cannot run a program where somewhere in the program it is not clear what to do next!).

And even if we succeeded in “learning” how to unambiguously check the syntax of an infinite combination of programs that could be written, would a deep neural network also be able to execute programs — and without symbolic processing (symbol tables, tables that keep track of nested functions calls, along with their local variables, types, etc.)? Everyone with basic knowledge of computing science theory — and computability in general, knows this is a joke.

The moral of the story is this: we should all heed the wisdom of Albert Einstein and Aristotle, who both said (using different words): “The more I learn, the more I realize how much I don’t know”. Those that have not worked in Computational NLU and have not built the algorithms and structures required to keep track of scope and variable binding, performing the correct reference resolutions, PP attachments, scope resolution, etc. not to mention the required typed ontologies and knowledge structures needed to do several required inferences, do not appreciate all the complexities involved in discovering the one and only one thought that some utterance or question is intended to convey.

___

ONTOLOGIK — Medium