NATURAL LANGUAGE UNDERSTANDING USING BRAIN-LIKE APPROACH: SEMANTIC ENGINE USING BRAIN-LIKE APPROACH (SEBLA) DERIVES SEMANTICS OF WORDS AND SENTENCES

Natural Language Understanding (NLU) is a complex open problem. NLU complexity is mainly related to semantics: abstraction, representation, real meaning, and computational complexity. While existing approaches can solve some specific problems, they do not address Natural Language problems in a natural way. This invention describes a Semantic Engine using Brain-Like approach (SEBLA) that uses Brain-Like algorithms to solve the key NLU problem (semantics and its sub-problems). The main theme of SEBLA is to use each word as an object with all important features, most importantly the semantics. The next main theme is to use the semantics of each word to derive the meaning of a sentence as we do as humans. Similarly, the semantics of sentences are used to derive the meaning of a paragraph. The 3rd main theme is to use natural semantics as opposed to existing “mechanical semantics” used in Predicate logic, Ontology or the like.

DETAILED DESCRIPTION OF THE INVENTION

The “recall” phase can also have the on-line training as a continuous learning process, mainly for refinement.FIG. 1(a) shows the training phase. An input sentence (or words) is provided to both SEBLA and Semantic database (SD). SD is formed by taking a natural language sentence corpora and defining semantics by some Natural Language experts. The format of the semantics may vary. Since SEBLA derives semantics of a sentence using semantics of words and natural language grammar, ideally a relatively small SD is needed. In some cases—e.g. for small vocabulary applications, an SD may not be needed at all. So, SD is mainly used for refinement of SEBLA semantics. Since a natural language corpora may have trillions of sentences (e.g. Google Trillion Sentence database [NY2010]), manually creating Semantics for all sentences in such a large corpora is a daunting task. Hence, this invention recommends to use a small size SD to ensure that SEBLA has a good start and then use just SEBLA itself and (when needed) an SD for refinement. However, instead of manually creating a large SD, all sentences for which SEBLA correctly derived semantics during its Recall phase over time, can be automatically added to the SD. This can be done by properly monitoring SEBLA's performance on a continuous basis.

For any incorrect performance of SEBLA, it will try to re-train itself. Retraining includes fine tuning of the semantics of the words as appropriate. A large language corpora is still useful in three major ways: to refine and enhance the World Knowledge (WK), to refine and enhance Function words of a word, and to help better use of the grammar of the language.

The Machine Learning (Ml) box shown inFIG. 1(a) andFIG. 1(b) can use almost any machine learning algorithm as appropriate.

The main theme of our approach is to use each word as object with all important features, most importantly the semantics. In our human natural language based communication, we understand the meaning of every word even when it is standalone without any context. Sometimes a word may have multiple meanings which get resolved with the context in a sentence. The next main theme is to use the semantics of each word to develop the meaning of a sentence as we do in our natural language understanding as humans.

At the word level, the key question we have addressed is how to represent the semantics for each word and how to associate appropriate world knowledge with each word. By using the representation and semantic feature of each word, along with the world knowledge associated with each word, the meaning of a sentence is derived by applying the grammar of the language and appropriate rules to combine words. Key features of the words and appropriate rules to combine them are learned/refined using large text corpora and machine learning algorithms. The inference engine (Intelligent Agent) will determine the meaning of a sentence by using the word semantics and appropriate rules to combine the words in a sentence.FIG. 2shows SEBLA's key blocks. Before describing the Intelligent Information Retrieval (IIR) system using SEBLA (FIG. 1(b)), we describe details of SEBLA.

The input text (can also be generated by a Speech Recognition Engine, ASR) is parsed into word categories like noun, verb, pronoun, adjective, etc. This is done by the block named Parsing the input Text. The parsed words are then further refined and checked by grammar rules using the block Grammar Checking/Reformatting. Unnecessary words like “the”, “a” are removed as they do not carry much value for the “core” semantic meaning

The RFW (Retrieve Function words and World Knowledge (WK) words for each word in the sentence) block retrieves 2 types of words for each word in the refined sentence. These words are retrieved form libraries of Function Words and WK words. A sample of such word libraries/tables are shown inFIG. 3.

The output of RFW is then processed by CSS (Calculate Semantics of the Sentence) block. The algorithm for such semantic calculation is described below.

Algorithm to Calculate Semantics:

A. Sentence Level

We consider each parsed word in the refined sentence. For each of such words,a) Get the Function wordsb) Get the World Knowledge (WK) words

E.g. for the word “ball”, the function words are {ball, move, roll, round, play.} as shown inFIG. 3(a).

Similarly, the function word for “go” is {go, move, not static, . . . } and the function word for “school” is {school, study, student, teacher, learn}. We can add other related words which are usually implied—e.g. for “school”, “a place to” {study}, a place where {students} go etc. But in general a short list of function words suffices and makes it simpler. Note that the word itself is included in its Function word. This can be done in the WK and thus, we may not include the word itself in its Function word.

Now let's consider the sentence,“I go to school”. For semantic retrieval, we will use “I go school” as simplifying the verb phrase (VP) “go to school” to “go school” helps. This is a “declarative” type sentence. From language standpoint, the word “I” (noun Phrase, NP) is the subject. “go” is the verb which is part of VP “go school” where “school” is a noun. Now, we need to apply the function words to calculate the semantics of the first two words i.e. “I go”. In doing so, we first take the Function words for “I” and Function words for “go”. Thus, we have,I {person, he, she, living object, . . . |eat, go, fly, all verbs} go {move, walk, run, . . . }. Then, we take only the subject words for “I” and verb words for “go”, which yields the following semantics,

Note that the main words are there to visualize it better. The real semantics is represented by all words under the curly braces { }.

Consider another sentence,“I open door”, the semantics of which is

We can now ask a question like“I am doing what”. The semantics of this sentence is

A match operation between equations (3) and (4) will yield

Note: For other words (i.e. without an auxiliary verb) that imply open (e.g. ajar), the answer will be same as “ajar” would be included in the Function word of “open”. In this case, we do not need to use the WK but it won't hurt even if it is in the WK.

Equation (5) can then be processed to yield the answer“I am opening the door”, after some refinement using grammar.

To better explain this, let's use an invalid sentence, e.g.“Door walks” which is not valid as in the Function words for “door”, “walk” is not there. Besides, “door” is not a living thing and hence it will not be supported by the WK (further explained below under “World Knowledge” below). So, the semantics of this would be NULL or a question mark “?”.

Another example will make this a bit more clear. Consider the following:

Now, if we ask

“is vegetarian dishes served at Maharani?”  (7)the system will not be able to answer correctly unless we also define a semantics for “Vegetarian Dish” or define that “food” is same as “dish” etc. This means, almost everything would need to be clearly defined (which is what is best described by “mechanical semantics”). But with SEBLA based NLU, the answer for the above question will be “Yes” without adding any special semantics for “Vegetarian Dish”. The “mechanical semantics” nature becomes more prominent when we use more complex predicates e.g. when we use universal and existential quantifiers, and/or add constructs to represent time.

Our SEBLA based approach shown above for “declarative” sentences will work in a similar way for other types of English (and other languages) sentences including“imperative”, “yes/no”, “wh-structure”, and “wh-nonsubject-structure”.

NOTE:It is important to note that ML (Maximum Likelihood) based performance commonly used in prediction (e.g. when one types words in a search field on a search engine it shows the next word(s) automatically) will be improved with natural semantics. Currently, mainly ML (and sometimes other techniques including existing semantics methods) is used for prediction. By using proposed more natural semantics (e.g. using SEBLA), the meaning of the typed words will be more clear; thus helping better prediction of the next word(s). It will also help using natural sentences in the search field than special word combinations, e.g. when using advanced search.

However, for more complex cases, we will have issues like “which word(s) of the Function words to take in calculating semantics”. For this we use Membership function of the Function words so that word(s) with the highest membership function will be picked (FIG. 3(a)). Then we use WK words to further refine the semantics and reject function words not very appropriate. For some cases, words with highest membership value may not provide optimal semantics as the WK words may dominate and Function words with lower membership value may produce optimal semantics.

The membership values can be refined using large Language corpora as well as semantics corpora. This is also true for WK i.e. it can be refined/enhanced by learning via language and/or semantics corpora. In general, WK is in the WK table. E.g. person, he, she, etc. are in general live people. But some of them might be dead. In such cases, there need to be facts about that either in the Fact database or WK database. So, if John is alive, then“John plays ball” can be valid. WK knows that all live people are associated with all verbs etc.

More Complex Sentences:

SEBLA based approach also works with more complex sentences. Consider the sentence,

“I am trying [VP (Verb Phrase) to find a flight that goes from Pittsburgh to Denver after 2 pm”  (8)

Here, the basic idea is to use the sentence starting at top level and classify it as having a Noun Phrase (NP) and Verb Phrase (VP). Then, deal with the complexity of the VP using similar way as described above. So, the first level semantics isI {person} trying {doing something, working on,} to find {looking, trying to look, . . . },as the main verb of the VP “find a flight that goes from Pittsburgh to Denver after 2 pm” is “find” or “to find”.

Now, we can focus on “a flight that goes from Pittsburgh to Denver after 2 pm”.

This reduces to “flight goes” as the rest i.e. “from Pittsburgh to Denver after 2 pm” is from a city to another city after “time” 2 pm. The semantics for the words before the cities is

The semantics of the rest of the sentence is sort of constant up to the “time 2:00 pm”. The WK can be used to handle the time if a question related to time is asked. Now, if we ask“what the person is trying to do”,the answer will be “the person is trying to find a flight” and then add “from Pittsburgh to Denver after 2 pm”. This is because semantics of the first few words in the question will match/partially match (with high confidence) with the fact. The semantics of the last word “do” in the question, will be checked against the word “to find” in the fact sentence. By WK, “do” goes with almost all verbs. Hence, it will match with “find” or “to find”.

But if the question is“the person is trying to find a flight from which city to which city”, then the system will look into “from” and “to”, as the semantics of the question up to which city will match with the semantics of the sentence up to “from” etc. as explained above.

In case there are multiple similar facts and hence, possible, multiple matches, we can have various approaches to resolve that:a. Provide all possible answers for which match is high.b. Ask user more questions to help determine the best answer.c. Use the discourse or relationships of the semantics of some previous words or some words after the key word(s) or previous/later sentences (see case B below).d. Some other approaches.

In summary, the existing “syntactic parsing” techniques can be used for parsing as needed BUT the Function words will help the missing gap of existing “semantic parsing” that uses “mechanical semantics”. The key challenge is to properly defining and learning/refining the Function words of a word. Function words inherently describe the meaning or semantics and thus avoid separately defining semantics using existing approaches like Predicate Logic or Ontology.

Algorithm to Calculate Semantics:

B. Paragraph Level

Similar algorithms can be used in calculating semantics for multiple sentences and paragraphs. However, some modifications are needed for the following reasons:1. Within a sentence, words are used in a constrained way using grammar. But between sentences there is no such grammar.2. Usually, a group of sentences carry a theme within a context and there are relations between sentences.

Thus, to calculate the semantics between sentences, we will use word semantics as before BUT with some modifications. This is also true for a single long sentence segmented by comma, semicolon, “but”, “as” and the like. We also need to take account for “discourse” i.e. coherence or co-reference to words in previous sentences. There are some good existing solutions mainly for a small domain problem. But, in general Computational Discourse (CD) in natural language is an unsolved problem. However, with our SEBLA based scheme, the CD problem can be solved to a good extent for large domains.

In calculating semantics in a long sentence, the previous, next and other words can further influence/refine the semantics. For convenience, we have included this aspect in calculating semantics of multiple sentences.

First, let's consider only 2 sentences. As before, we will generate semantics for sentence 1 (S1) and sentence 2 (S2). But instead of finding a match to get an answer (as shown above for Q & A), we need to do a match to see any relationship between S1 and S2. If there is some relationship it will help CD which in turn will help summarization or drawing inference. Any possible relationship between S1 and S2 is calculated by extracting the core semantics of both sentences. Consider the following 2 sentences:

The semantics of the sentences areJohn {person} hid {hide, remove, putting in secret place, doing not a good thing, doing a bad thing, . . . }.He {person} drunk {drunk, abnormal, under influence, bad, doing a bad thing, not doing a good thing, not in good state . . . }

Here, via matching, “he” is related to John (co-reference), and “hid” is related to “drunk”. So, S2 sort of explains the action in S1 (coherence). If we are looking for just coherence, we can take the core semantics of S1 and S2. The core semantic words of S1 is {hid keys} and the same for S2 is {was drunk} or just {drunk}. If we use the Function words for these word pairs, “drunk” will match with “hid” as shown above, and thus, the system will find the relationship between S1 and S2. The core semantic words are basically verb (action) and object of the action.FIG. 4shows key steps in calculating semantics of multiple sentences and paragraphs.

Now let's consider semantics of a paragraph. Consider the following paragraph:

TheIntelligent Internet (IINT) will take the Internet to a new level (S1). It will allow existing as well as significant number of new users to enjoy the existing and various new benefits of the Internet (S2). IINT will affect their lives in a positive way with Economic, Social, Cultural and other developments globally (S3)  (11)

The core semantics of each sentences are as follows:

Using the Function words in (12), we see relation between “internet” & “enjoy” (12a and 12b) and “internet” & “lives” (12a and 12c) and “enjoy” & “lives” (12b and 12c). Thus, the core semantics of these sentences can be represented as

Now, we can derive the summary by using these core semantic words, matching with input sentences and compressing them, yielding

The Intelligent Internet (IINT) will allow existing as well as significant number of new users to enjoy the existing and various new benefits of the Internet. IINT will affect their lives in a positive way with Economic, Social, Cultural and other developments globally  (14)

The first sentence is dropped as the action word “take” did not match with any similar words and hence not in (13). However, the first sentence can be added if evaluation (see below) provides a low score to the “summary” (i.e. for cases where dropping a sentence(s) may lower the score too much. In general, a sentence not having any action does not belong to the “summary”.

This may not be the best summary. In general, summarization is an iterative process—take minimum words for action(s) and associated object(s) and find the core semantics. Then calculate the summary. Then evaluate the summary using some evaluation techniques (including existing standard evaluation techniques e.g. ROGUE, Pyramid Method). If the evaluation score is low, relax the “action words” i.e. take more action words and associated objects and repeat the process. E.g. if we take both action words “allow” and “enjoy” in S2, then “allow” will match with “take” in S1. Then we will have S1 in the summary etc. Additionally, machine learning (ML) can be used to further improve the summarization. Besides, simple reduction can also be used e.g. “as well as” in S1 can be replaced with “and”.

The same process can be applied to more sentences i.e. multiple paragraphs as paragraphs are joining of multiple sentences. If there is no good semantics between two paragraphs, then those two paragraphs are talking about different things and cannot be summarized i.e. we need to keep them as is or use higher level semantics (e.g. in drawing inferences).

SEBLA can help all key tasks in Summarization including general summarization, question specific summarization, and creating abstracts. SEBLA can also be used for many other NLP applications e.g. Information Extraction (including temporal and event processing), Question & Answer (including factoid & more general), Drawing Inference, Machine Translation, Conversational System and more.

Note that existing schemes for most of such applications can also be used in combination with the proposed methods to further refine/enhance as appropriate.

To further explain the concept, we have described an Intelligent Information Retrieval (IIR) system inFIG. 1(b) using SEBLA. It is important to note that an Intelligent Agent is used along with the SEBLA based NLU to perform all the related tasks, namely, deriving the semantics of the words in a query using Brain-Like approach, deriving the semantics of the query sentence, understanding the query sentence, taking appropriate action based on the understanding of the query sentence, accessing all relevant desired information and thena. (for Information Retrieval and Q & A) filtering unrelated information from the retrieved information, assembling filtered retrieved information and presenting the assembled information in a succinct and logical way to the user. For Q & A, the information presented will be the answer i.e. very succinct. In case of IIR, it can be a small set of relevant information.b. (for summarization) filtering unrelated information from the retrieved information, assembling filtered retrieved information and creating a good summary.c. (for drawing inference) summarization as well as adding sentences to express the inference based on the retrieved content (e.g. in case of Business Intelligence).

[Note: An Intelligent is needed for other similar applications like Q&A, Language Translation, and Conversational System].

The information retrieval through existing IR and search engines are mainly based on string search. Thus, the search process needs to deal with many data to find matches. And all matched data are extracted even though many data are not relevant and desired. Accordingly, such engines produce many (often thousands of) results, and human knowledge and intelligence are needed to retrieve the desired information from such search results. This requirement usually limits the usage of search engines to experienced and educated users. There are FOUR key issues with the current approaches:a. Search process needs to deal with very large data.b. String search results contain many undesired and unrelated results.c. String search results may not contain the desired results and user may need to do multiple searches by various search word combinations.d. String search results may NOT contain the desired information even after trying major key word combinations as a user may skip key words of similar meaning.

The semantic capability of SEBLA addresses these issues in TWO broad ways:A. Retrieve expanded and more related information and then get most desired information by filtering.B. Retrieve far less but more related and appropriate information and then get more refined desired information.

Approach in A (FIG. 1(b)) is useful when string search data is not too large and conventional search engines can be used. The key steps using approach A are:1. In the query sentence/string to understand the meaning of each word and sentence.2. Generate all related sets of query strings using semantic meaning of each word and sentence (thus generating lot more appropriate search results that are related to the input words and sentences).3. Extract the most appropriate and related results from the extended search results. This is achieved by employing the semantics and rendering (FIG. 1(c)).

Many words have multiple synonyms. By understanding the semantics of each word, a complete (or nearly complete) set of synonyms will be generated. Without semantic meaning, only limited predefined synonyms can be used as done for some words in existing search engines. This is also true for sentences. By understanding each search sentence, corresponding equivalent search sentences and corresponding words will be generated. The sentence level semantics will be used to refine the word list to help reduce search results when submitted to search engines.

If a user only presents search key words and no sentences, then, using NLU, a set of most relevant search words will be generated. This is be done by creating different word combinations (including all synonyms), deriving the semantic meanings, and then appropriate filtering to derive the most appropriate set of search word combinations.

If a user presents sentences, then similar sentences using synonym words will be generated by keeping the context same. Then a corresponding set of key words will be generated. This is important as existing search engines mainly work on string search and do not depend on the meaning of the sentences or words. However, they do strongly consider word combinations.

Approach in B is useful when string search data are very large and conventional search engines can take too long. This approach is more appropriate for Big Data. The key steps using this approach are:1. In the query sentence/string to understand the meaning of each word and sentence.2. Calculate the semantics of each title/indexed item (targets from the standard string based match with the query but before retrieval of the associated content) and calculate semantic matching or overlap of the query with each target. Then select the target (s) with high semantic matching. In this case a new search method using semantic matching (instead of string matching) will be needed. Searching with semantic meaning will retrieve very appropriate and much less information.3. Extract the most appropriate and related results from the search results in step #2.NOTES:(a) Approach A and B can also be combined to get more appropriate results for some applications.(b) Search algorithm of existing search engines may be modified to retrieve not all string matched content but retrieve contents only with high semantic match after doing a semantic match with the potential targets.