An explainable topological search engine with Giotto-tda | by Matteo Caorsi | Towards Data Science

The co-author of the blogpost: F. Cornell.
In this blog post, based on the work by F. Cornell, we explain how one can use Giotto-tda [2] to cluster words and create a semantical search engine, retrieving documents similar to one's query, thereby showing how we can use topological techniques in the context of Natural Language Processing (NLP).
We will be using the classical Reuters dataset, consisting of around 10K news documents collected from Reuters (inherently available in the nltk package). We will also use pre-trained word vectors, available through the gensim API. These are the word2vec-google-news-300.
If you are new to TDA, I recommend you read the following blog post as an introduction on getting started with giotto-tda [2]. This one is also useful to better understand the Mapper algorithm used below.
The reader can find all the implementation details and the fully working code in this Jupyter notebook.
A problem with many search engine techniques today is that they rely on the words to exist in the corpus of a document that are already indexed.
When we search for certain terms in a collection of documents, many methods often look for the specific words, and this can be troublesome. A great example of a TF-IDF based approach for a search engine is the Kaggle notebook by Amit Kumar Jaiswal. Nevertheless, it also points out the weakness of this approach. Words in the query that do not exist in the documents vocabulary are removed from the query, and we might lose important information regarding what the user is searching for.
A tool that allows for constructing a search engine mitigating this problem is the Mapper algorithm in Giotto-tda. By taking the word vectors of all the words existing in a collection of documents, we can create a graph from these word vectors by using the Mapper method. Each node will then work as a cluster of words that are similar and often replaceable by each other. Due to the inherent nature of word vectors, words are perceived as similar when they are used in the same context (see [2]) and can be measured with the use of the cosine similarity. We, therefore, use the cosine distance as our distance metric when constructing our graph.
In essence, the mapper allows us to cluster the words into an arbitrary number of clusters, not a pre-determined number of clusters, with words that are similar in terms of semantical meaning. If a word does not exist, we can rely on its neighbours, defined through the vertices it belongs to, to fetch a document that might be about similar topics.
This mapper graph can then be used to query the data and retrieve documents. Each document has words belonging to different nodes, as different words end up in different nodes based on which words they are similar to. A document will therefore implicitly belongs to many different nodes as it is a collection of words.
We can explain a query in a few steps. Let’s say a word is searched for, e.g. Soybean. We then look up which nodes this word belongs to: the same node would also contains other words and therefore be associated with many different documents. We look at all documents that have words in those nodes, and the documents are ranked based on their percentage of words contained in the same node as the query. In the simple example of Soybean, we simply rank all documents based on the percentage of the words they have in the same nodes as Soybean. The output of the query is the list of documents, ranked as just described. If a word does not exist in the corpus, we can simply leverage the word vectors and derive the most similar word that exists in the corpus.
The construction of the Mapper graph therefore works as an efficient way of clustering words and implicitly creating what one might call a topological signature for each document.
For a brief mathematical formulation of the problem, please refer to the Jupyter notebook associated with this blog post.
Let’s take a look at an example where we can clearly see the semantics of the mapper graph at work.
Below is a search for “oil”, for which one of the top results is “SINGAPORE PETROLEUM CO RAISES OIL PRODUCT POSTING”. The first result perfectly matches the most used connotation of the word “oil” as fossil fuel, and the second result is about minerals and fossil fuels as well! The topological search engine captures semantics and meanings in the documents that BM25 does not (see the benchmark below for comparison)!
RESULT 0 for the query 'oil':  SINGAPORE PETROLEUM CO RAISES OIL PRODUCT POSTINGS  Singapore Petroleum Co Pte Ltd said it  will raise posted prices for its products from June 19, by one  cent/gallon for lpg, naphtha and gasoline, two cents for gas  oil and by 60 cents/barrel for marine diesel oil.      New prices are - lpg 36.0 cents/gallon, chemical naphtha  47, unleaded reformate 65.8, 0.4 gm lead 97 octane 61.3, 95  octane 59.3, 92 octane 55.5, 85 octane 49.5, 0.125 gm lead 97  octane 64.3, 92 octane 58.5, 85 octan...RESULT 1 for the query 'oil':  HOUSTON METALS' MINE YIELDS POSITIVE RESULTS  &lt;Houston Metals Corp> said the  the first phase of the underground rehabilitation, extensive  drilling and bulk sampling program at its Silver Queen Mine has  yielded positive results.      Houston said representative assays from the 2,750 ft and  2,600 ft levels at the south end of the mine established ore  deposits in the following ranges: copper, 3.7 pct to 5.08 pct,  lead .99 pct to 1.5 pct, zinc six pct to 9.6 pct, silver 15.63  pct ...RESULT 2 for the query 'oil':  API SAYS DISTILLATE, GAS STOCKS OFF IN WEEK  Distillate fuel stocks held in  primary storage fell by 4.4 mln barrels in the week ended Feb  27 to 127.10 mln barrels from 131.50 mln the previous week, the  American Petroleum Institute (API) said.      In its weekly statistical bulletin, the oil industry trade  group said gasoline stocks fell 30,000 barrels to 252.92 mln  barrels from a revised 252.95 mln, while crude oil stocks rose  700,000 barrels to 329.38 mln from a revised 328.68 mln...
Here below, we compare the Mapper-based search engine with a classical method, the BM25 search method. Following its guide on how to use it, we rank and retrieve results for the same keyword searches as in the topological method. We search for “oil” to compare with the topological approach.
RESULT 0 for the query 'oil':  DIAMOND SHAMROCK (DIA) CUTS CRUDE PRICES  Diamond Shamrock Corp said that  effective today it had cut its contract prices for crude oil by  1.50 dlrs a barrel.      The reduction brings its posted price for West Texas  Intermediate to 16.00 dlrs a barrel, the copany said.      "The price reduction today was made in the light of falling  oil product prices and a weak crude oil market," a company  spokeswoman said.      Diamond is the latest in a line of U.S. oil companies that  have cut...RESULT 1 for the query 'oil':  GLOBAL PETROLEUM &lt;GNR> UPS HEAVY FUEL PRICES  Global Petroleum Corp said it had raised  the contract prices for heavy fuel oil from 25 cts to one dlr  per barrel, effective today.      The company said 0.3 pct fuel oil is up one dlr a barrel to  22.25 dlrs a barrel. They said 0.5 pct fuel oil is up by 50 cts  to 21.95 dlrs a barrel.      Global raised one pct fuel oil by 35 cts to 20.25 dlrs a  barrel. The company raised 2.2 pct fuel by 25 cts to 19.50 dlrs  a barrel. Global raised 2...RESULT 2 for the query 'oil':  ENERGY ANALYST PROPOSES U.S. OIL TARIFF  Energy analyst Edward Krapels said  the United States should consider an oil tariff to keep U.S.  dependence on imports below 50 pct.      "On the supply side, the argument in favor of a contingent,  variable import tariff is most persuasive," Krapels, president  of Energy Security Analysis, Inc said in a statement at a House  Energy and Power subcommittee hearing.      "An optimal tariff would be one implemented only if the  international price...
The semantic richness oil as “fossil fuel” is completely lost in the top ranked documents: all top results explicitly contain the word “oil”.
In this work, we demonstrated a proof-of-concept, showing how it is possible to use Giotto-tda and its packages within NLP to create a semantic search engine. As seen with some examples (see oil) the Topological Search Engine captures semantics that BM25, a classical search engine, does not manage to capture.
There are however a few points that need to be highlighted.
Edges are not explicitly leveraged, which probably can be of value. In an extension of this, the edges should be used in some way to increase the search radius and possibly retrieve more relevant search results. As an example, if a document has many of its words in a neighbour to the node of the query, it can still be a highly relevant document for the query in question.
Constructing a Mapper graph is in many ways an arbitrary process, in which it is not always clear which filtering function and which overlap fractions and number of partitions we should use. In further work, one should investigate whether it is better with richer representations containing more nodes (higher number of partitions) and rich connectivity, as opposed to larger nodes (with fewer partitions) and lower connectivity. A higher connectivity means more words will belong to more nodes, yielding a more descriptive representation of each document. It is clear that this method can be further refined by further examination.
More work is absolutely needed to improve these results; most of all, this method can clearly be fine-tuned to work better. Either way, this blog post serves as a demonstration that Giotto-tda and topology have their places within the realm of Natural Language Processing. It also shows, for a few sets of queries, that the topological search engine is not necessarily inferior to classical methods such as BM25; with more adjustments, we truly hope it could probably perform even better.
[1] T. Mikolov, Distributed Representations of Words and Phrases and their Compositionality, Advances in neural information processing systems 26 (2013): 3111–3119.
[2] G. Tauzin et al., giotto-tda: A Topological Data Analysis Toolkit for Machine Learning and Data Exploration, ArXiv2004.02551