Autocomplete with Crabs 🦀
By: Riccardo Valentini
Tags: Search, Rust, Backend, Autocomplete, Public Transportation

Here are some more detailed definitions of the terms and concepts, which in combination make up the autocomplete engine:

Matcher: a Matcher is a data set that can be queried for a list of suggestions (= data points from the data set), given a user query. Which suggestions from the data set are “matching” a query is up to the Matcher, the current implementation supports “starts with” and “contains” semantics. In order to return suitable suggestions as fast as possible, the indices use special data structures that are optimized for this purpose. E.g. to find out quickly if a query string is contained in one of the 730k ZHV items, we use a suffix array data structure provided by the awesome suffix crate.

Ranker: a Ranker takes the output of a Matcher (a vector of suggestions) and assigns a floating-point number from the range [0,1] to each suggestion, which indicates how well a suggestion fits the query. A Ranker can rank a suggestion based on any of the available attributes of this particular suggestion. This means that two different Rankers can have a very different opinion on the same suggestion. E.g. for Ranker-A a good fit might be a suggestion that is spatially close to the user, whereas for Ranker-B it could mean that the query contains the keyword “Flughafen”.

Selector: a Selector consists of a Matcher and one or more Rankers. The vector of suggestions that the Matcher outputs is given to every single Ranker of the Selector. Every Ranker computes a ranking value for every suggestion it receives. This means a single suggestion ends up with n ranking values if there are n Rankers defined for the Selector. The final ranking value for this suggestion is then simply computed as the weighted average over all n ranking values. After all ranking values are computed, the Selector sorts the suggestions and returns the highest ranked results.

Strategy: application of one or more Selectors to a user query, where the outputs of the selectors are merged into a single vector of ranked suggestions in the end. Depending on the query string (e.g. how many characters?) and if there is a current GPS position for the user available, a different Strategy can be chosen to answer the query. The application’s algorithm that transform a query string into a list of suggestions is therefore implemented as a “Strategy”.

Here is an example of a simple Selector configuration 🦀

Matcher::ZhvLandmark means that this particular Selector uses a Landmark-Index (see next section for details) computed over the ZHV dataset as Matcher. This also means that this particular Selector will only ever return elements from the ZHV data set. As you can see there are 4 different Rankers defined for this Selector. These are also no regular Rankers, but WeightedRankers, which means the ranking value that each of these WeightedRankers produces, does not equally contribute to the final ranking value. In this configuration for example, the rating given by the PrefixPattern Ranker is considered more valuable and decisive, compared to the rating given by the SpecialKeywords Ranker. Let’s have a closer look at one of the Rankers to better understand how it works.

Here is the implementation of the Ranker called “Distance” that is used in the Selector above: 🦀

This Ranker only uses the user’s GPS-position (position: Option<Location> parameter of the function) and the GPS-position of the suggestion (item.latitude and item.longitude) to calculate the distance between the two. The closer they are together, the higher the ranking value that this Ranker assigns to the particular suggestion. In case there is no user GPS-position available (the None branch of the pattern matching), the Ranker returns the value 0.5, which represents a “neutral” ranking value.

Any Ranker must implement the “Rankable” trait, which is defined as 🦀

To wrap everything up in the end, a Strategy is simply defined by the following trait, which any struct that wants to be a Selector must implement: 🦀

The required apply() method takes three arguments:

the query string that was submitted by the user, an optional GPS-position of the user, which some of the Rankers can use to compute their ranking value, the application state containing all data indices.

It returns a sorted vector of tuples containing a SaftItem (= internal representation of a user suggestion) and a floating-point number representing the ranking value. This vector of suggestion can be more less directly mapped into JSON and returned to the user.

As the data flow diagram from above shows, a user query can invoke different strategies within the autocomplete engine. Strategies can differ in the following points:

Matcher: which data set(s) should be used to answer the query? Rankers: which Rankers should be used to decide which suggestions are most relevant? Weight of the Rankers: how decisive should individual Rankers be for that decision?

This leaves us with an adaptable and flexible way of generating autocomplete suggestions, while maintaining a simple and straightforward program flow.

Landmark Indexing

In this section we will have a look at a concrete Strategy called “Landmark Indexing” that is used every time the user’s query string is really short. The name is somehow borrowed from a routing strategy called “Landmark routing”, because the idea behind is a bit similar. But before we go into depth here, let’s describe the problem first that “Landmark Indexing” tries to solve by comparing two different user queries, a very short query and a longer one:

“B” “Oskar-von-Miller-”

Executed on the OSM and ZHV data set, query (1) has more than 100k matches, which means that the names of more than 100k public transport nodes start with a capital “B”. The matching itself is not a problem in this case, the used data structures (mostly the suffix array that was mentioned already in the last section) also provide excellent performance, when there is a large number of matches. The expensive part is the ranking however. To rank each of the 100k public transport nodes by multiple Rankers and also to sort them afterwards can become very expensive depending on the number of items. Although a single ranking operation might be cheap (e.g. check if a special keyword is contained in the query string), in total it takes time to process all matches. Running a full index search for single letter queries like this turned out to be quite inefficient and let to processing times far beyond 1s for some characters. Query (2) on the other hand only has around 100 matches across both data sets, which leads to lightning fast processing times of around 2ms. To sum this up: the longer the query, the cheaper it is to process it!

So what can we do to avoid such long running full index searches on very short queries? We can try to treat very short queries differently. Looking at query (1) again, the single letter “B” alone does not contain much information about what the user is looking for. Gathering every single public transport node starting with the letter “B” and then trying to rank all of them does not create much value for the user. Instead a smarter approach is to keep a separate smaller index of public transport nodes that are “more likely” to be selected by users and then just serve very short queries through this index. This not only reduces the processing times drastically, but also increases the probability that a user receives helpful suggestions upon his first key press. The next problem is then however: how can we know which public transport nodes are “more likely” to be selected? We don’t have any data on the frequency of public transport searches, but as a workaround, we used a metric based on the ZHV data set instead. Using the DHID of a ZHV item, it is possible to compute a size metric for public transport nodes. The hierarchical structure of the DHID makes it possible to count how many “sub-nodes” a public transport node has. Each platform of a large train station would be considered a “sub-node” for example. The logic behind it is the following: the more sub-nodes a public transport node has, the more transport connections are being served through this node, which means more people will be interested in this node. Our working hypothesis is therefore: on very short user queries it is most helpful for the user, if we just return the largest public transport nodes that we can find in the user’s proximity. These “largest transport nodes” are what we call Landmarks.

We haven’t talked about the proximity part though! Since we want to build an autocomplete engine that makes good use of the user’s GPS position, it makes sense to include it in this Landmark Indexing approach. Instead of building a single index, we chose the 200 biggest German cities and build an index for each by taking the 10k biggest public transportation nodes within a 30km radius around each city. If a user now submits a very short query together with his current GPS-position, we select the Landmark index that is closest and only consider the public transport nodes contained in this index for this query. By applying this strategy, we can achieve processing times as fast as 20ms, depending on the GPS-location and query string.

The following image is a visualization of the Landmark Index distribution across Germany. As you can see, there is a lot of overlapping, especially near Cologne, which means we are storing duplicates in this area. This approach could very well use some fine-tuning to be more memory efficient, but to demonstrate the concept, this coarse partitioning using 200 indices should be sufficient:

To summarize the idea of Landmark Indexing: for very short queries that are not distinctive enough yet, serve the user with some common Landmarks that match the query. As soon as the user entered enough characters for a more precise search, take all available data into account.

Performance

Let’s have a look at the overall performance of the autocomplete engine by doing some benchmark tests. Since it makes a big difference if a query is served using the Landmark Index or not, we also conducted two different benchmark tests to distinguish between both cases:

Queries of length 3 or shorter, which are always handled through a Landmark Index Queries of length 4 or longer, which are never handled by a Landmark index

The Strategy configuration of the engine that was used for the benchmarks makes sure that all queries with more than 3 characters always trigger a full index search. Given this configuration, we took the “worst case” query length from both groups, which is query length 1 for the group (1) and query length 4 for the group (2). This means the most expensive query in terms of execution time for the Landmark Index is a single character query, because this results in the largest possible number of matching suggestions. The same holds true for queries of length 4. There are no shorter queries that trigger a full Index search, therefore length 4 queries will generate the largest possible result sets. Note that both benchmarks use randomly generated alphabetical characters in their queries. Also note that the “cut-off point” for Landmark processing was set to 3 characters, because for queries of length 4 or longer, the full index search always completes fast enough for our data sets.

The benchmarks are executed using the awesome criterion crate, which not only provides the wrapper code to conduct performance evaluations, but also outputs very nice reports with all the charts and statistics that you need.

The following two graphs are taken directly from the criterion reports and show the runtime probability distributions for both benchmarks: