GraphQL Demystified (The n+1 problem)
By: Philip Starritt
Tags: GraphQL, Evooq Software Dev, Java, Software Development, Spring Boot

The n+1 Problem

GraphQL fields resolve in an independent manner. This means that each field will trigger the execution of a matching resolver method. What if the resolver executes an expensive operation such as a network request?

Hello, n+1 problem.

The n+1 problem

Naive resolver implementation

In the above example, the server sends a single request to the stock datastore. A request is then sent to the price service for each stock. For example, if there are 20 stocks, then 20 requests are made to the price service. Forming a total of 21 requests. The server should be able to retrieve all stock prices in two requests. One to fetch the stocks and another to fetch all the necessary prices.

Without correcting this behavior, one can expect huge implications in their distributed system — Network, performance, latency, etc.

The Solution

DataLoader

“DataLoader is a generic utility to be used as part of your application’s data fetching layer to provide a simplified and consistent API over various remote data sources such as databases or web services via batching and caching.”

Like GraphQL, the dataloader concept originated at Facebook. Fortunately, a JavaScript implementation was open-sourced, followed by a pure Java8 port.

Batching is a dataloader’s primary feature. A dataloader will load a key and return a promise. The keys are grouped and supplied to a batch function. Promises are correlated and completed with the batch function’s result.

It is essential that all backing APIs support batching.

Attempt 1: BatchDataLoader

Creating the DataLoader

The GraphQL server defines and registers the required dataloaders into the DataLoaderRegistry . The DataLoaderRegistry provides access dataloader references during resolver execution.

DataLoader correlating based on an ordered list

DataLoader Scope

The scope of a dataloader is very important. In our case, we always create a new DataLoaderRegistry per GraphQL query. This ensures that the dataloader cache and batch function will be specific to the user requesting it. If data can be safely shared among queries, then one can consider sharing a dataloader instance among web requests/queries.

Loading keys to the DataLoader

The DataFetchingEnvironment has access to the DataLoaderRegistry’s dataloaders. Load each stock Id into the appropriate dataloader. The dataloader will dispatch upon loading the last stock Id, or if the dataloader’s max batch size has been reached. If the max batch size is reached and all keys have not been loaded, multiple batch requests will be made.

Dataloader Max Batch Size

The dataloader max batch size should equal the downstream service’s recommended max batch size. Ideally, this should be set to an optimal figure provided by the downstream batch service team. The max batch size can be set during dataloader construction. Each dataloader can have a different max batch size.

Problems

This dataloader implementation requires a 1:1 ordered mapping of batch loaded keys to the values returned. This approach is error-prone and presents many risks. If the batch service returned an unordered or partial list, then the correlation between CompleteableFuture and batch entry may be incorrect. This leads to a data error that fails silently.

Attempt 2: MappedBatchDataLoader

The MappedBatchDataLoader correlates the CompletableFutures based on a returned Map<Key, Value>. If a loaded key is not present in the map, the correlating CompletableFuture is completed with null. In the majority of use-cases, this approach is safer than correlating on list position.

For example:

If you query a database with 10 keys, the resultset may contain less than 10 results. Additionally, they may be out-of-order.

If you query an external service, can you guarantee that the developers will not introduce a bug and return data out-of-order?

DataLoader correlating based on Map<K, V>

Passing an additional object to the DataLoader

An additional object can be supplied to the dataloader load method. This is known as the context object. This value is commonly used to perform a finishing function on each batch response entry.

This approach provides us a way to pass additional information to the dataloader whilst retaining optimal key-lookup performance.

Passing a context object

The key contexts are available inside the BatchLoaderEnvironment.

Thread Context World

Often ThreadLocal variables store user information. Key use-cases include MDC, correlation ids, and spring security’s SecurityContext.

Dataloaders execute in a different thread. The loading thread’s threadlocal variables must propagate to the dataloader’s threadlocal. Delete the dataloader’s threadlocal variables upon dataloader completion.

Thread Local Propagation

When using Spring Security, the SecurityContext can be propagated using Spring’s DelegatingSecurityContextExecutorService .

Security Context Thread Propagation

Conclusion

The default GraphQL field resolver implementation may lead to the dreaded n+1 problem. At scale, the n+1 problem will escalate rapidly, causing severe performance problems.

The dataloader pattern offers a simple solution to enable optimal network requests within a GraphQL query.

Happy coding :)

Philip