Reducing bandwidth and number of requests on REST APIs
By: Goncalo Oliveira
Tags: Restful, Dotnet, API, Restful Api, Rest

REST APIs have become the standard in the software industry and anyone involved in it has developed for or interacted with one. Although other options have appeared, such as GraphQL, they are generally more complex and require different tools and different approaches that aren’t suitable for many use cases. For that reason, REST stands as the de facto standard.

Nonetheless, REST APIs also have disadvantages. When we request a particular resource, we can’t typically choose which fields we want to retrieve, but instead, we get all of the fields available, even though we just need a couple.



---

{

"id": 10,

"name": "John Doe",

"dob": "1990-01-23",

"phoneNumber": "55000000000",

"email": "

"address": {

"street": "fake road 121",

"city": "London",

"country": "United Kingdom"

},

"preferences": {

...

}

} GET /users/10---"id": 10,"name": "John Doe","dob": "1990-01-23","phoneNumber": "55000000000","email": " johndoe@email.com ","address": {"street": "fake road 121","city": "London","country": "United Kingdom"},"preferences": {...

Another issue, is that if you want to get additional data related with the first request, you need to do a new request to gather it, which can quickly accumulate the number of requests our application makes.

GET /users/10/teams

---

[{

"id": 13,

"name": "Team A"

},

{

"id": 18,

"name": "Team B"

}]

In an attempt to fix this, the REST Schema Spec was written. It describes a set of extensions that allows us to tell the API exactly what we want to receive from the request. The idea is that its implementation is generic, coherent and across the entire API and not specific for a particular resource.

The Spec itself is language agnostic, as it describes a concept that could be implemented in a multitude of programming languages or frameworks. At the time of writing the REST Schema Spec is in version 0.1 .

Schema-Mapping

Let’s say that we want to retrieve a list of users, but we are only interested in the id and the name of the user, nothing else. Without using the Schema-Mapping the request would be something like

GET /users

And the response would probably give us more information than we need

To define the data we would like to receive, we need to write a schema spec in JSON or YAML. I’ll be using JSON, because the resulting base64 is smaller (even smaller if minified first).

{

"spec": {

"_": ["id", "name"]

}

}

This schema is then encoded as a base64 or base64Url string and passed through in the X-Schema-Map header. I’ll be using base64Url, again, because the resulting string is smaller (doesn’t require the = padding at the end).

GET /users

X-Schema-Map: eyJzcGVjIjp7Il8iOlsiaWQiLCJuYW1lIl19fQ

This would give us the following, much smaller, response

[{

"id": 10,

"name": "John Doe"

},

{

"id": 10,

"name": "John Doe"

}]

Now, granted that the original example didn’t bring that much data, but it is a 70% reduction and anyone that has worked with real-life scenarios knows that this can reduce drastically the bandwidth and improve the speed of the request and even the handling of the data on the client.

Schema-Include

Now let’s say that we wanted to retrieve data for a particular user, but we also want to retrieve the teams he belongs to. As we saw earlier, there are two different resources for this. First we would get the user

And then the user’s teams

GET /users/10/teams

---

[{

"id": 13,

"name": "Team A"

},

{

"id": 18,

"name": "Team B"

}]

We could achieve this by using the Schema-Mapping and define all the properties we wanted to retrieve from the user, including the teams. However, because we want to get all of the default data from the user, plus the teams, we can just define the additional properties to retrieve beyond the default ones.

{

"spec": {

"_": ["teams"]

}

}

As before, we encode the schema as a base64Url string, but this time we pass it through in the X-Schema-Include header.

GET /users/10

X-Schema-Include: eyJzcGVjIjp7Il8iOlsidGVhbXMiXX19

And with a single request, we get all of the user’s details

Closing Up

Using the same standard REST APIs, it is possible to come up with strategies that reduce the bandwidth and the number of requests an application needs to do. The REST Schema Spec tries to describe just that and I believe that the idea is an interesting one, though it would be great to get your feedback on it, so feel free to leave your comments.

For those working in a .NET environment, there’s already a C# implementation for ASP.NET and it is fully functional, though you should keep in mind it’s still a work in progress. If you are the adventurous type, feel free to have a look or contribute even.