Solutions Architect Reacts: Introduction and Newsfeed Architecture
By: Kevin Ling
Tags: Firebase, Product, Software Development, Consulting, Architecture

Photo by David Travis on Unsplash

As a solutions architect I always start with the high level vision. What are we trying to achieve? (We’ll detail our Toggle framework for problem decomposition in another post) I’m going to start off this first post with a challenge that we came across recently in our UpForThat app.

Vision

A mobile app which serves a user up-to-date local information based on their location.

Our latest app UpForThat (upforthat.co)

Further Requirements

I need the app to pick up and display feed items (e.g. news, events, ads) relevant to user’s current location.

I have multiple feeds of information that need to be aggregated into one “Home Feed”.

The “Home Feed” needs to be performant and load fast.

The “Home Feed” should still be populated with feed items even when the phone is offline.

Similar Implementations

Facebook Feed

Instagram Feed

Other news aggregators e.g. Feedly

Technologies

React Native

Firebase (https://firebase.google.com/)

As explained in the intro, technologies aren’t important. However I will detail the specific features we are leveraging, which might guide you in your own implementation and technology choice.

With any problem we need to first decompose it into bite-sized chunks. Our thoughts on this this process will be a topic for another post — Toggled Framework.

Single Feed

Our simple case can be boiled down to:

I want to retrieve and display the relevant data based on a user’s current location. In the simplest case we can take one feed.

Single Feed Architecture

This can be done by a simple GET request with the user’s location being passed in as a parameter.

Generically the API gateway is the interface to our backend services, usually this also handles user authentication. React Native Firebase handles authentication automatically for us.

The cloud function handles the GET request and does a query against the collection. The cloud function then returns the feed items related to a user’s location which is then relayed back to the device. Cloud functions are GCP/Firebase’s version of a server-less function.

So far so good. In terms of the overall vision we’re still missing a few things. There are three requirements this implementation does not meet:

Currently we only serve one feed. We want to aggregate more than one feed into a single “Home Feed”

Every time we initiate the “Home Feed” view on the device we need to wait for the request to complete before displaying the results. This may take a long time depending on the query being executed on the database.

When the request is executed we have to wait for a reply, if we are offline the GET request will not receive a reply. This means that the user interface may be blank or at best show a loading spinner until we are back online.

Multiple Feed

Let’s make our simple case more complicated:

I want to retrieve and display the relevant data based on a user’s current location. The feed will be an aggregate of multiple feeds.

Multiple Feed Architecture

So expanding on our single feed architecture. We are going to add an additional feed into the mix.

The key to this is allowing the cloud function to do the heavy lifting for us. As a principle we want the backend server to do as much of the computation as possible to allow the client (mobile device) to concentrate on visualisation and not worry about object manipulation.

With that in mind the cloud function queries each collection and aggregate the data into one object.

For efficiency I have parallelised the database querying using Promise.all in NodeJS. There are a few ways to combine data from different data sources but I’m going to highlight two:

Interleaving is the simplest way to combine data from different data sources. All this means is taking the different data sources distributing it evenly to make a single datasource.

For example, for coordinate data, x0 y0 z0 w0 x1 y1 z1 w1 x2 y2 z2 w2 is interleaved whilst x0 x1 x2 x3 y0 y1 y2 y3 z0 z1 z2 z3 w0 w1 w2 w3 is not. — Wikipedia

Algorithmic methods is a more sophisticated approach which may not present a particular pattern to the user. But could be dependent on a number of user specific factors, think Instagram feed.

The new Instagram algorithm dictates the order of the posts that users see when they’re scrolling through their feed. Based on specific signals, it prioritizes posts, pushing the most relevant ones towards the top and giving them the most visibility, while other content ends up being placed further down in one’s feed. — Shopify

This was a good iteration on our initial design, so let’s keep the ball rolling. In terms of our overall vision we still haven’t tackled the asynchronous loading of the “Home Feed”. Each time the feed is loaded we have to wait for a response. This can lead to bad user experience and does not fit the following requirements:

The “Home Feed” needs to be performant and load fast.

The “Home Feed” should still be populated with feed items even when the phone is offline.

Performant and User Friendly Loading

Again let us first frame our problem in its simplest form:

I want to retrieve and display the data from the backend in a performant way which minimises the need for long waiting times and lets a user have a consistent user experience whether on or offline.

Performant and User Friendly Loading Architecture

This is where I leverage some specific Firebase functionality and where I will give you hints on how you can substitute your own technologies. There are a couple things to know before I explain the architecture:

Firebase provides the client (React Native app) the ability to subscribe to changes in specific collections and documents within Firestore (Firebase’s collections database). To achieve this one could potentially implement their own long polling or web-socket solution or use other technologies that might you give you the same functionality.

Firestore supports offline data persistence. This feature caches a copy of the Firestore data that your app is actively using, so your app can access the data when the device is offline. You can write, read, listen to, and query the cached data. When the device comes back online, Cloud Firestore synchronizes any local changes made by your app to the Cloud Firestore backend. Again to achieve this you can potentially implement your own solution or use another technology.

With that in mind I’ll walk through the above architecture:

1a. Same as before, the mobile device sends a GET request along with the user’s current location to our API gateway to kickstart the feed aggregation process. As a reminder this is where we can identify the user. This is required for step 4.

1b. At the same time however our mobile device creates a subscription to the user’s document in the “Home Feed” collection. As an automatic operation, in addition to receiving subsequent updates my client loads up the previous “Home Feed” items either from cache (offline) or from the live collection itself (online). Setting me up to display information asynchronously.

2. Same as before, I kickstart a cloud function to aggregate the feeds.

3a + 3b. As before, I query the relevant collections and aggregate the results.

4. Once the resulting “Home Feed” object has been created I POST the result into “Home Feed” collection specifically with a reference to the user who kicked off the process in the first place (retrieved in step 1a.)

5. Because of the client’s subscription to the user’s “Home Feed” document in step 1b I automatically get pushed an update with the new “Home Feed” object. The client then displays this to the user.

Conclusion

And that’s it for now! Hopefully that gives you an idea of how you might tackle a similar problem and some reference architecture to use as a foundation for your own designs.

Remember that all designs are iterative so we’d love to hear how you might want to improve on our architecture or how you’ve implemented similar solutions.

Checkout our UpForThat in full and see what we’ve been working on recently: