State fetching and updating strategies in react/redux applications with a lot of data
Tags: help, react, redux, discuss
Published: 2020-02-01 08:49:08

Hello to everybody!

This post is mostly a question and a rather general one. **I want people to share their experience**.

I would like to know how experienced people fetch data in react/redux applications, more precisely, in admin dashboards or similar applications, where there is a lot of data which cannot be fetched at once in the init phase.

I haven't much experience in creating admin dashboards and recently I got a task to create one. Let's look at the concrete example.

### Business logic and URLs.

I have a list of workers. Each worker entity has a list of different fields including a region. There are too many workers to fetch them all, but in each region, there are no more than 50-100 of them. So it's logical to add a region filter and show them only for the selected region. 

I use React Router, so the url for worker list is `/workers/:regionCode`.

When I select a worker to edit, the url is `/workers/edit/:workerId`.

When I create a new worker, the url is `/workers/create`.

When a worker is created, I redirect the app to `/workers/edit/:newWorkerId`, so that a user can continue to edit the worker.


### State organization

When a user switches a region, the new list of workers is fetched from the server. No workers from other regions are kept in the state.

When one edits a worker, he can change the region. And if he changes the region, that means I have to save the worker somewhere, but according to my state shape there is just no such place, since the worker is from another region now.

Moreover, if a user looks at the list of workers of one region, but then go to the url `/workers/edit/:workerFromAnotherRegion`, I have no data to show.

### Possible solutions I came up with

1) Remove React Router and make the life easier (to not allow to follow another region link) and make it impossible to switch the region of the worker when editing it. Instead, make the region an app-wide global option, and when it's switched the whole app is switched to another region.   

The solution is possible, but it will be rather hard to change the region of 
existing workers - one has to remove it from one region and create in another, because it will be just impossible to do otherwise.

2) Don't keep any state globally at all. Each page/component should load its state on each render. That is, the list of workers loads all workers of the region (or just their names for brevity). When a worker is selected, it's loaded again, despite the fact it might be loaded for the list. When it's created, after the redirect to another url, the component fetch the newly created data again by the id provided in the url `/workers/edit/:newWorkerId`.

In other words, just use the data from the url, and behave as classic web pages behave - load data on each transition. 

This approach seems to be the easiest in terms of state organization and state synchronization with the server. The only downside there may be a delay on each page transition, and maybe there will be more code, since components may fetch similar (or the same) data from the same API endpoints.

3) Change the state shape - normalize it a bit. Keep workers by their ids, and either select workers for each region traversing the registry or keep a list of workers ids for each region. If there is no worker id list for the region, do not add a newly created/edited worker id to an empty list, since the empty list is a signal, that the full list of workers wasn't fetched for the region. The lists of ids are formed when a corresponding list is to be rendered. 

The pros are that we cache data and don't reselect the same data twice. Page transitions and region switches are much faster in general case.

But at the same time, each component isn't guaranteed the needed data is already available in the state. That is, the worker edit form, which may be rendered when a user followed the corresponding url `/workers/edit/:workerId`, has to be ready for the absence of data, and it should emit a special action to load the missed data. It means that in terms of code complexity, this approach isn't simpler than the second one. All components still have to fetch data, although it can be made in a middleware, e.g. in a redux-saga. Also, we have to properly update the state, when data edited, to keep it in sync with the server, which adds more complexity.

### Conclusion 

I dislike the first approach, I relatively like the second one, and I have mixed feelings to the third (whether it's worth implementing).

But this post is written to ask other people to share their experience, so 

1. I would like other devs to tell me what approaches do you adhere to in such tasks in terms of data fetching and routing?
2. Are there any pitfalls in the listed approaches? 
3. How do you keep state in sync with the server? 
4. What approach would you advice for the task described above?