Setting up a large scale React application
By: Bruno Sampaio
Tags: Typescript, Tdd, JavaScript, React, Programming

This post is based on the series of posts: Modernizing a jQuery frontend with React. If you want to get a better overview of the motivation for this post we recommend you first read our initial post.

Nowadays, it is very easy to set up a small React application, or to start one from scratch. Especially if you use create-react-app. Most projects probably just need a few dependencies (for example, for state management and for internationalization) and a src folder with at least a components folder. I guess this is how most React projects start. Typically, though, as the project grows the number of dependencies, components, reducers and other shared utilities tends to increase in a, sometimes, not so controlled manner. What do you do when it is no longer clear why some dependencies are needed or how they work together? Or when you have so many components it becomes difficult to find the one you need? What do you do when you want to find a component but you don’t really remember its name?

These are just some examples of the questions we came across while rebuilding our frontend at Karify. We knew that the number of dependencies and components was eventually going to grow out of hand. That meant we needed a plan that would be scalable enough to keep up with future development. This plan involved defining conventions for our file and folder structure, code quality and also defining the overall architecture. Most importantly, all this should be easy to pick up by new developers without requiring them to have much insight into all our dependencies and code style.

At the moment of writing, we have about 1200 JavaScript files of which 350 are components with 80% of unit test coverage. Since we still believe in the architecture and conventions we created we thought it would be a good idea to share them. In the following sections, I’ll go through how we set up our project and also some of the lessons we learned.

How to organize files and folders?

We went through multiple phases until we figured out how we wanted to organize our React frontend. Initially, we thought of placing it in the same repository as our jQuery frontend. However, the imposed folder structure of our backend framework made this option undesirable. Next, we thought of moving it into a separate repository. This worked well at first, but with time we started thinking about creating other frontends, such as a React Native frontend, which motivated the need for a component library. This led us to split this new repository into two separate repositories: one for the component library and one for the new React frontend. Even though this seemed like a good idea, it resulted in a very complex review process. The relationship between changes in the two repositories became unclear. Finally, we chose to bring them together again into one repository, but this time as a monorepo.

We chose a monorepo because we wanted to create a separation between our component library and our frontend applications. The difference between our monorepo and others out there is that we don’t really need to publish the packages inside of it. For us, the packages serve only as a means for modularity and separation of concerns. It is especially useful to have different packages for every different application since we can specify different dependencies and scripts for each one of them.

We set up our monorepo using yarn workspaces with the following configuration in our root package.json :

"workspaces": [

"app/*",

"lib/*",

"tool/*"

]

Now some of you might wonder why we didn’t simply use a packages folder like in other monorepos. Mainly because we wanted to create a separation between our applications and our component library. Besides that, we also knew we needed to create some tooling of our own. So, we came up with the folders you see above and here is an explanation for each one of them:

app : all packages in this folder refer to frontend applications like our Karify frontend, some internal frontends and also our Storybook;

: all packages in this folder refer to frontend applications like our Karify frontend, some internal frontends and also our Storybook; lib : all packages in this folder expose shared utilities to our frontend applications and are as application-agnostic as possible. These packages basically form our component library. Some examples would be our typography , media and primitive packages;

: all packages in this folder expose shared utilities to our frontend applications and are as application-agnostic as possible. These packages basically form our component library. Some examples would be our , and packages; tool: all packages in this folder are Node.js specific and either expose tools that we created ourselves or are configuration and utilities for tools we depend on. Some examples would be webpack utilities, linter configurations, and a file system linter.

Regardless of where we place them, all our packages always have a src folder and optionally have a bin folder. The src folder of our app and lib packages might contain some of the following folders:

actions : contains action creator functions of which the return value can be passed to the dispatch function from redux or useReducer ;

: contains action creator functions of which the return value can be passed to the dispatch function from or ; components : contains component folders with their respective definition, translations, unit tests, snapshots and stories (if applicable);

: contains component folders with their respective definition, translations, unit tests, snapshots and stories (if applicable); constants : holds constant values reused across different environments and also different utilities;

: holds constant values reused across different environments and also different utilities; fetch : holds type definitions for the payloads from our API and the respective async actions to retrieve them;

: holds type definitions for the payloads from our API and the respective async actions to retrieve them; helpers : holds utilities that do not fit in any of the other categories;

: holds utilities that do not fit in any of the other categories; reducers : contains reducers to be used in our redux store or useReducer ;

: contains reducers to be used in our store or ; routes : holds route definitions to be used in react-router components or history functions;

: holds route definitions to be used in components or functions; selectors: contains helper functions that read or transform data from our redux state or our API payloads.

This folder structure allows us to write really modular code since it creates a clear separation of concerns between different concepts defined by our dependencies. This helps with the lookup of variables, functions, or components in the repository, independently of us knowing if they exist or not. Furthermore, it also helps keeping the contents of these folders to a minimum, which in turn makes them easier to process.

One challenge with this new folder structure is to make sure we stick to it. It is tempting to start creating different folders in different packages and organizing files differently between them. While that might not always be a bad idea we would end up with a mess if we don’t do it consistently. To help with that, we created a file system linter which I’ll describe in more detail in the next section.

How to enforce a style guide?

In the same way that we wanted to have a consistent file and folder structure, we also wanted to have as much consistency in our code as possible. This is something that we already did quite well in our jQuery frontend but it could be improved especially when it came to CSS. Therefore, we tried to define a style guide from the beginning and to enforce it with a linter. Rules that can’t be specified with a linter are enforced during code reviews.

Setting up a linter in a monorepo is the same as in any other repository, it is great because you can run it once to validate the entire repository. If you are not familiar with any linters I recommend you take a look at ESLint and Stylelint which are the ones we use.

Having a JavaScript linter proved especially useful for the following use cases:

Enforcing the use of accessibility aware components over their HTML counterparts: during design, we defined several accessibility guidelines for anchors, buttons, images, and icons. Then in code, we wanted to implement those guidelines and make sure we don’t forget about them in the future. We did this with the react/forbid-elements rule from eslint-plugin-react. Example of what it looks like:

'react/forbid-elements': [

'error',

{

forbid: [

{

element: 'img',

message: 'Use "<Image>" instead. This is important for accessibility reasons.',

},

],

},

],

Forbidding imports of application packages from inside library packages and forbidding the import of application packages inside other applications: mainly to avoid circular dependencies between packages in the monorepo and to make sure we stick to the separation of concerns we created. We forbid this with the import/no-restricted-paths rule from eslint-plugin-import.

In addition to JavaScript and CSS linting, we also have our own file system linter. This is how we make sure that we stick to our folder structure. Since this linter is ours we can always change it if we decide to change the structure. Here are some examples of the rules we have:

Validate the structure of our component folders: make sure there is always an index.ts and a .tsx file with the same name as the folder.

and a file with the same name as the folder. Validate our package.json files: make sure there is always one per package and that it is set to private in order to avoid us publishing packages accidentally.

What type system to use?

Nowadays, the answer to the above question is probably a lot clearer to everyone. Simply go for TypeScript! No matter the size of your project, it might slow you down in some cases, but, in our opinion, it is worth the quality and strictness it adds to your code.

Unfortunately, when we started this project it was still very common to use prop-types. In the beginning that was enough, but as the project grew we really started missing the possibility to define types for more than just components. We could see the value they would add to, for instance, our reducer and selector functions. However, adding a different type system would require a lot of refactoring in order to get to a point where the entire code base has types.

In the end, we did it anyway but we made the mistake of trying Flow first because it looked easier to integrate into our codebase. While that was true, Flow also became a pain to work with since it didn’t integrate well with our IDE, it randomly failed to spot some type errors, and creating generic types was a nightmare. For those reasons, we ended up moving everything over to TypeScript. If we knew then what we know now, we would have used TypeScript from the get-go.

The direction in which TypeScript evolved in the last few years helped make this transition much easier. The deprecation of TSLint over ESLint was especially useful for us.

What testing approach to use?

When we started it wasn’t exactly obvious for us what testing tools to use. Nowadays, I would say that jest and cypress are the best out there for unit testing and integration testing, respectively. Their configuration is well documented and not complicated. It is just a pity that cypress does not support the Fetch API and its API does not support async /await. It took us some time to figure that out in the beginning. Hopefully, this will change in the near future.

Finding out how to best write unit tests for our application was quite difficult at first. Over time we tried out things like snapshot testing, the test renderer, the shallow renderer and the testing library. In the end, we decided to stick to the shallow renderer to validate the output of components and use the test renderer to test the internal logic of components.

In our opinion, the testing library is great for small projects but the fact that it relies on the dom renderer has a big impact on the performance of tests. Furthermore, we think that the argument against snapshot testing with shallow rendering does not make sense when you have a very deep layer of components. For us, these snapshots are very useful to validate all the possible outputs of a component. It is important to keep them readable, though. This can be accomplished by keeping your components small and by defining the toJSON method for object props that are not relevant for the snapshot.

In order to keep us motivated to write unit tests and to not forget about them, we also defined a coverage threshold. This is very easy to configure when using jest. You don’t need to give it much thought. Just start with a global threshold and improve it over time. We started at 60% and over time, as coverage increased, we moved the threshold to 80%. For us, this seems like a good enough threshold since we also don’t think it is realistic or necessary to target 100%.

How to bootstrap the application?

Usually, starting a React application is as simple as writing something like ReactDOM.render(<App />, document.getElementById(‘#root’)); . However, this gets more complex when you also want to support SSR (Server-Side Rendering). Additionally, if you also need other dependencies besides React, they might need separate configurations for client-side and server-side. For instance, we use react-intl for internationalization, react-redux for global state management, react-router for routing and redux-sagas for managing asynchronous actions. These dependencies require some setting up, which can get complex very easily.

Our solution to this problem was based on the Strategy and Abstract Factory design patterns. We basically created two different classes/strategies: one for client-side configuration and one for server-side configuration. Both of them receive the configuration for the application being bootstrapped which includes its name, logo, reducers, routes, default language, sagas, etc. The reducers, routes and sagas can come from different packages in our monorepo. This configuration is then used to create the redux store, create the sagas middleware, create the router history object, fetch the translations and finally render the application. To give an example, below is a simplified version of how the signature of both our strategies looks like:

type BootstrapConfiguration = {

logo: string,

name: string,

reducers: ReducersMapObject,

routes: Route[],

sagas: Saga[],

}; class AbstractBootstrap {

configuration: BootstrapConfiguration;

intl: IntlShape;

store: Store;

rootSaga: Task; abstract public run(): void;

abstract public render<T>(): T;

abstract protected createIntl(): IntlShape;

abstract protected createRootSaga(): Task;

abstract protected createStore(): Store;

} // Strategy for client-side

class WebBootstrap extends AbstractBootstrap {

constructor(config: BootstrapConfiguration);

public render<ReactNode>(): ReactNode;

} // Strategy for server-side

class ServerBootstrap extends AbstractBootstrap {

constructor(config: BootstrapConfiguration);

public render<string>(): string;

}

We found this separation useful since there are several differences in how our store, sagas, internationalization object and history object are set up, depending on the environment. For instance, the redux store on the client-side is created using the preloaded state from the server and the redux devtools enhancer, while on the server-side it doesn’t need any of that. Another example is our internalization object which on the client-side gets the current language from navigator.languages while on the server-side it gets it from the Accept-Language HTTP header.

It is important to note that we came up with this solution a while ago already. At the time it was still a lot more common to use classes in React applications and there were no easy solutions for SSR. Over time React changed towards a more functional style and solutions like Next.js arose. On that note, if you are looking for a solution for this same problem, we recommend that you also look into what’s possible nowadays, since you might find a simpler and more functional way.

How to maintain the code quality?

Linters, tests and types are useful to improve the quality of your code, but it is easy to forget to check if they are all valid before merging some changes into your master branch. The best solution is to do this automatically. Some prefer to do this on every commit with git hooks preventing you from committing unless everything passes. However, we think this is too intrusive since you could be working on a branch for several days until it’s ready. For this reason, we validate our commits using CI (Continuous Integration) pipelines which only run when the branch is associated with a merge request. This way, we avoid running pipelines that are expected to fail since most of the time we only create a merge request when we think the code is ready.

Our pipelines start by installing all our dependencies and then validate the types, run the linters, run the unit tests, build the application and run the cypress tests. Almost all of this happens in parallel. If any of these steps fail then the pipeline fails and the branch can’t be merged. Below is an example of a running pipeline.

The most difficult part of setting up this pipeline was, and still is, to keep it fast. We went through a lot of optimization phases and at the moment it has been running stable at around 20 minutes. We could probably improve this by running some cypress tests in parallel but, for now, this is acceptable.

Conclusion

Setting up a React large scale application is no easy job. There are a lot of choices you have to make and a lot of tools you have to configure. There is not just one right answer on how to do this.

We are still happy with our setup and we hope it inspires others struggling with setting up their own applications. But before you follow our example, try to make sure this is the right set up for you or your company. Most importantly, add only the dependencies you need. Don’t over-complicate it.

If you have questions, feedback, or suggestions regarding what you read here I would be happy to hear from you in the comments section below.