State vs knowledge: you should make your apps a bit more complex to keep them simple
Tags: architecture, webdev, javascript
Published: 2020-03-17 15:49:11

Generally, every client web app - SPA, PWA, whatever - real soul is state.

We may brag on React, Vue or Svelte, we may shamefully (or proudly) continue to use jQuery, but what really defines the way we think, the way we tighten up with back-end, APIs and storage? State and the way you manage it.

And ton of people are struggling from state manager fatigue. Redux? [Yes](https://www.reddit.com/r/reactjs/comments/ce95u0/am_i_the_only_one_who_feels_redux_is_very/), [yes](https://dev.to/bettercodingacademy/redux-is-seriously-overrated-change-my-mind-45pm) and [yes](https://www.quora.com/Why-is-Redux-confusing). RxJS? [Sure](https://www.quora.com/What-makes-RxJS-so-difficult). MobX? Why would it have [pitfalls.html](https://mobx.js.org/best/pitfalls.html) page in docs if it simple?

I think there is a solution, but first we have to fully draw the problem. 

When choosing the state manager, you are choosing the way you think. There is a lot of choices nowadays. Most popular are:

- **Flux/Redux-style** state, a global store with actions & reducers - well, tons of them. I would personally note [Redux](https://redux.js.org/) itself, [Effector](https://github.com/zerobias/effector), [Storeon](https://github.com/storeon/storeon), [Unstated](https://github.com/jamiebuilds/unstated), and [Reatom](https://github.com/artalar/reatom). This is not "best of" list. It is about different ways on how it can look like. Also, each of them has something very unique (from my point of view), so all of them are worth giving a glance - just to check out various concepts, not to use in production!

  This approach can be defined as **imperative**/Turing-complete & **global**.

- **Observables** & **pipes**. Most well-known ones are [RxJS](https://rxjs.dev/) and [MobX](https://mobx.js.org/). Less known - [Kefir](https://kefirjs.github.io/kefir/), [Bacon](https://baconjs.github.io/), [CycleJS](https://cycle.js.org/). [Svelte](https://svelte.dev/) goes here too. They differ a lot, but it comes from one core difference - RxJS allows ["strange loops"](https://en.wikipedia.org/wiki/Strange_loop), when you can pipe observables through observables, and MobX just creates "reactive" boxes over the variables and computations.

  It may sound weird, but they are aiming to be **local/ad-hoc** & **declarative**, yet still Turing-complete (I will write an article about that one day). They allow you to describe how data will be transformed, not what exactly to do with it. On some level of, um, enlightment, RxJS developers starts to avoid writing functions as much as possible, preferring to use libraries like [Lodash](https://lodash.com), or [Ramda](https://ramdajs.com/), or [io-ts](https://github.com/gcanti/io-ts), and their code actually starts tasting LISPy and looking like JSON or YAML, not real code.

  Speaking on local, I mean that you may have component-level observable, or application-level, or you can pass observable as an argument - you can do whatever you want with any of data sources.

- **GraphQL**-alike. [Apollo](https://www.apollographql.com/) and [Relay](https://relay.dev/) are best of examples, but you can find ton of them. Special mentions goes to [Falcor](https://netflix.github.io/falcor/) (Netflix alternative to GraphQL query language), [GunDB](https://gun.eco/) and [PouchDB](https://pouchdb.com/). Moreover, there are implementations & integrations with Redux, MobX, RxJS - any of them. But actual store does not matter; what really matters is the way to state the expectations. It is 100% declarative - in comparison to Flux-way imperative data reducement. And it is global.

So we have 2 dimensions of state management. One is local/global, second - declaration/imperative orders. And that makes us to ask the questions.

|        | imperative  | declatative |
| :----- | ----------- | ----------- |
| GLOBAL | Flux        | GraphQL     |
| LOCAL  | Observables | **?????**   |

I should probably make a note here. Terms **"global"** and **"local"** may be a bit confusing here, as long as you can place Rx observable as a global variable, and you can [dynamically load](https://redux-dynamic-modules.js.org/#/) redux stores. 

Rule of thumb here is: if something is forced to have globally unique ID getter as intended behavior - it is **global**. No matter how ID is used - it can be`window[key]`, or `require('stores/' + key)`, or `dynamicModuleLocator.get(key)`. 

If something is intended to emerge within some other entity lifecycle - say, React or Angular component, or API queue manager, or whatever else - it is **local**, despite the fact you *can* assign it to `window[key]`. Otherwise you would have to consider everything possibly global.

### The missing link

This may seem weird.

I cannot recall any local & declarative state manager. With chances, you will name some esoteric or experimental state managers, but nothing from ["state of js" list](https://2019.stateofjs.com/data-layer/other-tools/) & nothing I was able to find.

And, probably, the answer is the following:

> We are blending two different entities into one.
>
> We have **state**, which is runtime-level, and we do not care on whether it is local or global; but what we do know is that it is best manipulated in imperative manner.
>
> And we have **knowledge** about what is happening. We can affect this knowledge, but we can always predict the consequences of our affection; so, we can declare that we want to **know** something about our system state.
>
> When we are using single store for both state and knowledge, we're doing something fundamentally wrong.

### Models

We've been thinking for the whole time that anything we were manipulating were just models. Model of checkbox, model of blog post, of SQL record, or relation graph; however, we were struggling a ton of times when we were curious on how to handle and marry our local state and remote state knowledge.

But it is a way of thinking we brought from our experience of building the APIs.

However, when you start to ask people how they make various complex applications with internal state on server, you will get the answer: they differ state and API responses.

Usually, they use the following combination:

- knowledge layer: automatically caching wrappers around API calls with some invalidation logic. What is tricky here is that it is usually hidden.
- explicit state layer: sometimes it is [finite state machine](https://en.wikipedia.org/wiki/Finite-state_machine) or [statechart](https://en.wikipedia.org/wiki/State_diagram), sometimes it is some class with data. Sometimes - observables (RxJava, RxRuby, RxSwift, RxWhatever - you got the point) with logic encoded in its topology. Sometimes - some in-house or even ad-hoc solution, maybe even blended with other application parts.

### The solution

I think it is the time to separate state and knowledge. This is even more vital for modern web apps than logic and view separation. We need to keep in mind that some variables we use are ones that came from external system (back-end or 3rd party), and we must keep in mind they were **provided** to us. And some - are fully ours and we can manipulate them as we wish.

We should clearly understand, that some of our strings, arrays and objects are coming from state, and some - from knowledge on system. **Knowledge** is something global, something that describes the whole system - or parts of it that are available for us. Every single piece of knowledge should be labeled: you should know where this entity came from, and when it should be invalidated. **GraphQL** is nice solution for it, but you can choose or build your own. Every piece of **knowledge** should be transferable. Consider them as [DTOs](https://en.wikipedia.org/wiki/Data_transfer_object). Knowledge cannot have JS functions, or bindings to your local system - but if you need Turing-complete logic, you can transfer some Lisp-flavored object. I once had that experience, and keeping something like `{extractor: ["$fn", ["useMax"], ["return", ["ifelse", "useMax", "getMaxViewport", "getMinViewport"]]]}` felt weird, but it worked.

**State** is how you represent current application state. It is OK if it is local - but keep in mind you will probably have to bind different parts of system together. Most important things here are that you can keep functions there, and that when you grab some data from **knowledge** - e.g. you are editing blog post you already wrote - you should either copy the data, not re-use the object, or keep the diff, which is even better. Why it is better? Simple example: you have something like JIRA - with tons of fields to edit. You update one, and simultaneously someone else is altering another. If you will send the whole state to the server, you will overwrite another guy's work. If you only send your diff, you will not. Advanced version of that is called [CRDT](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type).

### So, once again: 

You are working with two worlds in your application. 

One, the **knowledge** is a reflection of something remote. You cannot download the whole DB to your browser, so you only get the parts of it. You can use **imperative** `getBlogPost(id)` or **declarative** `@gql("blogPosts(id){...}") class extends Component`. Both are fine, but when using declarative approach, you are hiding the ability to create complex logic you actually do not need.

You should preserve data immutable. You can use [ImmutableJS](https://immutable-js.github.io/immutable-js/), [Object.freeze](https://developer.mozilla.org/ru/docs/Web/JavaScript/Reference/Global_Objects/Object/freeze), use TypeScript's [readonly](https://www.typescriptlang.org/docs/handbook/classes.html#readonly-modifier) or just keep an eye on that. If you do that, you can even do the trick and start keeping your **knowledge** in Shared Worker or Service Worker.

Second, the **state** is your own kingdom. I personally advice to use [XState](https://xstate.js.org/) to represent any complex piece of logic (anything bigger than the counter). But you can use anything you want. Just keep it away from **knowledge**. 

Any interaction between this two worlds should me kept in userland and should be loud and clear.

I'm not limiting you on some specific libraries, it is all on architecture and way of thinking. I suddenly understood few weeks ago that I was using this approach unknowingly and like a hidden pattern, but it is the thing that should be as explicit as possible.

Give this idea a try, and you will see how your mind will slowly become lest restless.