guardian/skeleton — pull request #19 by janua (2013-11-21 15:52:04+00:00)
Title: One config file

TLDR; We are moving all of the `config.json` files into one big `config.json` file.

---

After starting the recent notification and fast flowing news work, I have run into a few problems.

When a collection updates, we need to update this collection in the agents in `Facia`. Since the `Collections` in `Facia` are stored as a `List[(Config, Collection)]`, with no form of caching or memoization, there is a bit of redundancy where we could potentially be storing the `Collection` twice (or more!). This works perfect currently, but now that we must update one collection, we would have to search all collections to see if it needs updating. So now we have moved to a more singular view of `Collections`, in the form of a big `Map` used as a cache.
But this introduces new problems;
How do you update one collection, if the collection you are updating has a different `contentApiQuery`in two different `configs.json` (E.g. Imagine `uk/config.json` and `us/config.json` referred to `uk/news/special-story`, but the tail was different in each because of a different `contentApiQuery` in each `config.json`)

The logical answer to this is to add the restriction of having one `contentApiQuery` per `Collection` by adding the `contentApiQuery` to the collection, but this isn't where it should live. It would make this Skeleton repo less relevant as you would not be able to blanket update the `contentApiQuery` easily without affecting what was inside the curated lists.

Another solution is to have another place to store this longer term data for each collection, for example, maybe a `meta.json` beside `collection.json`. But this causes A LOT more requests.

We have decided to put all the `config.json` files into one big `config.json` file at the root of `config`, and inside this define the collections that are in use. Then for each `Front`, have references to each collection.

This is good:
- One request for the `config.json` file in `Facia`
- Restriction of one `contentApiQuery` per collection (This is a good thing)
- Keeps this `Skeleton` repo still relevant.
- Less requests (!)

I didn't need to write all this, but I thought it would be good to put it down somewhere as to why we moved to this structure.