artsy/positron — pull request #22 by craigspaeth (2014-10-06 20:56:22+00:00)
Title: @dzucconi (cc @dylanfareed & @dblock): Replace Spooky, API v2, and halbone with an internal api that could sync to both

Here we go...

First of all let me say that I’m sorry and I appreciate all the work that has gone into Spooky and helping me work with HAL & API v2. I hesitated on doing this for a while, but I honestly think it’s for the better of the project. I’m totally willing to accept that my [NIH syndrome](http://www.wikiwand.com/en/Not_invented_here) could be setting me out to sail on to my own little Node.js island and am willing to own the responsibility & consequences of that.

Also let me preface by saying that, despite my disliking of HAL, I think API v2 & especially Spooky are two really nicely architected projects. I had no problems with Spooky’s codebase—I thought it was a really nicely designed Rails API that was a pleasure to work with. I also think API v2 is a promising successor to v1—I just personally don’t want to interface with it through a HAL client and think halbone is making life harder.

I thought it over a lot and it seems to me that there are more compelling reasons to use an internal API like this instead of using Spooky/v2 directly. Sorry for the rant—I’ll get to explaining the PR after this…

----------

HAL was making writing client code a huge pain. It seems like I’m constantly making updates to halbone or writing [awkward](https://github.com/artsy/positron/pull/5) [work arounds](https://github.com/artsy/positron/pull/20/files#diff-63284d4c10a9bf9f13c45458daba8808R70) to what is usually straight forward code. Now that I’m looking at doing more complex state stuff potentially with [React](http://facebook.github.io/react/), I’m only imagining more painful fussing like this.

I’ve considered other options to alleviate this living HAL I’m in. (Thanks @dylanfareed for the amazing pun!)

**1. Dropping positron and doing a more traditional Rails app on top of Spooky (or a separate volt-esque stack)**

This was appealing b/c of the reusability of Watt for similar styling of Writer/CMS & the chance to dive into the amazing tools Dylan has built to make this easy. However the breadth of the work for the posting tool is going to be in it’s thick-client [section tool UI](http://cl.ly/image/121Q02441K0d). I could add Backbone/React on top of Rails, but Node is simply better equipped to write higher quality & well tested Javascript (browserify/jsdom/etc.). Also I selfishly have more fun writing node/JS code, and regardless of tech I’ll probably be the primary maintainer of this app.

**2. Rewriting Spooky to be non-HAL**

This was my first instinct. However, it doesn't seem like a trivial amount of work. Along with the rewriting of Rabl views, there’s a fair amount of field names that need to be changed/minor schema modifications I have yet to embark on. I also feel articles (mostly self-contained documents) are actually well suited to a document database like Mongo. Although that’s sort of beside the point b/c I think Spooky & PG could work out just as well (or maybe better down the line)—just a little more work on the client to span out calls along the various endpoints.

The main reasons I think an internal api is more appealing than rewriting Spooky to be non-HAL is…

1. Having the backend in Positron makes it a self-contained open source project that anyone could boot up with an Artsy client id & secret. It also eases setup with less coordination between projects.
2. Embedding the API should help stability. I can easily write full-stack integration tests and since the API is coupled to the client, surprise breaking API changes are less of an issue.
3. Along with the minor view/schema updates to Spooky, I also have to build out authorization. As it stands Positron/Spooky is letting any user CRUD any articles, and I would have to build some form of Gravity syncing access control either in Spooky or as an intermediate layer in Positron. It seems simpler to just centralize it in Positron rather than coordinating b/t Gravity + Spooky.

All that being said. I am more than happy to continue using Spooky as sort of repository of normalized article data that could make it easier for SQL/Rails users to interact with. The plan currently is to sync Positron articles via background API requests to Gravity—this could just as easily include syncing to Spooky.

--------

So finally on to the code....

This introduces a somewhat-thin API layer over MongoDB. The project now has two root folders /client and /api. In /client is the Ezel app as you mostly know it now. Under /api there is an express app that mounts sub apps based on resource—currently just /articles and /users.

API model code is wrapped up in a library of persistance/validation/business transactions using [mongojs](https://github.com/mafintosh/mongojs) and Walmart’s [Joi](https://github.com/hapijs/joi) for validation.

Positron users are data fetched from Gravity API v2 and flattened along with an access token—acting like a combination of Artsy user data & session. All API endpoints require a valid Gravity access token for authentication. Authorization is handled case-by-case inside the routes.

Tests now involve api-level tests that do unit testing on models & routes, api integration tests with superagent, your usual suite of Ezel client tests, and a small set of full-stack integration tests at the root level.

Sorry for the large diff. FWIW most of it is the new shrinkwrap and tests (~400 lines of actual new api code).