DoSomething/ds-mdata-responder — pull request #367 by tongxiang (2014-12-17 16:37:41+00:00)
Title: migrate JSON configs to Mongo

#### What's this PR do?
This PR a) sets up the infrastructure which allows us to store and use SMS config files (the config files which indicate the relevant MData/OIPs the campaign/game uses). It also b) ports over the **Tips** and **DonorsChoose** campaign flows entirely over to Mongo. (Note, however, that I haven't yet deleted their config files from the repo, since other users will need to import those files into their local db. Check out the notes below for directions on how to import a JSON config into Mongo.)

**WHAT SPECIFICALLY HAS CHANGED?**
1. We've refactored the DonorsChoose and Tips config files so that each individual config object is a valid JSON object.
2. We've created new config Mongoose models for both the DonorsChoose and Tips config objects.
3. We've set up our responder to use two database connections--`connectionOperations` for day-to-day responder operations database calls (for instance, creating SMS game users or game docs), and `connectionConfig` strictly for configuration database calls (fetching SMS config docs, and eventually, creating new and editing SMS config docs.) Note that we've created new files for each one of these new connections, the logic in each file prevents each connection from being reopened.
4. Given these new connections, we've refactored *all* the Mongoose models in the responder to use a specific connection.
5. Creating two database connections corresponds to the creation of two separate databases. Now, we have the `config` database which stores unique collections, each corresponding to each SMS config model.
6. We ran the `mongoimport` command (full command listed below) in order to import the `tips_config` and `donorschoose_config`JSON data into the `config` database.
7. We created a function, `smsConfigsLoader`, which is ran on app initialization to 1) fetch all the requisite sms configs from the database and 2) populate a config object with the contents of each config. The server is started after this function returns completely.
8. We also created a new global function, `app.getConfig(model, id)`, which is called anywhere the old sms config file was previously referenced. We added this reference to both the tips and donations flow code, as well as the code testing that code.
9. We also refactored the test file `_init.test.js` to make sure that the configs are properly loaded before tests are run, and we removed references to starting a new `express()` object in order to make sure tha tthe configs object persists across all the tests.

**WORKFLOW FOR PORTING OVER A CAMPAIGN TO MONGO**
1. Change config file into array, JSON-parseable object
2. add xxxxConfigModel.js file, to which connection gets passed.
3. require xxxxConfigModel.js to smsConfigsLoader.js (in configModelArray)
4. to get config file into mongo, run `mongoimport --db config --collection tips_configs < app/lib/ds-routing/config/tips.json --jsonArray` (replace `tips_configs` and `app/lib/ds-routing/config/tips.json` with the appropriate collection name and path string to the config. Note collection name is pluralized model name
i.e., model: donorschoose_config, collection: donorschoose_configs)
5. remove all references to the config file
6. replace all the config file invocations with `app.getConfig(modelName, id) function calls (note that the function uses the non-pluralized model name, not the pluralized collection name.)
7. remove references to configs in test files, add app.geConfig() function calls to test files
8. `npm test`!

#### Where should the reviewer start?
First, run the `mongoimport` commands to populate your local instance of the DB with the config files! Then, start by checking out how the `smsConfigsLoader.js` function works within `server.js`. Next, check out how the `connectionsOperations.js` and `connectionsConfig.js` connection-creators work. Third, check out the new configModels. Finally, check out how references to the sms config files have been changed, and how the `init.test.js` test file has changed!

#### How should this be manually tested?
`npm test`! I also used postman to manually test the donations flow.

#### Any background context you want to provide?
#### What are the relevant tickets?
#### Questions:
1. Would it be a good idea for us to attach the `mongoimport` command to an `npm` command in our package.json, so that when the repo is first installed, our config files are automatically converted into Mongo collections?
2. Within our `smsConfigsLoader.js` code, should we throw an exception if all the config files don't load correctly? What would be the right way to do this so that in the future, our deployment engine knows that it should restart the deployment process?
3. Within `smsConfigsLoader.js`, I had to reference a nasty `__proto__.constructor.modelName` property in order for a specific config file array to be saved in the configObject with the modelName which corresponds to it. (See screenshot below.) Is there a better way to do this?

![screenshot 2014-12-17 11 30 58](https://cloud.githubusercontent.com/assets/5678066/5474538/382d182c-85e0-11e4-80ae-b8502140f745.png)