Created two new tables: keywords and wordcounts.  The Keyword model represents a word from each article, and includes some analysis of that word such as its Porter stem, its Metaphone and its synonyms.  This is in order to enable more thorough query expansion, and to take some control over stemming.  The wordcounts table is used both as the join table between Article and Keyword, and also as a record of how many times a word appears in each article.  This will be useful for finding related articles to a given article, as we can use the frequency of any given keyword as a heuristic for similarity.  In addition, Searchify has been setup to also index these new fields, so that when you search the Article model it also searches through the keywords table.  Test driven, so I know the database schema works.  Next step is to write the code which populates these new tables, before finally creating new functionality based on the new data. [#31861017]
Added a module in lib/rails_nlp which scans each word in each text or string field in each article, and does some linguistic analysis on it.  There is another module in lib/rails_nlp which is a very simple wrapper for the BigHugeThesaurus API.  You need to set ENV['BHT_API_KEY'] to be a secret key for big huge thesaurus - you can get one for free on the website.  If you run the rake task 'rake admin:analysemodels' you will find it takes a very long time - the trick is to uncomment the :update_tank_indexes callback of the Article model, since it is the Searchify indexing which slows things down.  I hope to find a more robust solution to this soon.  In addition, I've commented the Gemfile since I feel slightly guilty adding all these gems without explaining what they all do.  Next step: edit the search query to use the new data we are indexing!  I look forward to the challenge of finding a search formula which gives the best results given all this data :).  Possible expansions to the Keyword table include a field for more advanced linguistic analysis / entity extraction, such is offered by OpenCalais or AlchemyAPI.  [#31861017]
edited readme, removed sensitive info, pluralised tanker indexes
edited readme, removed sensitive info, pluralised tanker indexes [#32421027]
Merge branch 'nlp' of github.com:codeforamerica/honolulu_answers into nlp
added BHT_API_KEY big huge thesaurus api key line to the sample .env file
written some test for the current functionality, moved the search results to a new view, set up routes.  Removed the home.js file and tried to fit the existing style.  Works well enough for a commit to be made - just some styling and prolems with tests which I will deal with once thsi branch is merged with the nlp changes.  [#32420975]
merging nlp into js-refactor
improved the logging of search queries, will be more useful now that we drain logs to papertrail
search now works with the query expansion module.  I imagine search will now be overly permissive - unable to test properly until I have the articles loaded into my local database.  in order to do this, i will need to write the rake database seeds file, story ID #32465277 (since this branches schema differs from anything on heroku, taps will not work).  I think this finishes [#33422541] - as soon as the seeds file is created i can deliver.  next steps are: thoroughly testing the search, experiment with boosting certain attributes.
refactored again to still use the javascript by default, and only render the seperate html index page when necessary
dumped the database into a seeds file
edited seeds file so that db:seeds:load works properly
Big merge with the js-refactor branch. Also meant to say, finishes [#32465277] [#32422541] [#31713855] [#31323457]
removed logging in the nlp module to cleanup output
tweaking search boosts and tidying up the query formatting in the controller
fixed styling issues