Allow darklaunch control over search index queue operations

This patch wires darklaunch data received from request headers into
the chef_object_db where all interaction with the search index queue
occurs. This sets us up to add feature flipped behavior for testing
new search infrastrucutre. For example, we can add code to do in-line
flatten/expand and POST to a solr4 instance while continuing to put
items on the current search index queue.

Two new xdarklaunch flags are introduced:

send_to_solr4 - this flag enables currently stubbed code for taking an
action that coincides with putting an add or delete on the queue for
indexing.

disable_send_to_rabbit - if set to true, this flag prevents the code
from sending messages to the rabbit based search indexing queue. We
would use this as part of a migration strategy once an org is on the
new search infrastructure.
Add chef_object_db:bulk_delete_from_solr/4

Exporting this new helper makes it easier to share code with EC and
encapsulates the darklaunch behavior around adding/removing items from
the solr index.
Simplify batch_reindex by extracting batch_fold helper

Introduce new helper module chefp (chef + FP = chefp) for functional
programming helpers. Currently only contains batch_fold which will
fold over a list partitioning it into user specified batches.
Add ez_reindex_direct/2 to reindex to solr without rabbitmq

The ez_reindex_direct function bulk_gets all objects of the specified
type, does flatten/expand, and POSTs the data (in batches of size
matching the bulk_get batch size) to solr.

Add helper all_ids_from_name_id_dict/1 and decompress_and_decode/1.
Add error logging and error checking to direct reindexing
Use chef_object_db:add_to_solr in named_data endpoint
Refine darklaunch control of search index queue ops via vhost

This fleshes out darklaunch control of search indexing
operations. The darklaunch flags are:

    disable_rabbit_vhost_chef: if true, DO NOT send data to the default
    /chef rabbitmq vhost.

    rabbit_vhost_chef_solr4: if true, send data to the /chef-solr4
    vhost in rabbitmq.

With latest darklaunch code (which defaults unset keys to false) this
should be safe to load even if disable_rabbit_vhost_chef is not set.

Using multiple vhosts is the easiest approach to allow us to test new
search infrastructure without impacting the live instance while
minimizng the patches needed. The downside of the approach implemented
here is that the vhosts are hard-coded, as is the mapping of
darklaunch key to vhost name.
Send query load to aux solr (ignore results) via darklaunch

If darklaunch key "query_aux_solr" is set to true, then query the
auxillary solr (for testing), ignoring the results. The default solr
server is always queried and its result always returned.

Relies on new chef_wm app config:

    {solr_urls, [{default, "http://default_solr"},
                 {aux, "http://aux_solr"}]}

The solr requests are executed in parallel and the request process
waits for both requests to finish before returning. The latency of the
slower of the two requests will be paid for by the request.
Redo darklaunch switching for testing solr via multiple vhosts

Remove toggle for disabling default vhost. For now, we will always
send data to the default rabbitmq vhost.

Introduce darklaunch key "rabbit_aux_vhost". If true, data for
indexing is sent to the auxillary rabbitmq vhost for indexing.

This relies on a new app config value to map default and aux to vhost
names:

    {rabbitmq_vhosts, [{default, <<"/chef">>},
                       {aux, <<"/chef-solr4">>}]}
chef_reindex: ez_reindex_direct_from_file

Update ez_reindex_direct to reindex all of an orgs
indexes. Use this updated function to iterate over
all of the orgnames in the file.
Correct argument order
Log results raw
Log solr1.4 -> solr4 diffs
Correct undefined to no_headers
fix dialyzer errors
rebar.confg: bump chef_index dep for dialyzer checks
Use default when no aux rabbit queue
Default darklaunch aux rabbit to false
Add lager to deps
PR Comments