Switch credentials file for config

The file we pass in to go along with the query is no longer going to
contain just the credentials the collector will need. It should also
include some information about the backdrop service it should talk to
(base url and token).
Create an entry point to backdrop_collector

In order to unify the few collectors that we currently have we need a
single entry point to run them.

This script will pull the module to execute from the query config, in a
field called "entrypoint". It will expect a function called main in the
module which it will pass the credentials, backdrop information and the
query as arguments.

I've had to introduce a log directory as part of this commit as when
setting up the logging it will be expected to exist. This means we can
also drop the creation and deletion of the directory from the
logging_setup tests.

Thanks to the entry_points setting in setup.py this new script will be
exposed as backdrop_collector when installed into the system.
Add flag to log out to console

By default backdrop_collector will write out its logs to
log/collector.log and remove stdout/err logging. I've added a cli flag
so that when testing the collector you can see the output as you run it.
Move tests into correct namespace

As I port over the collectors it is going to get messy in the tests
directory, unless everything is properly namespaced.
Never had a README.rst so read from README.md
setup.py requirements, including test reqs

As this is supposed to be a 'real deal' python module we should
structure as such. I have moved our test requirements into the setup.py
and setup nose so that a simple `python setup.py test` should run the
tests.
Ignore .egg files

Since moving the requirements I seem to end up with a bunch of .egg dirs
in the project root. To stop anyone accidently committing these I have
ignored them.
Bring backdrop.collector.write up to date

This was still refering to 'buckets' where as now we should only be
dealing with data sets.

I have also added a static method that will accept the data-set config
that is assembled in main.py.
Add --dry-run as an argument

Sometimes when working on a collector we would like to be able to see
what is going to be submitted to backdrop but not actually execute the
POST. I've added an argument to `backdrop_collector` so that you can
cause backdrop.collector.write.DataSet.post to output to the logs rather
than executing its POST.
Update .travis.yml for new deps

I forgot to change how travis ran the tests after updating how the deps
are install and the tests run. Our travis.yml now matches the structure.
Correct the tools import namespace

During the restructure I did not change the import namespace for the
tools module. Whoops...
Console logging should be at INFO level

When switching to console logging it did not match the level of file
logging, this meant that message were not being outputted and led to my
confusion. They are now at parity.
Include the url being written to when --dry-run

It is important to know the url that would have been written to as well
as the headers and body.
Entrypoint should be called with start and end

All collectors care about the time period they should be collecting
over, so these parameters should be passed throught to the entrypoint.
Migrate backdrop-ga-collector and plugins

This is the first of our existing collectors moving into the central
backdrop-collector. I have moved both the collector and the plugins that
were extending its functionality in the same seem. I moved them both as
the plugins seemed linked to the ga collector (the only user as far as I
can see) and it was causing me some pain keeping them apart. If the
feeling is against merging the plugins too this can be revisited.

During the migration I have changed a couple of things:
  - As there is no longer a `collect.py` to kick it off I have created
    an entrypoint in backdrop/collector/ga/__init__.py so that it can be
    referenced in the query.json as backdrop.collector.ga. There seemed
    to be some unused code in __init__ so I nuked that while I was at it
  - ga.py was renamed to core.py as it seemed a bit weird having a
    ga/ga.py
  - The plugins have been moved into the backdrop.collector.ga package
    as it seemed to be the only collector they were being used by at the
    moment
  - I pulled the tests out of the plugin modules and moved them into
    sepearate test modules so that they would play nicely with setup.py
  - the ga collector now shares backdrop.collector.write.DataSet to push
    its data into backdrop. Yay for more shared code and it makes
    creating functionality like --dry-run easier
  - As part of moving to writing with DataSet I removed the JSONEncoder
    that was in the ga collector. This seemed to be doing virtually the
    same thing as the one in DataSet. The only difference was that the
    encoder DataSet used would force the time into UTC before submitting,
    which generally seems like a good thing

I've tested the merged collector with a few converted queries
(govuk_most_viewed, gov-uk-content-devices-count and
gov-uk-content-social-rate) and they all seemed to still work as
expected.

The dataType field which used to be passed into the query is now
derieved from the data-group and data-type combo that generate the url
for the collector to write to. There is a small chance that one of the
queries has a different dataType to the actual data type of the data set
that it is writing to; this could lead to inconsistency in the data as
the field is used to derive the _id of a document if no idMapping is
provided.
Migrate GA realtime collector

In comparison with the full ga collector this has been a fairly easy
migration. I've had to alter the tests a bit as Bucket has become
DataSet and the way to create one has changed. Otherwise straight
forward and tested with govuk realtime.

Unsure as to whether we really need to depend on the google api client
again as gapy does but I've left it in.
Migrate pingdom collector into backdrop-collector

The only changes I had to make were namespaces and switching to DataSet.
Allow cron input to have comments and whitespace

As our crontab configuration files grow bigger it would be good to be
able to include comments in them to explain groups and schedules.

Not allowing blank lines or lines with just whitespace has caught a few
people out when they have been working with the crontab config files.
Now with comments they seem imperative to help keep our config files
understandable.
cronjobs should always start at main.py

The custom entry script in the cron config was a work around for dealing
with subtle variations in a collector. This is now baked into the
configuration of each query and the entry point is found and run inside of
main.py.
Add crontab generating post deploy hook