Significant re-working of the userdata handling and introduction of
vendordata.

Vendordata is a datasource provided userdata-like blob that is parsed
similiarly to userdata, execept at the user's pleasure.


cloudinit/config/cc_scripts_vendor.py: added vendor script cloud config

cloudinit/config/cc_vendor_scripts_per_boot.py: added vendor per boot
    cloud config

cloudinit/config/cc_vendor_scripts_per_instance.py: added vendor per
    instance vendor cloud config

cloudinit/config/cc_vendor_scripts_per_once.py: added per once vendor
    cloud config script

doc/examples/cloud-config-vendor-data.txt: documentation of vendor-data
    examples

doc/vendordata.txt: documentation of vendordata for vendors

(RENAMED) tests/unittests/test_userdata.py => tests/unittests/test_userdata.py
      TO: tests/unittests/test_userdata.py => tests/unittests/test_data.py:
    userdata test cases are not expanded to confirm superiority over vendor
    data.

bin/cloud-init: change instances of 'consume_userdata' to 'consume_data'

cloudinit/handlers/cloud_config.py: Added vendor script handling to default
    cloud-config modules

cloudinit/handlers/shell_script.py: Added ability to change the path key to
    support vendor provided 'vendor-scripts'. Defaults to 'script'.

cloudinit/helpers.py:
    - Changed ConfigMerger to include handling of vendordata.
    - Changed helpers to include paths for vendordata.

cloudinit/sources/__init__.py: Added functions for helping vendordata
    - get_vendordata_raw(): returns vendordata unprocessed
    - get_vendordata(): returns vendordata through userdata processor
    - has_vendordata(): indicator if vendordata is present
    - consume_vendordata(): datasource directive for indicating explict
        user approval of vendordata consumption. Defaults to 'false'

cloudinit/stages.py: Re-jiggered for handling of vendordata
    - _initial_subdirs(): added vendor script definition
    - update(): added self._store_vendordata()
    - [ADDED] _store_vendordata(): store vendordata
    - _get_default_handlers(): modified to allow for filtering
        which handlers will run against vendordata
    - [ADDED] _do_handlers(): moved logic from consume_userdata
        to _do_handlers(). This allows _consume_vendordata() and
        _consume_userdata() to use the same code path.
    - [RENAMED] consume_userdata() to _consume_userdata()
    - [ADDED] _consume_vendordata() for handling vendordata
        - run after userdata to get user cloud-config
        - uses ConfigMerger to get the configuration from the
            instance perspective about whether or not to use
            vendordata
    - [ADDED] consume_data() to call _consume_{user,vendor}data

cloudinit/util.py:
    - [ADDED] get_nested_option_as_list() used by cc_vendor* for
        getting a nested value from a dict and returned as a list
    - runparts(): added 'exe_prefix' for running exe with a prefix,
        used by cc_vendor*

config/cloud.cfg: Added vendor script execution as default

tests/unittests/test_runs/test_merge_run.py: changed consume_userdata() to
    consume_data()

tests/unittests/test_runs/test_simple_run.py: changed consume_userdata() to
    consume_data()
Merge with upstream sources
Added vendordata to SmartOS
Bring back the ec2 utils, non-boto userdata/metadata reading

This replacement uses our own userdata/metadata ec2 webservice
parser that we can easily modify, it also automatically allows
for reading the ec2 userdata/metdata from files and also brings
in the usage of requests instead of boto's usage of urllib which
did not support ssl properly.
Add ec2 utils tests and httpretty requirement for http mocking
Add a maybe_json helper function
Only check for json objects instead of also arrays
Updated non-json message
provide default 'output' setting to log to /var/log/cloud-init-output.log

This has been "best practice" for quite some time, and its a common
request of "where is the output of my user-data programs".

http://askubuntu.com/questions/345344/where-are-the-logs-for-my-user-data-script-cloud-init
Require sudo package.
merge from trunk
header, comment cleanup
config/cloud.cfg: fix typo
remove vendor-scripts-per-{boot,instance,once}

I don't see a real need for these.  The intent of the 'per-boot' or
'per-instance' or 'per-once' config modules is to handle running
scripts that were already inserted into the instance.

If the vendor is doing that, then there is value in vendor-data.

Ie, they'd already modified the image, they might as well have just
put the stuff in that they wanted.
replace get_nested_option_as_list with get_cfg_by_path, improve ruparts

this makes runparts take exe_prefix and do string to list conversion
inside. that means we don't have to do it in cc_scripts_vendor.

Also, get_nested_option_as_list was essentially get_cfg_by_path anyway.
cloudinit/settings.py: add vendor_data to built in config
DataSource: remove has_vendordata and consume_vendordata, drop filters

remove apply_filter from get_vendordata.  I can't think of a good
reason to filter vendor-data per instance-id.

remove has_vendordata and consume_vendordata.
has vendordata is always "true", whether or not there is something
to operate is determined by:
 if ds.vendordata_raw()

consume_vendordata is based on config entirely.
simplify consume_vendordata, move exclusion, consume_vendordata per instance

this simplifies consume_vendordata a bit, changes consume_vendordata to
be by default "PER_INSTANCE" (like userdata).

The other thing we do here is move the exlusion to be handled when walking the
data.  The benefit of doing it then is that we can exclude part-handlers.

Without the ability to exlude part handlers, exclusion is basically useless
(since part-handlers could accomplish anything they wanted).
Fixed typos
DataSourceOpenNebula:parse_shell_config skip 'SECONDS' var if seen

SECONDS is a special variable in bash, it gets set to the time the
shell has been alive.  This would cause us to fail randomly (if the
process happened to take more than 1 second, then SECONDS would
be defined).