Script Broker

A simple broker that executes a script on the node. This provides a
lightweight broker with no external service dependencies. It is suitable
for one-time configuration post os install, but also flexible enough to
setup more complicated long term solutions (e.g. local puppet apply).

Features:

 * Optionally send auxillary data along with the script.
 * Sends the metadata along with the script.
 * Fully configurable where the files land on the node.

Interface:

The script will be called with one parameter: The path to the metadata file.
The metadata file contains all the node metadata (including tags) and the
configuration of the script broker (including the paths to local resources). It
is JSON 'pretty' formatted and should be amiable to parsing.

Example:

  On the Razor Server:

    # cat /root/hello_world.sh
    #!/bin/bash
    set -e
    set -u

    echo "Hello World!"
    echo "Metadata Path: $1"

    # razor broker
    Broker Targets:
     Name =>  hello_world
     Description =>  Hello
     Plugin =>  script
     UUID =>  4kp0o3tchb5NW5pdcV6pRf
     Script =>  /root/hello_world.sh
     Script Path =>  /tmp/razor_script
     Data =>
     Data Path =>  /tmp/razor_script_data
     Metadata Path =>  /tmp/razor_script_metadata
     Log Path =>  /tmp/razor_script.log

  On the Node after the script broker has run:

    # cd /tmp
    # ls
    razor_complete.log  razor_script  razor_script.log  razor_script_metadata
    # cat razor_script.log
    Hello World!
    Metadata Path: /tmp/razor_script_metadata
    # cat /tmp/razor_script_metadata
    {
      "username": "root",
      "password": "electric",
      "metadata": {
        "razor_tags": "ubuntu-precise,cpus_16,virtualbox_vm,OracleCorporation,nics_1,memsize_1015MiB",
        "razor_node_uuid": "49jbMtCv4gn0BLphWfmJ8p",
        "razor_active_model_uuid": "4IfYuiaCQ1fY9D1mQUZ0Vr",
        "razor_model_uuid": "60WqyDfq9yF3HSSpLWutpf",
        "razor_model_name": "ubuntu_precise",
        "razor_model_description": "Ubuntu Precise Model",
        "razor_model_template": "linux_deploy",
        "razor_policy_count": "27"
      },
      "uuid": "49jbMtCv4gn0BLphWfmJ8p",
      "ipaddress": "10.0.1.123",
      "script": "/vagrant/echo_script.sh",
      "script_path": "/tmp/razor_script",
      "data": "",
      "data_path": "/tmp/razor_script_data",
      "metadata_path": "/tmp/razor_script_metadata",
      "log_path": "/tmp/razor_script.log"
    }
Manual Broker Retry

Allow the broker to be invoked manually via a policy callback:

    razor policy callback <active model> broker

This will trigger the broker_agent_handoff method on the policy. It
takes no arguments and simply restarts the broker handoff process. This
may be useful for testing out brokers or retrying a failed broker.

Fixes #420
Merge pull request #437 from calebcase/master

Merge arbitrary script-based broker hand-off
Fix broker plugin listing after script broker was merged

When the script broker was merged it included a new error class, specific to
the broker, that allowed differentiation of remote script errors from other
classes of error.

Unfortunately, that error class was defined at the broker level that Razor
expected to be able to enumerate only broker plugin class children -
triggering a failure in an attempt to list all available broker plugins.

This fixes that by moving the custom error class down a level, to where it no
longer trips up the metaprogramming - and adds a spec test to ensure that this
doesn't explode again.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Additional testing for other meta-programming template code

This adds tests to the other consumers of the meta-programming
`get_child_templates` method in slices - to ensure that we don't accidentally
repeat the script broker failure in one of the other areas that make
assumptions about what class children exist under a namespace.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Merge pull request #445 from daniel-pittman/bug/master/broker-plugin-list-fails-after-437-script-broker-merged

broker plugin list fails after #437 - script broker - merged
Merge `filtering` module into the one place it is used

The `ProjectRazor::Filtering` module defined a mechanism for general subset
hash comparison, used as part of object retrieval.  This ended up with a
single consumer, the `ProjectRazor::Data` object.

That made it kind of confusing to look into a distinct file when trying to
figure out the implementation - especially because the mixin nature jumped
between the two files for what, in essence, was a single class.

This simple merges the code directly into the data object, and eliminates the
separate file.  That produces the same final result with less files and
lines overall.

It also called out that the `sanitize_hash` method was duplicated between the
utility module - and, so, all classes through the `ProjectRazor::Object` base
class that everything else is derived from - and the exact duplicate
implementation in the filtering module.

As part of the move that was simply dropped; the same code is run, but now
only a single copy exists rather than the two identical copies we had before.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Remove a reimplementation of Hash#has_key? from Razor

The `find_key_match` method in the Filtering module - and, now, the data
class - was an open-coded implementation of `Hash#has_key?`.  This eliminates
it in favour of the original method.

It also eliminates the spurious temporary that could only ever have the same
value as the key we were searching on, in favor of an `unless` statement, so
that the actual purpose of the decision point was made clear.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Simplify various data store array and hash operations

This eliminates some extra array and hash allocations, in favour of streaming
Ruby constructs like `select` and `map`.  This marginally reduces the line
count, as well as substantially improving the clarity of purpose of the code.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Eliminate unused `bson_to_hash` helper

The general utility, imported indirectly to almost every class in the project,
had a `bson_to_hash` helper that was ultimately unused.  (Well, except for a
recursive call inside itself.)

This deletes it, with no functional change, but a small reduction in line
count and missing test count.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Eliminate unused `is_valid_{type,template}?` helpers

The `is_valid_type?` helper, and the alias `is_valid_template?`, were
indirectly part of the API of pretty much every class in the codebase through
their importation into the common base class.

They were also unused.  This eliminates them, giving no functional change but
a reduction in line count and test deficit.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Inline the behaviour of `is_valid_json?` in both callsites

The general utility `is_valid_json?` would perform a full JSON parse, and then
discard the results; it treated any error of any sort as a reason to ignore
the problem and carry on.

This was called in two places in a single slice, and can easily enough be
replaced by simply adding a `rescue` block that will ignore all errors.

This cuts our JSON parsing time in half, not that it was terribly significant,
and reduces the complexity of the code overall.

It also makes obvious the somewhat untoward behaviour of that code in terms of
input sanitisation - which while it doesn't fix any problems, at least makes
it clear where they are for future attention.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Make `sanitize_hash` a one-liner for clarity

The original implementation of `sanitize_hash` was not very idiomatic, and
ended up spending a lot of lines of code replicating the basic behaviour of
inject and destructuring bind; this replaces it with the obvious one-liner.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Merge `SliceUtil::Common` module into the one place it is used

The code from `SliceUtil::Common` was actually part of the base `Slice` class,
but was extracted into a separate module.

Despite that, it was extremely intimate with the behaviours of the slice
class, to the point of depending on the exact member variable structure, and
the call sequence of various components.

This merges that code directly into the base class, making it much clearer
that this is not in any meaningful way distinct code from the one consumer.

There should be no functional changes from this, simply a few lines of
boilerplate code removed and a less confusing overall layout of the code.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Eliminate `command_shift` method designed only for test stubbing

The `command_shift` method was introduced to improve the ability to stub
behaviour of the Slice base class in tests - but, ultimately, is not actually
needed as the natural behaviour is literally fine for testing.

This removes it, inlining the simple array shift in the two places it
was used.

There is no functional change, since the handful of stubs in the tests were
eliminated during the previous commit.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Eliminate several vestigial `command_array` accessors from Slice

There were several definitions of the `command_array` accessor on the `Slice`
object, one through the base object and another defined in the mixin
`SliceUtil::Common` class.

None of them were actually consistently or meaningfully used in the code,
so despite the claim that this might be useful for stubbing, it really had no
practical value.

This commit simply eliminates that code, reducing confusion, API surface area,
and mental bandwidth dealing with this code.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Inline the confusingly named `iterate_obj` method on Slice

The Slice base class offered the confusingly named `iterate_obj` method
publicly, and to all children.  This was, in fact, part of the
`add_uri_to_object_hash` method, walking the first layer of values in the map
and transforming any maps in arrays in values by adding the URI to them, also.

This replaces the two call sequence with a single call that applies the
recursive modification, which avoids future developers needing to wonder why
both existed, and what their relationship was.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Eliminate unused `get_next_arg` method

The common slice code included a `get_next_arg` method that did a pile of
bookkeeping, but that was never actually used anywhere in the code.

Most of the code either doesn't care about the data that was tracked, or
open-coded some-but-not-all of the same changes; since this is never actually
used, and the data flows are incredibly complex, but the product more or less
does what is expected the shortest path seems to be to eliminate the unused
common function.

There is no behavioural change resulting from this commit.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Eliminate unused `get_name_value_args` method

The Slice common code had a method, `get_name_value_args`, that was destined
to be a central point to standardize error handling around slice
command dispatching.

It never made it there, though: it was simply unused, and has been eliminated
from the codebase.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Eliminate unused `throw_uuid_not_allowed_error` helper method

The common slice code had a bunch of helpers that worked in conjunction with
the `get_command_map` abstraction to build a standard set of error handlers
for illegal command line syntax.

The `throw_uuid_not_allowed_error` helper was never used, however, so we can
eliminate it from the codebase.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>