Check for `/etc/razor/config.yaml` also

This updates the system to check for a config file under `/etc/razor` before
the one embedded in the razor root.  This is, generally speaking, going to be
the more correct location for configuration.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Implement Puppet Labs packaging infrastructure

This is all support data and files for our internal packaging toolchain --
while you can use this from the outside, it isn't super-focused on
enabling that.

Both RPM and Debian packages are supported.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Handle internal PL builder JRuby location

The JRuby installation on our internal build infrastructure hides away in a
non-standard path.  This updates our Rakefile to handle that much more smoothly.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Fix build platforms and destinations

This updates our internal build lists to cover the platforms we actually want
packages for.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Burn version information into shipped sources

When we produce a release from our repo, this triggers the toolchain to "burn"
the version information into the source code, avoiding any runtime logic.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Improve JRuby detection handling, and error reporting

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Don't ask bundler to retry downloads

Turns out we don't have broad access to the `--retry` flag to bundler, which
is only available in very recent versions.  Theoretically we don't need it
anyway, so removing it is the short path.

Since that exposes us to some hassles caused by unreliable mirrors upstream,
we manually implement the retrying, and hope that fixes things.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
* app.rb (/api): fix misplaced dup

When we generate the output for the commands in the entrypoint, we
erroneously modify the URL's for the command handlers in the @@commands
array. This leads to all commands having an 'id' containing the hostname
under which /api was accessed the very first time; all subsequent accesses
to /api will use that hostname, even if those accesses use a different
hostname.

As an example, if the first access is to http://localhost:8080/api, even
subsequent requests to http://razor:8080/api will use 'localhost' in the id
properties for commands.

The culprit for this behavior was that the entries in the @@commands array
were directly modified (as @@commands.dup makes a shallow copy, which is
pointless anyway since map creates a new array) With this change, the hash
for each command will not be modified, but a copy will be created and have
the id filled in appropriately.

In addition, to avoid this from happening again, we freeze the hash
describing each command.

Fixes https://tickets.puppetlabs.com/browse/RAZOR-93
Apply sanboot work-around to all virtual machines

To date almost all virtual machine engines -- VMWare, VirtualBox, KVM -- have
a firmware fault that causes them to fail to progress to the next configured
device in the boot sequence after a clean iPXE exit back to the PXE firmware.

This is extremely unfortunate, but it is actually harder for users to get at
firmware updates to address the bug for virtual machines than physical ones.

This extends the workaround from "KVM virtual machines" to "any virtual
machine", on the basis that they don't appear to suffer the "lock up" bug when
hit with sanboot to go direct to the boot sector of the first hard disk.

This does make it impossible to boot from Razor to, say, the second hard disk,
or a USB device, or whatever, unless that happens to configure as BIOS device
0x80 (eg: first hard disk), but ... that is less common than trying to boot to
the OS we just installed, I guess.

This also does nothing for users of physical hardware that suffer the same
firmware issue; we don't have enough information to construct a generic or
specific blacklist for those nodes yet.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
* doc/api.md: fix documentation for reboot-node

The argument is called 'name' not 'node'

https://tickets.puppetlabs.com/browse/RAZOR-111
* lib/razor/task.rb (all): fix a few problems

  * what should have been a list of names was a list of names for file-baed
    installers and a list of DB objects for DB based installers
  * if a task was invalid, the whole method would just throw an error;
    since the server will otherwise function happily even if a task in the
    filesystem is invalid, we will just not include such tasks in the
    returned list

Fixes https://tickets.puppetlabs.com/browse/RAZOR-109
Check that the policy name is 'simple', in particular that it is non-empty

Fixes https://tickets.puppetlabs.com/browse/RAZOR-107
create-broker command: cleanly report an error if configuration is not a Hash
Correct the default value for PE version

The default value in the documentation was wrong; this updates it to reflect
the current builds.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Use generic installer from upstream PE build

Newer PE builds have introduced a generic installer script, able to do
platform tag extraction for us.  This is much more generic than the code we
hand-managed, so we can reduce our broker install script to simply download
and delegate to that script.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Better error detection when using curl to download script

curl, by default, will return successfully even if the server sends back some
error codes that -- in practice -- we really want to fail.  This bumps it over
to exiting with error on more of those failures, and hopefully getting less
confusing apparently success.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Don't require JRuby to load the Rakefile

We depend on JRuby for one small part of the Rakefile, creating the vendor gem
bundle prior to packaging.  For the rest of the code we don't actually need
it, and raising an error if loaded with the wrong engine is
actually unhelpful.

Now we just let the commands fail later if appropriate.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Support a Secondary Node Identification Strategy:

Currently, an attempt to identify a node is made during boot using the
mac address only (potentially the UUID as well).  There is a risk,
particularly with highly integrated systems such as blades where the
NICs are onboard that, should the mainboard need to be replaced Razor
will not be able to identify the node and data loss is a possibilty
depending on the policy setup.  In this scenario, the mac and UUID would
have to be manually updated on the node in the Razor database before the
node is booted on the network and given that in large operations, tasks
such as mainboard replacement is usually carried out be a vendor tech,
lining this manual process up would be difficult.

This patch allows razor to have a second go at identifying a node using
the full compliment of facts available once the MK has loaded.  The
facts to be used are configured in the config file as regex strings. The
facts used will have to be guarenteed to have unique values across all
nodes and shouldnt change often on the node.  I have a fact that
generates a fact per partition yielding the UUID of that partition. In
testing, I configured a regex that would match any
'blockdevice_<dev>_uuid' fact.  In theory, unless the mainboard with the
dhcp_nic was replaced and then an offline modification of the filesytems
was made before Razor had seen the new MACs, Razor will identify the
machine using the partition UUIDs and update the hw_info.

The following behavior changes have been implemented:
  - /svc/boot will not result in a new node being created in the DB for
    an unknown node, it will simply result in the MK being loaded.
  - When loading the MK, the node id, in the case of a known node
    without a policy/recipe, will be added to the kernel params as a
    separate field and not attached to the register URL.
  - The '/:id' on /svc/checkin is now optional.  If it is missing, the
    secondary_lookup class method on Razor::Data::Node will be fired
    with the list of facts supplied.  The facts identified as matching
    the regexes in the config will be matched against the facts of each node
    in the DB.  If all facts match the facts of a node in the DB, the node
    is considered to be that node.  The node id is returned with the action
    in the return to the client and the hw_info and dhcp_mac is updated
    so that the standard lookup on subsequent boots will succeed.  If the
    node is not identified, a new node is created and the id of the new node
    is returned to the client.
Collapsing secondary_lookup into lookup.

This commit consolodates the 2 lookup methods on Razor::Data::Node.  It
also ensures that if no facts are configured for 'match_nodes_on_facts'
in the configuration file then the traditional workflow is unchanged. Ie

If config contains 'match_nodes_on_facts':
  - /svc/boot call will attempt to match node
  - If no match is found, booth the MK. DO NOT CREATE NEW NODE
  - MK does /svc/checkin (no ID)
  - Razor attempts to match the node based on facts
  - If no match, New node is created and then the checkin proceeds as
    that node. The id is included in the response for the client to use
    in susequent checkins
  - If a match is found, the checkin continues as that node, the hw_info
    for the node is updated and the id is sent back to the client to use
    on subsequent checkins.

If config does not contain 'match_nodes_on_facts'
  - /svc/boot call will attempt to match the node
  - If no match is found, Create a new node and send the id in the
    kernel boot params and boot the MK
  - If a match is found, do what ever the task assigned to the node
    dictates.
  - This is the tradidtional behavior.

Note that this PR depends on PR
https://github.com/puppetlabs/razor-el-mk/pull/18

Also note as in other commit, the microkerne_kernel_args sends the
register url and ID as separate params now.