Allow `conform!` in commands

In order to change the client=>server API, there needs to be some backwards-compatibility in effect. This `conform!` capability will allow a command to decide which data mutations it can perform on its data before running its validation.
Fixing invalid JSON returned on some errors

Invalid JSON was being returned whenever certain errors (TypeError, ArgumentError, etc.) were raised.
Add `element` and `references` to array validation

In order to facilitate some shorthand syntax (`["abc"] => [ {"name": "abc"} ]`), arrays will need to hold datatypes besides just objects and also assure relational integrity. The type `element` (aliased as `elements`) will be used to denote a basic datatype in an array. `references` will work like it does for hash. This includes a change to hash attribute default behavior, where now if no reference name is provided, it defaults to 'name', rather than the property's name. This was done to establish a clearer convention.

Part of this change is also moving incorrectly located tests for array attributes, which were previously in the array schema spec file.
Allow shorthand in `create-policy` command

Create Policy is a representative example of the shorthand involved. This converts create-policy attributes `repo`, `task`, `broker`, and `tags` to no longer need an object/hash with just the "name" property defined. In order to preserve backward-compatibility, this command now overrides `conform!`. The method upgrades commands written in the old syntax to the new syntax.

There was a previous regression where create-policy would not create tags that specified a `rule` array. This has been added back.
Shorthand references in other commands

`create-repo` and `move-policy` need the same updates for references from just a string rather than the string in an object's `name` property.

This was a good opportunity to fix up and refine the spec tests for both commands.

Fixes https://tickets.puppetlabs.com/browse/RAZOR-192
Allow marking newly discovered nodes as 'installed'

This adds a configuration option, `protect_new_nodes`, which if set to a true
value causes all newly discovered nodes to be marked installed at the exact
same moment.

This will result in the default behaviour of Razor changing from "install
based on policy" to "install based on policy once a node reinstall is
explicitly requested", a much safer default for users.

By default Razor will continue to assume that it controls everything on your
network, and treat any newly discovered node as a candidate for immediate
installation according to policy.

https://tickets.puppetlabs.com/browse/RAZOR-152

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Automatically type-check array and object entities in validation

When an `object` or `array` helper is used for an attribute, we already know
what type the entity is -- so we can automatically set that check in the
attribute code.

This allows things like `size` checks to be applied more naturally to
`object` or `array` attributes, too, without having to manually add the
`type` check.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Automatically maintain `installed_at` field of nodes

Previously every location in the code that set the `installed` field on a node
also hand-maintained the `installed_at` field.  This replaces that with an
extension to the setter that automatically updates the value, codifying this
behaviour for all external callers.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Add `register-node` command to pre-register nodes with Razor

In order to make brownfield deployments of Razor easier we allow users to
register nodes explicitly.  This command allows you to perform the same
registration that would happen when a new node checked in, ahead of time.

In order for this to be effective the hw_info must contain enough information
that the node can successfully be matched during the iPXE boot phase.

If the node matches an existing node, in keeping with the overall policy
of commands declaring desired state, the node installed field will be
updated to match the value in this command.

The final state is that a node with the supplied hardware information,
and the desired installed state, will be present in the database,
regardless of it existing before hand or not.

https://tickets.puppetlabs.com/browse/RAZOR-151

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Fixing validation for `create-policy`

The command validation for `create-policy` was not allowing the properties `enabled` and `max_count` to be included, even though the documentation includes them.

Fixes https://tickets.puppetlabs.com/browse/RAZOR-203
Delete unpacked iso on repo deletion

When issuing the `delete-repo` command, the unpacked repo was being left on the filesystem. This change cleans up this folder, if it exists.

Fixes https://tickets.puppetlabs.com/browse/RAZOR-202
Fix retry of commands

If a command failed for a retry-able reason, an item that could not be encoded into EDN was being placed on the queue since it contained a Command object. This fixes that by preserving the original command arguments for re-queuing.
Implement `sanboot` metadata field to activate workaround

A non-trivial number of systems in the field, both virtual and physical, have
an issue where the normal, standard-defined mechanism for continuing on to
local boot after finishing the PXE boot process does not work.

While the standard says that after a clean exit the system should proceed to
the next firmware configured boot device, some machines either return to PXE
booting, or simply report a boot failure, at that point.

In these cases we can implement a work-around in our boot code: instead of
returning to the firmware, we can simply load the boot sector from the first
hard disk and execute it directly.

This works, as the firmware is never again involved, so the boot process will
proceed -- at the cost that we ignore any firmware configuration for boot
sequence, logging, etc.

This commit adds the `sanboot` metadata field to nodes or, more accurately,
checks for the presence of that field.  If it is a boolean true, the
workaround is applied to the node during local boot.

This allows users who have such damaged systems, and cannot resolve the
problem correctly through, eg, a firmware update -- perhaps because the vendor
does not supply one -- to work around the problem to some degree.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Whitespace / layout cleanup, no functional changes

This fixes a trivial whitespace and layout issue in the spec file, making no
functional changes.  Separated to clarify that.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Use positive assertions, not just "no exceptions", in spec tests

This moves to using a positive assertion that the target responds to a desired
message, rather than an implicit test where the call is made and any exception
is assumed to be "does not respond to".

This ensures the particular test is more robust against future changes, by
focusing on the exact test required.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Fix array validation element declarations without index

In changes made to "clarify" the function arguments in the array element
handling shortcut where an index was omitted, the code was changed to require
a nil value, but a default of an empty hash was supplied.

This broke the shortcut; this change restores it, as well as clarifying the
actual meaning of the content -- nothing, rather than nil, since it is an
object rather than `nil`.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Simplify authz template handling

This simplifies the original "eval" driven templates for authz information,
replacing them with a much simpler and more standard sprintf named
parameter substitution.

This is safer, faster, and generally cleaner than the original; thanks to
@lutter for the tip-off.

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Add validation / authz information to command help

This extends the help rendering system by adding the schema data, created from
our validation input, to the output.  Actually rendering the schema down into
markdown is delegated to the schema object itself, since that avoids smearing
knowledge about how the code is implemented through the help subsystem.

This only adds the core infrastructure, and adds support for generating the
access control documentation to the code.  No customization of what is output
is yet available, since no need for it was immediately identified.

https://tickets.puppetlabs.com/browse/RAZOR-196
Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Implement schema attribute documentation

This adds generated documentation to the individual attributes in the
validation system, including author-supplied documentation and information
derived from the validation data.

https://tickets.puppetlabs.com/browse/RAZOR-197

Signed-off-by: Daniel Pittman <daniel@rimspace.net>
Correct nested schema authz handling

When authz handling was enabled, I was relatively lazy, and stashed away the
data through the entire hierarchy, but only tested it at the top level.

This worked fine at the time, but caused problems when it came to
documentation because each nested schema was unable to determine if it had
legitimate access control data, or simply the default present in all schema.

This reverses tack: instead of using a default, default to nothing, and
require all commands to declare their access control policy.  This test is
applied at runtime, not "compile" or load time, as other tests are, which does
delay the error report.

On the other hand, it applies this uniformly to all commands, which means that
the only way a command without access control would escape is if we had
literally zero tests for it.

In practice, this showed up eight commands that had only the default access
control patterns -- and in one, and only one, of those cases this was
legitimate.  The remaining seven were genuine shortfalls in the product.

So, it seems like this shift not only improved the ability to document code,
and eliminated "wrong" data from object dumps, but is actually better at
tracking down developer errors than the previous approach was.

https://tickets.puppetlabs.com/browse/RAZOR-197

Signed-off-by: Daniel Pittman <daniel@rimspace.net>