CDK-238. Move PartitionKey to SPI.
Add back org.kitesdk.data.PartitionKey
CDK-456: Added PartitionStrategy to Constraints.

This adds a PartitionStrategy to Constraints and updates its
EntityPredicate and KeyPredicate classes. The entity and key predicates
can now use the partition strategy to prepare the constraints in the
constructor so they can be applied quickly, rather than looking up
constraints for schema or partition fields each call to apply.

This also updates utility methods in SchemaUtil to work with a partition
strategy. It now has code to get a schema for fields in a partiiton
strategy and to build a key schema, which is needed by the CLI and the
HBase implementation.
CDK-456: Update AvroKeySchema to use SchemaUtil#keySchema.
CDK-456: Fix assumption in HBase avro util and tests.

The tests previously assumed that the key schema field names matched the
original field names. Now that the key fields are created based on the
partition strategy, this isn't possible because the source fields can be
used multiple times. The fix is to look up the correct source field name
using the partition strategy.

This also fixes tests that assumed the names matched by adding the
matching names explicitly in the partition strategy. This is allowed for
HBase, but was broken because FieldPartitioner itself enforced that
source and partition names had to be different. This removes that
restriction and adds a test to make sure it is properly enforced for
Hive tables (non-Hive HDFS produces a warning). The enforcement was
already implemented in the Compatibility class, it just wasn't used
because FieldPartitioner was doing it.
CDK-456: Fix review items.

* Removed partition strategy arg from toKeyRanges, alignedWithBoundaries
* Added @throws to javadoc for missing partition strategy
* Made predicate combination more thorough
* Added notes about source and partition name collisions
Add DurableParquetAppender.

This appender wraps an AvroAppender and a ParquetAppender to provide
durability by writing to Avro in tandem with writing parquet files
directly. After the file is closed, the avro copy is removed in a new
cleanup phase in the appender. Any failure requires recovery using the
Avro files.

The default is to use the regular ParquetAppender and the durable
appender can be enabled by setting the descriptor property
"kite.parquet.durable-writes" to true.
CDK-492: Make durable writes default and clean up.

Durable writes should be the default when writing to parquet, and not
using them should be an optimization. This switches the
kite.parquet.durable-writes to kite.parquet.non-durable-writes and adds
the setting to the DatasetKeyOutputFormat because durable writes are not
required when using an OutputCommitter in MR where failed task attempts
are run again.

This also refactors the FS descriptor properties into a common
FileSystemProperties class and moves the isTrue method into a
DescriptorUtil class in the SPI.
CDK-511: Replace guide markdown with redirect to new guide.

This replaces all of the guide markdown files with html pages that
redirect to the new locations, except for langingpage.md, which was
present but wasn't linked from the other pages.
CDK-530: Deprecate CrunchDatasets#asSource with type.

Now that Views carry a type, this is no longer needed. Because the type
parameter matched, it was forced to be the view's type already.
Remove type parameter in CrunchDatasets.asSource in HBase test.
CDK-533: Only show loader stack trace if debug is on.

This updates where Registration calls each loader so that any exceptions
will only produce stack traces if the log level is DEBUG. It also
updates the log message to be more clear that this is not an error, it
just means that some URIs could not be loaded.
Add toAvroMap command that converts a morphline record to an Avro record that contains a Map with string keys and array values where the array values can be null, boolean, int, long, float, double, string, bytes.
CDK-535. FileSystemDatasetRepository.partitionKeyForPath breaks backwards compatibility
CDK-536: Do not call setDataModelClass in Avro 1.7.4.

The AvroSerialization.setDataModelClass method was added in 1.7.5, which
isn't available on some clusters. It is now dynamically resolved and not
called if it is not present. In 1.7.4, the only data model available is
reflect.
CDK-537: Check for empty views and return 0 splits.

This updates the FS implementation's InputFormat to check the incoming
view using isEmpty to fix the setInputPaths bug. FileInputFormat assumes
that setInputPaths is called with at least one path. MR tests have been
added for both FS and HBase.
CDK-534: Add View#isEmpty test.

This adds a default implementation in AbstractRefinableView that
instantiates a DatasetReader and returns false if that reader has at
least one record. The javadoc for the new View method notes that this
may be expensive in some cases.
CDK-529: Add Datasets methods without type argument.

This simplifies code when the type parameter doesn't matter or when the
dataset that is constructed is not used.
CDK-529: Add tests for Datasets methods without type.
CDK-529: Use GenericRecord instead of GenericData.Record.

Use the interface rather than the concrete class for generic Datasets
methods. This prevents an incompatible API change if the generic record
implementation changes in the future.