CDK-139: Adding Configuration to DatasetDescriptor.

The last commit's addition of an optional location to the
DatasetDescriptor enables the MetadataProvider to determine the location
of Datasets and pass that location to a DatasetRepository. But, the
repository implementation was still dependent on having the correct
FileSystem. Adding the Configuration to the DatasetDescriptor as well
allows the FileSystemDatasetRepository to construct the correct
FileSystem for a descriptor, from the location URI and the
Configuration.
CDK-139: Adding list method to MetadataProvider.

If the DatasetRepository no longer determines the locations of data
sets, and instead relies on getting the location from the
MetadataProvider, then the repository can't list the available data
sets. The MetadataProvider determines what data sets are available, so
it should have the list() method. This is also mroe consistent with the
Hive implementation, where the repository already delegated to the
MetadataProvider.

The DatasetRepository interface still has exists and list methods, but
these should delegate to the MetadataProvider unless they need to be
hooked. For instance, if a Dataset has metadata but has not yet been
written to.
Clarifying assumptions in Hive MetadataProvider.

The FileSystem is now final and set in the constructors. Before, a call to save
assumed that the FileSystem was set if a dataDirectory was set, which happens
when an unmanaged HCatalogDatasetRepository creates (external) data sets.
Changes to the HCatalogDatasetRepository could easily cause a NPE by violating
these undocumented assumptions, so always setting up a FileSystem prevents
that. Also, nothing set the dataDirectory to null, so table locations weren't
read on subsequent saves.

In addition, this commit makes save more defensive. Rather than relying on
dataDirectory being null to signal that save should read the managed table's
location, it now handles both modes explicitly. In managed mode, the table is
created, read, and then dataDirectory is set (it is an outgoing field). In
unmanaged mode, the dataDirectory (an incoming field) is checked with a
Precondition at the start of the function, used to create the table metadata,
then set to null so that it doesn't leak into other save calls.
Refactoring HCatalogDatasetRepository for clarity.

Previously, both create and get would lazily create a special inline
FileSystemDatasetRepository for the purpose of opening Hive-managed Datasets
because the location of the those sets is controlled by Hive. This commit
consolidates those inline classes into a private inner class called
ExternalFileSystemDatasetRepository with docs to explain what's going on: it is
only used immediately after the MetadataProvider creates or loads because it
will only load the data set that the MP points to.

The special data set repo was previously stored in the same field as the normal
repo, used when Hive does _not_ manage the data set, which was confusing so it
is now stored in a new field: externalDatasetRepository. Both fields (the other
is fileSystemDatasetRepository) are final and initialized in the constructors.

Next, the Hive repository used to assume that all tables were in the
filesystem-backed repository that is opened in unmanaged mode (now null in
managed mode). This isn't necessarily the case, so the special external
repository is now used when loading and deleting for both modes because its
location is set from the Hive metadata when the set is loaded. An extra
consequence is that the DatsetDescriptor must be successfully loaded before it
can be deleted. With this change, the regular FS repository is only used for
creation.

This commit also removes the call to set the MetadataProvider's file system in
create. The FileSystem for the HCatalogMetadataProvider is now set in the
constructors for unmanaged mode, from the created FileSystemDatasetRepository
or from the FileSystem passed in. For managed mode, the handling has not
changed.

Known bug: in create, the FS repository determines the location of the data set
in create and calls metadataProvider.save. But the path where the data will be
stored must be set before calling save (or else the Precondition is not met).
This currently gets around the problem by guessing (accurately) where the FS
repository will put the data set, violating encapsulation.
[squash me] Initial commit (doesn't compile)
CDK-139: Updating FS impl to use Descr locations.

Now that DatasetDescriptors have getLocation and getConfiguration, the
FileSystem Dataset/Repo/Provider implementations can use the Descriptor
to pass Dataset paths. Now, the FileSystemMetadataProvider assigns a
location when creating a Dataset. FileSystemDatasetRepository uses the
paths that are set on the DatasetDescriptor.

Because location and configuration are being passed by the
DatasetDescriptor, all of the classes necessary to get a FileSystem and
Path for a FS-based Dataset are available. So this commit also removes
the FileSystem and root Path from the FS repo and the FileSystem option
from the FileSystemDataset.Builder. This makes the FS classes more
flexible and handles Configurations more cleanly: only the
MetadataProvider is instantiated with a Configuration, which is passed
via Immutable objects where it needs to go.

Several helper methods that were previously protected because they were
shared between the FS MetadataProvider and DatasetRepository are now
private and owned by the MetadataProvider.

Tests are up-to-date and passing for core, but this breaks the Hive
integration. This update is primarily to fix problems in the Hive
integration anyway.
CDK-139: Separating common tests out of FS.

This extracts tests that should pass on all DatasetRepository
implementations into the TestDatasetRepositories set. The FS tests now
inherit from this add add other tests, like making sure that the data
directory exists after a dataset is created. Minor updates to the FS
repository are included where tests found bugs.

A major update to the FS repository and common tests is the new
MemoryMetadataProvider, which is a simple provider with no durability
intended for testing. Both the common and FS tests use this provider
rather than the FS provider. MemoryMetadataProvider is tested by
TestMemoryMetadataProvider, which inherits from the new common set,
TestMetadataProviders.

The common tests include checks that verify IllegalArgumentException is
thrown, which has been added to the API documentation in the interface.
Similar additions haven't been made to the FS MetadataProvider yet.
Cleaning up DatasetReader tests.

Clean up to match the recent repo and provider tests
* Renamed TestDatasetReaderCommon to TestDatasetReaders
* Merged Test...DatasetReaderCommon and Test...DatasetReader classes
CDK-139: Updating FS MetadataProvider tests.

The FS MetadataProvider now uses the common test methods in
TestMetadataProviders. Tests are passing.
Fixing a deprecation warning in FS dataset test.

Should use the Iterator API rather than read().
[fix me] Adding mostly-working Hive implementation.

MetadataProviders are working, except that the External provider creates
MANAGED tables for some reason. Committing to check the logs on master.
Fixing TestMetadataProviders: update is optional.