Support generic Hadoop Compatible Filesystem (short version)

This is a short version of #388.
Interesting that the NPE is in the gluster code base.  When I tried this
with local files, I saw that the configs were loaded properly, so would
need to take a closer look at gluster init.

On Mon, Oct 6, 2014 at 10:13 AM, Huamin Chen <notifications@github.com>
wrote:

> In core/src/main/java/tachyon/UnderFileSystemHdfs.java:
>
> >
> > +    // Setting Hadoop compatible filesystem (HCFS) properties in command argument is not scalable
> > +    // for some HCFS e.g. ceph, that has numerous parameters.
> > +    // Providing a hadoop core-site.xml that is required for this type of underfs.
> > +    tConf.addResource("core-site.xml");
>
> @dcapwell <https://github.com/dcapwell> here is the error. The cause of
> the error is that mapred.system.dir cannot be found in Configuration,
> while it is set in here
> https://github.com/amplab/tachyon/pull/437/files#diff-37224ab9cb80b91d69d9c708ea509debR28
>
> Running tachyon.util.UnderfsUtilsTest
> Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.354 sec
> <<< FAILURE!
> loadUnderFsTest(tachyon.util.UnderfsUtilsTest) Time elapsed: 0.354 sec <<<
> ERROR!
> java.lang.RuntimeException: java.lang.NullPointerException
> at
> org.apache.hadoop.fs.glusterfs.GlusterVolume.sameVolume(GlusterVolume.java:98)
> at
> org.apache.hadoop.fs.glusterfs.GlusterVolume.setConf(GlusterVolume.java:138)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.initialize(RawLocalFileSystem.java:92)
> at
> org.apache.hadoop.fs.glusterfs.GlusterVolume.initialize(GlusterVolume.java:80)
> at
> org.apache.hadoop.fs.FilterFileSystem.initialize(FilterFileSystem.java:86)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2316)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:90)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2350)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2332)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:369)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
> at tachyon.UnderFileSystemHdfs.(UnderFileSystemHdfs.java:71)
> at tachyon.UnderFileSystemHdfs.getClient(UnderFileSystemHdfs.java:44)
> at tachyon.UnderFileSystem.get(UnderFileSystem.java:51)
> at tachyon.UnderFileSystem.get(UnderFileSystem.java:39)
> at
> tachyon.UnderFileSystemsUtils.mkdirIfNotExists(UnderFileSystemsUtils.java:30)
> at tachyon.master.LocalTachyonMaster.(LocalTachyonMaster.java:73)
> at tachyon.master.LocalTachyonMaster.create(LocalTachyonMaster.java:135)
> at tachyon.master.LocalTachyonCluster.start(LocalTachyonCluster.java:158)
> at tachyon.util.UnderfsUtilsTest.before(UnderfsUtilsTest.java:42)
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/amplab/tachyon/pull/437/files#r18469939>.
>