Merge pull request #1 from amplab/master

merge
Merge pull request #2 from amplab/master

merge from tachyon master
Merge pull request #3 from amplab/master

merge from tachyon master
Introduce fine-grained, hierarchical pinning

When a file is pinned, Tachyon will not evict it -- effectively, this
allows Tachyon to provide two-level priority caching. The only way to
configure the files that should be pinned, however, is via the
"pin list", which is determined by system properties that are set when
the Tachyon master is started, and cannot be changed at runtime or
at a finer granularity than a directory.

This PR introduces a per-Inode "pinned" flag. Only files will be pinned
in memory, but a folder's pinned flag is used as the default value
for new files and folders created inside that folder.

The pinned flag on a file or folder may be updated dynamically. Setting
the flag on a folder will cause us to recursively apply the update to
all children. This can be an expensive operation for large nodes.

A few implementation notes:
  - The pinlist has been removed.
  - InodeFile.mPin and MasterInfo.mFileIdPinList were reused, but with
    the updated semantics.

Outstanding issues:
  - This change does not provide backwards compatibility for EditLog,
    as I had to add a new field. We should have a more general solution
    that logs the Tachyon version in the EditLog so we can provide
    backwards compatibility, or else provide a script to upgrade old
    journals.
  - Use of cached ClientFileInfos can lead to unexpected behavior, because
    we have no way of pushing the fact that a file was either pinned
    or unpinned to the users. This only impacts the caching of files,
    not the eviction, as the latter always re-fetches the mFileIdPinList.
Canonicalize all host names

This is motivated by the Spark/Tachyon integration, where Spark requires
the host names for all block locations to match precisely with its view
of the universe. Since Spark does canonicalize all of its host names, it
seems reasonable for us to do so as well.
Downgrade "Master disconnected" log message to DEBUG level

This is particularly annoying during interactive shell sessions (e.g.,
spark-shell) where periods of not using Tachyon will visually interrupt
user input.
Merge pull request #217 from aarondav/fixes

Downgrade "Master disconnected" log message to DEBUG level
code format for NEtworkUtils.java
Merge pull request #216 from aarondav/canonicalAddress

Canonicalize all host names
Throw IOException instead of RuntimeException
update the delete semantics
Update web app and documentation
Merge pull request #212 from aarondav/propSetting

Introduce fine-grained, hierarchical pinning
format thrift
thrift gen
better handle worker's address
better handle TachyonFS's network address
Merge pull request #4 from amplab/master

merge from tachyon master
import class NetworkUtils
Merge pull request #218 from CrazyJvm/netutil-import

Netutil import