I know I'm late to the party here, but is TRY_CACHE the semantics we want? If it falls out of Tachyon's memory, it will not be saved to disk, correct? So replication would be the only way to avoid recomputation.

By the way, does replication actually work with this scheme? I'm not 100% convinced that creating the same file from multiple nodes and writing to the FileOutputStream will actually just work. For instance, it's likely the TachyonManager will throw a FileAlreadyExistsException, no?