Sorry, I wasn't too clear above. That issue of corrupted partitioned tables
has been addressed in the PR linked in the comments below.
These comments about the MemoryMetadataManager workaround address the
concern that reloading entire tables on repeated CACHE calls is redundant
if the first call works correctly (given the fix for partitioned tables).
On Feb 26, 2014 10:48 AM, "sundeepn" <notifications@github.com> wrote:

> In src/main/scala/shark/memstore2/MemoryMetadataManager.scala:
>
> > @@ -54,10 +54,16 @@ class MemoryMetadataManager extends LogHelper {
> >        databaseName: String,
> >        tableName: String,
> >        cacheMode: CacheType.CacheType): MemoryTable = {
> > -    val tableKey = MemoryMetadataManager.makeTableKey(databaseName, tableName)
> > -    val newTable = new MemoryTable(databaseName, tableName, cacheMode)
> > -    _tables.put(tableKey, newTable)
> > -    newTable
> > +	val tableKey = MemoryMetadataManager.makeTableKey(databaseName, tableName)
> > +    // Clear out any existing tables with the same key; prevent memory leak
> > +    if (containsTable(databaseName, tableName)) {
> > +      logInfo("Attempt to create new table when one already exists - " + tableKey)
>
> That feature does not work. Here is what fails for me:
>
> Create external partitioned table
> add partition
> cache table
> add partition
>
> the second partition is not in memory; and when I query the table, I get
> partial and incorrect results. i.e. I only get back incorrect results from
> the first partition and no error to the user. Its more than a
> exception/messaging issue.
>
> --
> Reply to this email directly or view it on GitHub<https://github.com/amplab/shark/pull/295/files#r10092827>
> .
>