Bug: NullPointerException when set 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 27/Jul/21 08:09
Affects Version/s: 2.4.0

The default value of {{hbase.hregion.memstore.mslab.indexchunksize.percent}} introduced by HBASE-24892 is 0.1, but when we use {{DefaultMemStore}} by default , which has no  {{IndexChunk}} and {{ChunkCreator.indexChunksPool}} is useless({{IndexChunk}} is only used by {{CompactingMemStore}}), so we set  {{hbase.hregion.memstore.mslab.indexchunksize.percent}} to 0 to save memory space, 
But after running a while, the {{RegionServer}} throws {{NullPointerException}}  and abort:
{code:java}
   Caused by: java.lang.NullPointerException
        at org.apache.hadoop.hbase.regionserver.ChunkCreator$MemStoreChunkPool.access$900(ChunkCreator.java:310)
        at org.apache.hadoop.hbase.regionserver.ChunkCreator.putbackChunks(ChunkCreator.java:608)
        at org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.recycleChunks(MemStoreLABImpl.java:297)
        at org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.close(MemStoreLABImpl.java:268)
        at org.apache.hadoop.hbase.regionserver.Segment.close(Segment.java:149)
        at org.apache.hadoop.hbase.regionserver.AbstractMemStore.clearSnapshot(AbstractMemStore.java:251)
        at org.apache.hadoop.hbase.regionserver.HStore.updateStorefiles(HStore.java:1244)
        at org.apache.hadoop.hbase.regionserver.HStore.access$700(HStore.java:137)
        at org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:2461)
        at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2963)
{code}

The problem is caused by line 608 in {{ChunkCreator.putbackChunks}} : {{Chunk.isIndexChunk}} incorrectly returns true for {{DataChunk}}  and
unexpectedly invokes {{indexChunksPool.putbackChunks}}， while {{indexChunksPool}}  is null: 

{code:java}
594 synchronized void putbackChunks(Set<Integer> chunks) {
595    // if there is no pool just try to clear the chunkIdMap in case there is something
596    if (dataChunksPool == null && indexChunksPool == null) {
597      this.removeChunks(chunks);
598      return;
599    }
600
601   // if there is a pool, go over all chunk IDs that came back, the chunks may be from pool or not
602    for (int chunkID : chunks) {
603     // translate chunk ID to chunk, if chunk initially wasn't in pool
604      // this translation will (most likely) return null
605      Chunk chunk = ChunkCreator.this.getChunk(chunkID);
606     if (chunk != null) {
607        if (chunk.isFromPool() && chunk.isIndexChunk()) {
608          indexChunksPool.putbackChunks(chunk);
{code}

For {{DataChunk}} , {{Chunk.isIndexChunk}} return true because  {{Chunk.isIndexChunk}}  determines the type of {{chunk}} based on {{Chunk.size}}
{code:java}
 boolean isIndexChunk() {
    return size == ChunkCreator.getInstance().getChunkSize(ChunkCreator.ChunkType.INDEX_CHUNK);
  }
{code}

and {{ChunkCreator.getChunkSize}} incorrectly return {{DataChunk}} size when {{ChunkCreator.indexChunksPool}} is null:
{code:java}
    int getChunkSize(ChunkType chunkType) {
    switch (chunkType) {
      case INDEX_CHUNK:
        if (indexChunksPool != null) {
          return indexChunksPool.getChunkSize();
        }
      case DATA_CHUNK:
        if (dataChunksPool != null) {
          return dataChunksPool.getChunkSize();
        } else { // When pools are empty
          return chunkSize;
        }
      default:
        throw new IllegalArgumentException(
                "chunkType must either be INDEX_CHUNK or DATA_CHUNK");
    }
  }
{code}
In my opinion, in addition to erroneous implementation of {{ChunkCreator.getChunkSize}}, we would better not determine the type of {{Chunk}} based on {{Chunk.size}}, because
{{hbase.hregion.memstore.mslab.indexchunksize.percent}} is set by user and the size of {{IndexChunk}} and {{DataChunk}} could be the same.Tagged a {{ChunkType}} to {{Chunk}} is a better choice.