Remove an unnecessary map from BufferManager