av9ash/GitBugs huggingface dataset:
Bug: Split procedure rollback can leave parent region state in SPLITTING after completion
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/Apr/24 15:42
Affects Version/s: 2.5.8

Depending on where the split procedure fails, SplitTableRegionProcedure rollback can leave the parent region's in memory RegionStateNode in SPLITTING after rollback is complete, when the parent region is still online on the assigned region server and according to meta. This leaves active HMaster believing that the parent region is offline according to its RegionStates, and causes subsequent procedures that require the region to be online like merge/split/move to fail to start. One workaround is to restart active HMaster to reset the in memory record of region states. 

Two example scenarios where this can happen:
 * If we get to SPLIT_TABLE_REGION_CLOSE_PARENT_REGION and the parent region has a replica which is in transition, the unassign procedure in that step is never created/rolled back and we are left with the parent region state in splitting.
 * If region quotas are enabled and a split is run for a region whose namespace is at its maximum region quota limit we will fail in SPLIT_TABLE_REGION_PRE_OPERATION with QuotaExceededException and we are left with the parent region state in splitting

The region replica case is demonstrated in TestSplitTableRegionProcedure.testRollbackForSplitTableRegionWithReplica.

To reproduce the region quota case in HBase shell:
{code:java}
> create_namespace 'test_ns', {'hbase.namespace.quota.maxregions'=> 2}
> create 'test_ns:test_table', 'f1', {NUMREGIONS => 2, SPLITALGO => 'UniformSplit'}
> region_a = <first region from list_regions 'test_ns:test_table'>
> region_b = <second region from list_regions 'test_ns:test_table'>

> split region_a, 'x'
# HMaster will report: 
pid=405, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.quotas.QuotaExceededException via master-split-regions:org.apache.hadoop.hbase.quotas.QuotaExceededException: Region split not possible for :<region_a> as quota limits are exceeded ; SplitTableRegionProcedure table=test_ns:test_table, parent=...

> merge_region region_a, region_b
ERROR: org.apache.hadoop.hbase.exceptions.MergeRegionException: org.apache.hadoop.hbase.client.DoNotRetryRegionException: <region_a> is not OPEN; state=SPLITTING

> stop_master # trigger hmaster failover 
> merge_region region_a, region_b # merge now succeeds {code}
Bug: Region quotas: Split/merge procedure rollback can lead to inaccurate account of region counts
Status: Open | Priority: Major | Resolution: 
Created: 29/Apr/24 23:17
Affects Version/s: 3.0.0-beta-2

When region quotas are enabled, HMaster keeps an in memory account of region counts through NamespaceStateManager. Region counts in NamespaceStateManager are incremented/decremented at the beginning stages of split/merge procedures, in SPLIT_TABLE_REGION_PRE_OPERATION/MERGE_TABLE_REGIONS_PRE_MERGE_OPERATION before any region is offlined. If the split/merge procedure gets rolled back after the region count change in NamespaceStateManager is made, the split/merge procedure rollback does not revert the region count change in NamespaceStateManager to reflect that the expected split/merge never succeeded. This leaves NamespaceStateManager with an inaccurate account of the number of regions, believing that there are more/less regions than actually exist.
Bug: Region quotas: Multi-region merge causes inaccurate accounting of region counts
Status: Open | Priority: Major | Resolution: 
Created: 29/Apr/24 22:31
Affects Version/s: 3.0.0-beta-2

There is support for merging more than two regions in one merge procedure with multi-region merge, but if region quotas are enabled, [NamespaceAuditor assumes that every merge is a two region merge|https://github.com/apache/hbase/blob/branch-3/hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceAuditor.java#L128-L129]. This causes an inaccurate in memory accounting of region counts in NamespaceStateManager, leading MasterQuotaManager to believe there are more regions than actually exist if multi-region merge is used. 

To demonstrate the issue:
1. Start with a table with 3 regions in a namespace with a region quota limit of 3
2. Merge all 3 regions leaving 1 region, NamespaceAuditor assumed it was a 2 region merge and believes the number of regions to be 2.
3. Split a region, number of regions is now 2, and NamespaceAuditor believe it to be 3.
4. Attempt another region split, which will fail because NamespaceAuditor believes the ns to be at its region limit of 3 when there are actually only 2 regions.
Bug: Use ZKClientConfig instead of system properties when setting zookeeper configurations
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/Apr/24 07:08

In HBASE-28340, we allow loading zookeeper configurations from hbase configurations, but then we use system properties to pass these parameters when creating zookeeper client.

For replication, we may want to use different zookeeper configurations comparing to the ones we use for starting this hbase cluster, so using system properties to pass these parameters is not suitable then.

We should make use of ZKClientConfig to pass these flags.
Bug: Bump up bouncycastle dependency from 1.76 to 1.78
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 25/Apr/24 10:12

org.bouncycastle : bcprov-jdk18on : 1.76 to be upgraded to latest  org.bouncycastle : bcprov-jdk18on : 1.78

Refer [link org.bouncycastle|https://security.snyk.io/package/maven/org.bouncycastle:bcprov-debug-jdk18on]
Bug: Adjust BlockCacheKey to use the file path instead of file name.
Status: Resolved | Priority: Major | Resolution: Won't Do
Created: 17/Apr/24 06:31

The time-based priority eviction policy relies on the presence of path in the BlockCacheKey to fetch the required metadata to check data hotness and decide whether or not to retain the block in the bucket cache.

Hence, the constructor of BlockCacheKey is adjusted to take the file path as the input parameter. The code paths that create the blockCacheKey and also the unit tests need to be adjusted to pass the path instead of file name.
Bug: Multiple issues preventing starting org.apache.hadoop.hbase.rest.PerformanceEvaluation
Status: Open | Priority: Major | Resolution: 
Created: 22/Apr/24 12:07

I am trying to run org.apache.hadoop.hbase.rest.PerformanceEvaluation.
It cannot connect to the ZK quorum specified in hbase-site.xml.

It implements the Configurable interface incorrectly.
Fixing the Configurable implementation results in connecing to ZK properly.

--host option does not work because it conflicts with --h for help
Bug: org.apache.hadoop.hbase.rest.PerformanceEvaluation does not evaluate REST performance
Status: Open | Priority: Major | Resolution: 
Created: 22/Apr/24 14:43

org.apache.hadoop.hbase.rest.PerformanceEvaluation only uses the REST interface for Admin tasks like creating tables.

All data access is done via the native RPC client, which makes the whole tool a big red herring.
Bug: Re-use ZstdDecompressCtx/ZstdCompressCtx for performance
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 04/Apr/24 19:59

The zstd documentation [recommends|https://facebook.github.io/zstd/zstd_manual.html#Chapter4] re-using context objects when possible, because their creation has some expense. They can be more cheaply reset than re-created. In {{ZstdDecompressor}} and {{{}ZstdCompressor{}}}, we create a new context object for every call to {{decompress()}} and {{{}compress(){}}}. In CPU profiles I've taken at my company, the constructor of {{ZstdDecompressCtx}} can sometimes represent 10-25% of the time spent in zstd decompression, which itself is 5-10% of a RegionServer's total CPU time. Avoiding this performance penalty won't lead to any massive performance boost, but is a nice little win.
Bug: Refactoring Data Tiering Management for Improved Extensibility and Maintainability
Status: Open | Priority: Major | Resolution: 
Created: 22/Apr/24 10:24

This Jira focuses on refactoring the Data Tiering Management module to enhance modularity and remove the Singleton pattern. The objective is to restructure the codebase for better separation of concerns and increased flexibility. This includes migrating away from the Singleton pattern in favor of a more modular approach, enabling easier integration of new data tiering types and handlers.
Bug: Missing fields in Get.toJSON
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/Apr/24 08:34

Missing fields in Get.toJSON conversion.


|Class|Whether Mapped to JSON?|add to json?|
|Get| | |
|row|Yes| |
|maxVersions|Yes| |
|cacheBlocks|Yes| |
|storeLimit|No|Yes|
|storeOffset|No|Yes|
|tr|Yes| |
|checkExistenceOnly|No|Yes|
|familyMap|Yes| |
| | | |
|Query| | |
|filter|Yes| |
|targetReplicaId|No|Yes|
|consistency|No|Yes|
|colFamTimeRangeMap|No|Yes|
|loadColumnFamiliesOnDemand|No|Yes|
| | | |
|OperationWithAttributes| | |
|attributes|partial, only ID attribute is set|Yes|
|priority|No|Yes|
Bug: [hbase-thirdparty] Finish hbase-thirdparty 4.1.7 release
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 19/Apr/24 09:33

# Release the artifacts on repository.apache.org
# Move the binaries from dist-dev to dist-release
# Push tag 4.1.7RC0 as tag rel/4.1.7
# Release thirdparty-4.1.7 on JIRA https://issues.apache.org/jira/projects/HBASE/versions/12354385
# Add release data on https://reporter.apache.org/addrelease.html?hbase
# Send announcement email
Bug: Better not use threads when parallel seek enabled and only one storescanner to seek
Status: Open | Priority: Major | Resolution: 
Created: 17/Apr/24 09:45

When parallel seek enabled, seeking through the scanners using multiple threads and waiting on the countdown lock to complete the seek on all the scanners. It would be better not to use threads when there is only one scanners to seek. Might not be significant improvement but will be useful when a region has one store file post major compaction.

{code:java}
  private void parallelSeek(final List<? extends KeyValueScanner> scanners, final Cell kv)
    throws IOException {
    if (scanners.isEmpty()) return;
    int storeFileScannerCount = scanners.size();
    CountDownLatch latch = new CountDownLatch(storeFileScannerCount);
    List<ParallelSeekHandler> handlers = new ArrayList<>(storeFileScannerCount);
    for (KeyValueScanner scanner : scanners) {
      if (scanner instanceof StoreFileScanner) {
        ParallelSeekHandler seekHandler = new ParallelSeekHandler(scanner, kv, this.readPt, latch);
        executor.submit(seekHandler);
        handlers.add(seekHandler);
      } else {
        scanner.seek(kv);
        latch.countDown();
      }
    }

    try {
      latch.await();
    } catch (InterruptedException ie) {
      throw (InterruptedIOException) new InterruptedIOException().initCause(ie);
    }
{code}
Bug: Implement eviction logic for scanners in Rest APIs to prevent scanner leakage
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 09/Apr/24 05:24

The REST API maintains a map of _ScannerInstanceResource_s (which are ultimately tracking Scanner objects).

The user is supposed to delete these after using them, but if for any reason it does not, then these objects are maintained indefinitely.

Implement logic to evict old scanners automatically.
Bug: Deprecate hbase-compression-xz
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 09/Apr/24 16:15

Refer to [https://lists.apache.org/thread/on62z40rwotrcc8w1l5n55rd4zldho5g] .

Deprecate in 2.5.x.

I will add a release note when resolving this issue.
Bug: Remove hbase-compression-xz
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 09/Apr/24 16:15

Refer to [https://lists.apache.org/thread/on62z40rwotrcc8w1l5n55rd4zldho5g] .

Deprecate in 2.5.x, remove in 2.6.

I will add a release note when resolving this issue.
Bug: Remove the need for ADMIN permissions for RSRpcServices#execRegionServerService
Status: Open | Priority: Major | Resolution: 
Created: 09/Apr/24 22:49
Affects Version/s: 2.4.17

We have introduced a new regionserver coproc within phoenix and all the permission related tests are failing with the following exception.
{noformat}
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.security.AccessDeniedException): org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user 'groupUser_N000042' (global, action=ADMIN)
	at org.apache.hadoop.hbase.security.access.AccessChecker.requireGlobalPermission(AccessChecker.java:152)
	at org.apache.hadoop.hbase.security.access.AccessChecker.requirePermission(AccessChecker.java:125)
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.requirePermission(RSRpcServices.java:1318)
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.rpcPreCheck(RSRpcServices.java:584)
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.execRegionServerService(RSRpcServices.java:3804)
	at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45016)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:415)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
	at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)
	at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)
{noformat}

This check is failing. [RSRpcServices|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3815]

{code}
  @Override
  public CoprocessorServiceResponse execRegionServerService(RpcController controller,
    CoprocessorServiceRequest request) throws ServiceException {
    rpcPreCheck("execRegionServerService");
    return server.execRegionServerService(controller, request);
  }

  private void rpcPreCheck(String requestName) throws ServiceException {
    try {
      checkOpen();
      requirePermission(requestName, Permission.Action.ADMIN);
    } catch (IOException ioe) {
      throw new ServiceException(ioe);
    }
  }
{code}

Why do we need ADMIN permissions to call region server coproc? We don't need ADMIN permissions to call all region co-procs. We require ADMIN permissions to execute some region coprocs (compactionSwitch, clearRegionBlockCache).

Can we change the permission to READ?
Bug: Implement enforcement to require Date Tiered Compaction for Time Range Data Tiering
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 09/Apr/24 15:00

The implementation should enforce the requirement of enabling Date Tiered Compaction for Time Range Data Tiering. This restriction ensures that users can fully benefit from Time Range Data Tiering functionality by disallowing its usage unless Date Tiered Compaction is enabled.
Bug: Troubleshooting the issue of the "hbase-cleanup.sh --cleanAcls" command not taking effect
Status: Patch Available | Priority: Major | Resolution: 
Created: 07/Apr/24 06:26
Affects Version/s: 2.4.15

When executing the "hbase-cleanup.sh --cleanAcls" command, it was noticed that the ACL permissions of nodes in ZooKeeper did not change. The reason is that in the hbase-cleanup.sh script, the contents of the execute_clean_acls method are as follows:
 
{code:java}
execute_clean_acls() { 
  command=$1; "$bin"/hbase org.apache.hadoop.hbase.zookeeper.ZkAclReset $command 2>&1 
} {code}
The corresponding type in the "hbase-zookeeper" module should be "org.apache.hadoop.hbase.zookeeper.ZKAclReset", so it is necessary to change "org.apache.hadoop.hbase.zookeeper.ZkAclReset" to "org.apache.hadoop.hbase.zookeeper.ZKAclReset" in the "execute_clean_acls" method.
Bug: Validate access check for each RegionServerCoprocessor Endpoint.
Status: Open | Priority: Major | Resolution: 
Created: 11/Apr/24 20:45

Currently we enforce ADMIN permissions for each regionserver coprocessor method. See HBASE-28508 for more details.
There can be regionserver endpoint implementation which don't want to enforce calling users to have ADMIN permissions. So there needs to be a way for RS coproc implementation to override the access checks, with the default being requiring ADMIN permissions.
Bug: Prompting table already exists after failing to create table with many region replications
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 01/Apr/24 15:38
Affects Version/s: 2.4.13

Reproduction steps:
{code:java}
# Create table with 65537 region replications 
# we would get errors as follow,  this step is no problem 
hbase:005:0> create 't01', 'info', {REGION_REPLICATION => 65537} 
ERROR: java.lang.IllegalArgumentException: ReplicaId cannot be greater than65535 
For usage try 'help "create"' 
Took 0.7590 seconds{code}


{code:java}
# list, and found the table does not exist, as follow 
hbase:006:0> list TABLE 
0 row(s) Took 0.0100 seconds 
=> []{code}


{code:java}
# we create this tale agin by the correct way 
# we would get message that this table already exists 
hbase:007:0> create 't01', 'info' 
ERROR: Table already exists: t01! 
For usage try 'help "create"' 
Took 0.1210 seconds {code}
 
Reason:
In the CreateTableProcedure, we update this table descriptor into HBase cluster at stage  CREATE_TABLE_WRITE_FS_LAYOUT
 
{code:java}
env.getMasterServices().getTableDescriptors().update(tableDescriptor, true); {code}
 
and then, we check if the Region Replication Count is legal at stage CREATE_TABLE_ADD_TO_META.
 
 
{code:java}
newRegions = addTableToMeta(env, tableDescriptor, newRegions);

// MutableRegionInfo.checkReplicaId 
private static int checkReplicaId(int regionId) {     
  if (regionId > MAX_REPLICA_ID) {         
    throw new IllegalArgumentException("ReplicaId cannot be greater than" + 
     MAX_REPLICA_ID);    
   }     
return regionId;
}{code}
 
 
So, we can not create the same name table by correct way after faling to create table with many region replications (exceed 65536), because the table descriptor has been updated into cluster and there is no rollback.

So i think we can check if the region replication count at stage CREATE_TABLE_PRE_OPERATION to avoid this problem
Bug: Keep entries in draining ZNode when HMaster is configured to reject decommissioned hosts
Status: Open | Priority: Major | Resolution: 
Created: 08/Apr/24 13:55

Last month we shipped a config feature to allow the HMaster to reject decommissioned hosts (see: [HBASE-28342|https://issues.apache.org/jira/browse/HBASE-28342]). After some testing internally at HubSpot, we discovered that the HMaster loses the entry of a decommissioned host from the draining ZNode when the RegionServer restarts or becomes dead.

Our proposal for this fix would be to keep entries in the draining ZNode as long as the HMaster is configured to reject decommissioned hosts.
Bug: Division by zero encountered when using BloomFilter with a misconfiguration
Status: Open | Priority: Major | Resolution: 
Created: 05/Apr/24 08:47

h2. What happened:

Got Division by 0 when trying to run org.apache.hadoop.hbase.procedure2.store.region.TestHFileProcedurePrettyPrinter#test with io.storefile.bloom.max.fold set too large.
h2. Buggy code:

In BloomFilterUtil.java:
{code:java}
  public static int optimalFunctionCount(int maxKeys, long bitSize) {
    long i = bitSize / maxKeys; // here maxKeys might be zero
    ...
  }
{code}
h2. StackTrace:
{code:java}
org.apache.hadoop.hbase.DroppedSnapshotException: region: master:store,,1.1595e783b53d99cd5eef43b6debb2682.
…
Caused by: java.lang.ArithmeticException: / by zero
	at org.apache.hadoop.hbase.util.BloomFilterUtil.optimalFunctionCount(BloomFilterUtil.java:143)
	at org.apache.hadoop.hbase.util.BloomFilterUtil.createBySize(BloomFilterUtil.java:167){code}
h2. How to reproduce:

(1) Set io.storefile.bloom.max.fold to 73830463
(2) Run test: org.apache.hadoop.hbase.procedure2.store.region.TestHFileProcedurePrettyPrinter#test
Bug: Make WAL rolling exception clear
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 23/Apr/24 22:54

Occasionally we see errors like this that doesn't really give much clue what went wrong.

{noformat}
2024-04-22 08:08:02,026 ERROR org.apache.hadoop.hbase.master.HMaster: ***** ABORTING master ccycloud-7.ozn-hb973chf3oz.xyz,22001,1713770648404: Log rolling failed *****
java.lang.RuntimeException
        at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALMetadata(AsyncProtobufLogWriter.java:217)
        at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(AsyncProtobufLogWriter.java:223)
        at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:164)
        at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116)
        at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:726)
        at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:129)
        at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:886)
        at org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(AbstractWALRoller.java:304)
        at org.apache.hadoop.hbase.wal.AbstractWALRoller.run(AbstractWALRoller.java:211)
{noformat}

In this case, it was due to a time out exception. It would be helpful to make the log message more friendly.
Bug: Avoid expensive allocation in createRegionSpan
Status: Patch Available | Priority: Minor | Resolution: 
Created: 05/Apr/24 11:32
Affects Version/s: 2.5.0

On our busy clusters, the alloc profile shows that createRegionSpan() is responsible for 15-20% of all the allocations. These allocations comes from getRegionNameAsString().

getRegionNameAsString() takes the region name and encode invisible characters in their hex representation. This requires the use of a StringBuilder and thus generate new strings every time.

This becomes really expensive on a cluster with high number of requests. We have a patch that replaced the call with getEncodedName() instead. It seems better to just take the encoded region name (the md5 part) and use that in trace attributes, because:
- it's fixed in size (the full region name can be much longer depending on the rowkey size),
- it's enough information to link a trace to a region,
- it doesn't require any new allocation.
Bug: Add build information to the REST server version endpoint
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 10/Jun/24 04:50

There is currently no way to check the REST server version / build number remotely.

The */version/cluster* endpoint takes the version from master (fair enough),
and the */version/rest* does not include the build information.

We should add a version field to the /version/rest endpoint, which reports the version of the REST server component.

We should also log this at startup, just like we log the cluster version now.

We may have to add and store the version in the hbase-rest code during build, similarly to how do it for the other components.
Bug: copy_tables_desc.rb script should handle scenarios where the namespace does not exist in the target cluster
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 10/May/24 06:40
Affects Version/s: 2.4.17

When utilizing the {{copy_tables_desc.rb}} script to duplicate tables to a target cluster, if the specified table's namespace is nonexistent in the target cluster, the script fails to execute successfully. It is recommended to incorporate logic within the script for detecting and handling scenarios where the namespace does not exist.