av9ash/GitBugs huggingface dataset:
Bug: Client metrics are missing response and request size data when using netty
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 11/Aug/23 20:47

CallStats.setResponseSizeBytes and setRequestSizeBytes are only called in BlockingRpcConnection. So the rpcCallResponseSizeBytes_ and rpcCallRequestSizeBytes_ values are all zeroes when using netty (our default). We might need to add some calls to it in NettyRpcDuplexHandler.
Bug: [hbase-thirdparty] Bump thirdparty dependencies
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 19/Sep/23 02:40

protobuf 3.21.12 -> 3.24.3
netty 4.1.87.Final -> 4.1.97.Final
guava 31.1-jre -> 32.1.2-jre
error_prone 2.18.0 -> 2.21.1
jetty 9.4.50.v20221201 -> 9.4.52.v20230823
jersey 2.38 -> 2.40
javassist 3.25.0-GA -> 3.29.2-GA
jackson 2.14.1 -> 2.15.2
Bug: Provide the switch to avoid reopening regions in the alter sync command
Status: Resolved | Priority: Major | Resolution: Invalid
Created: 29/Sep/23 16:52
Affects Version/s: 2.0.0-alpha-1

As part of the sub-task, as HBase 2 supports both Async and Sync API, this task is to add this support/feature to HBase 2's Sync API.
Bug: Port the switch to avoid reopening regions in the alter async in HBase 2
Status: Resolved | Priority: Major | Resolution: Invalid
Created: 29/Sep/23 16:59
Affects Version/s: 2.0.0-alpha-1

As part of the sub-task, as HBase 2 supports both Async and Sync API, this task is to port the feature added in HBase 3 alter(async default) layer to HBase 2's async side. 

There is a separate sub-task for adding it to HBase 2's sync side.
Bug: Reduce seeks from beginning of block in StoreFileScanner.seekToPreviousRow
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 24/Aug/23 04:01

Currently, for non-RIV1 DBE encodings, each call to [StoreFileScanner.seekToPreviousRow|https://github.com/apache/hbase/blob/89ca7f4ade84c84a246281c71898543b6161c099/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java#L493-L506] (a common operation in reverse scans) results in two seeks: 
 # Seek from the beginning of the block to before the given row to find the prior row
 # Seek from the beginning of the block to the first cell of the prior row

So if there are "N" rows in a block, a reverse scan through each row results in seeking past summation from i=1 to N (2(i-1)) rows.

 

This is a particularly expensive operation for tall tables that have many rows in a block.

 

By introducing a state variable "previousRow" to StoreFileScanner, I believe that we could modify the seeking algorithm to be:
 # Seek from the beginning of the block to before the given row to find the prior row
 # Seek from the beginning of the block to before the row that is before the row that was just seeked to (i.e. 2 rows back). _Save_ this as a hint for where the prior row is in "previousRow"
 # Reseek from "previousRow" (2 rows back from start) to 1 row back from start (to the actual previousRow)

Then the rest of the calls where a "previousRow" is present, you just need to seek to the beginning of the block once instead of twice, i.e. 
 # seek from the beginning of the block to right before the beginning of your "previousRow" marker. Save this as the new "previousRow" marker
 # Reseek to the next row (i.e. your previous "previousRow" marker)

 

If there are "N" rows in a block, a reverse scan from row N to row 0 results in seeking past approximately summation from i=1 to N (i-1) rows i.e. 50% less than the current behavior.

 

See the attached diagrams for the current and proposed behavior.
Bug: TLSv1.3-compatible cipher suites are not used by default
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 29/Sep/23 18:52
Affects Version/s: 2.5.5

The X509Util class has a system for choosing the cipher suites to support on TLS connections, if {{hbase.rpc.tls.ciphersuites}} is not provided. It also allows you choose what protocol you want via the {{hbase.rpc.tls.enabledProtocols}} config. If {{hbase.rpc.tls.enabledProtocols}} is set to {{{}TLSv1.3{}}}, and {{hbase.rpc.tls.ciphersuites}} is not set, the user of X509Util cannot form any working TLS connections.

This is because all the cipher suites chosen by X509Utils are pre-TLSv1.3 suites, and so are rejected during connection handshakes. [TLSv1.3 requires or suggests support|https://www.rfc-editor.org/rfc/rfc8446#section-9.1] for these suites, none of which are shared in common with TLSv1.2:
 * {{TLS_AES_256_GCM_SHA384}}
 * {{TLS_CHACHA20_POLY1305_SHA256}}
 * {{TLS_AES_128_GCM_SHA256}}

Of these, the intersection of [BoringSSL|https://github.com/google/boringssl/blob/master/ssl/test/runner/cipher_suites.go] and [Java 11+|https://docs.oracle.com/en/java/javase/11/docs/specs/security/standard-names.html#jsse-cipher-suite-names] support {{TLS_AES_128_GCM_SHA256}} and {{{}TLS_AES_256_GCM_SHA384{}}}, so those should be added to the defaults in X509Util. BoringSSL and the JVM are the two crypto providers used here.
Bug: Configurably use scanner timeout as rpc timeout for scanner next calls
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 14/Sep/23 13:31

In the AsyncTable implementation, scanner next() calls use "hbase.client.scanner.timeout.period" as the rpc timeout. The reason is described in comments, and makes a lot of sense:
{code:java}
// As we have a call sequence for scan, it is useless to have a different rpc timeout which is
// less than the scan timeout. If the server does not respond in time(usually this will not
// happen as we have heartbeat now), we will get an OutOfOrderScannerNextException when
// resending the next request and the only way to fix this is to close the scanner and open a
// new one. {code}
The branch-2 HTable implementation still uses the old behavior – the next() call passes the read rpc timeout as the rpc timeout, and uses the scanner timeout period as the operation timeout. This can lead to the above behavior.

It would be nice to provide users a migration path to AsyncTable's behavior, in the form of a config flag which causes HTable to use "hbase.client.scanner.timeout.period" as rpc timeout for next() calls like AsyncTable does.
Bug: Adding capability to move_servers_rsgroup for all servers without explicitly passing servers
Status: Open | Priority: Major | Resolution: 
Created: 28/Aug/23 03:35

In case of requirement for moving entire servers from a region server to a destination server,  currently hbase client has to get list of servers from a rs group and then pass these servers for movement as below.

{code:bash}
  hbase> move_servers_rsgroup 'dest',['server1:port','server2:port']
{code}

However idea should be to move all servers by specifying source and destination rs group and the list servers can be handled internally be hbase code instead of hbase client code/shell.

Expected:
 {code:bash}
  hbase> move_all_servers_rsgroup 'dest','src'
{code}
Bug: [HBCK2] extraRegionsInMeta need supoort delete extra regions when table not exist
Status: Open | Priority: Major | Resolution: 
Created: 28/Sep/23 07:52
Affects Version/s: 2.4.14

Some times, we delete one table dir on hdfs, we need use hbck2 to fix region info in hbase:meta...

but some times, we use hbase shell delete the table state uncarefully，just like execute:

deleteall 'hbase:meta','t1,xxx' (it not prevented...)

then the table state lose

when we want to use extraRegionsInMeta to remove these unuseful regions in meta...but it failed,beacuse the table is not exist..

I think we should support do extraRegionsInMeta when table not exists...or there are other method to fix it?
Bug: Add more comments to explain why replication log queue could never be empty for normal replication queue
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Sep/23 08:14

In HBASE-28037, [~Xiaolin Ha] found that there could be a very small window that even for a normal replication source, its queue could be empty.

This is because we will only enqueue the wal file to the queue in postLogRoll, where the old WAL writer has already been closed, so if the replication is fast enough, we could reach the end of the queue before enqueuing the new wal file.

The code for branch-2+ has been refactored a lot so we opened a new issue for fixing this.
Bug: [hbase-thirdparty] Release hbase-thirdparty 4.1.5
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 19/Sep/23 02:38

Netty 4.1.97.Final is out, which includes our TLS improvements https://github.com/netty/netty/pull/13551 contributed by [~bbeaudreault]. Let's upgrade the dependencies and make a new thirdparty release.
Bug: TableRequests metrics broken after altering table
Status: Open | Priority: Major | Resolution: 
Created: 29/Sep/23 20:46

Alerting a table reopens all regions. I noticed after altering one of my tables that afterwards the TableRequests metrics no longer update. They seem stuck at whatever they were prior. I'm thinking this has something to do with the refactor done in https://issues.apache.org/jira/browse/HBASE-27681, which attempts to deregister metrics when a table is removed from the server. I think when it gets re-registered the new metrics aren't linked to whatever the jmx adapter is looking at.

[~tianhang] FYI
Bug: procedureWAL can not delete after bypass procedure recurisive
Status: Open | Priority: Major | Resolution: 
Created: 09/Aug/23 06:41
Affects Version/s: 2.2.7

We found 1.2T logs in /hbase1/MasterProcWALs, because WALProcedureStore can not delete wal logs after bypass parent procedure recurisive. 

When bypass TRSP recurisive, TRSP and ORP/CRP will run on race. TRSP call store.delete for ORP/CRP, then ORP/CRP release lock and call store.update to override delete. 

 
{code:java}
1. OPR bypassing and upd wal
EntryType=PROCEDURE_WAL_UPDATE
pid=241, ppid=240, state=RUNNABLE, bypass=true; org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure submittedTime=1691398235997, lastUpdate=1691398236007
stackIndexes=[2]
2. TRSP bypassing and upd wal by OPR
EntryType=PROCEDURE_WAL_UPDATE
pid=240, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, bypass=true; TransitRegionStateProcedure table=t4, region=57f33b87c08a532869d4eaf9b16e9162, ASSIGN submittedTime=1691398235706, lastUpdate=1691398235997
stackIndexes=[0, 1]
3. TRSP bypassing and upd wal by self
EntryType=PROCEDURE_WAL_UPDATE
pid=240, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, bypass=true; TransitRegionStateProcedure table=t4, region=57f33b87c08a532869d4eaf9b16e9162, ASSIGN submittedTime=1691398235706, lastUpdate=1691398235997
stackIndexes=[0, 1]
4. ORP running, get lock and upd wal 
EntryType=PROCEDURE_WAL_UPDATE
pid=241, ppid=240, state=RUNNABLE, bypass=true; org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure submittedTime=1691398235997, lastUpdate=1691398236007
stackIndexes=[2]
6. TRSP bypassing, set status and upd wal
EntryType=PROCEDURE_WAL_UPDATE
pid=240, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_OPENED, bypass=true; TransitRegionStateProcedure table=t4, region=57f33b87c08a532869d4eaf9b16e9162, ASSIGN submittedTime=1691398235706, lastUpdate=1691398251253
stackIndexes=[0, 1]
7. ORP running end, upd wal
EntryType=PROCEDURE_WAL_UPDATE
pid=241, ppid=240, state=SUCCESS, bypass=true; org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure submittedTime=1691398235997, lastUpdate=1691398251253
stackIndexes=[2, 3]
8. TRSP running end, delete wal for child
EntryType=PROCEDURE_WAL_DELETE
pid=240, state=SUCCESS, bypass=true; TransitRegionStateProcedure table=t4, region=57f33b87c08a532869d4eaf9b16e9162, ASSIGN submittedTime=1691398235706, lastUpdate=1691398251260
stackIndexes=[0, 1, 4]
9. TRSP release lock and upd wal
EntryType=PROCEDURE_WAL_UPDATE
pid=240, state=SUCCESS, bypass=true; TransitRegionStateProcedure table=t4, region=57f33b87c08a532869d4eaf9b16e9162, ASSIGN submittedTime=1691398235706, lastUpdate=1691398251260
stackIndexes=[0, 1, 4]
10. ORP release lock and upd wal
EntryType=PROCEDURE_WAL_UPDATE
pid=241, ppid=240, state=SUCCESS, bypass=true; org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure submittedTime=1691398235997, lastUpdate=1691398251253
stackIndexes=[2, 3]
11. TRSP delete wal in compled
EntryType=PROCEDURE_WAL_DELETE
EntryType=PROCEDURE_WAL_DELETE {code}
Bug: Snapshot corruptions due to non-atomic rename within same filesystem
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 23/Aug/23 21:35

Sequence of events that can lead to snapshot corruptions:
 # Create snapshot using admin command
 # Active master triggers async snapshot creation
 # If the snapshot operation doesn't complete within 5 min, client gets exception

{code:java}
org.apache.hadoop.hbase.snapshot.SnapshotCreationException: Snapshot 'T1_1691888405683_1691888440827_1' wasn't completed in expectedTime:600000 ms   {code}
 # Client initiates snapshot deletion after this error
 # In the snapshot completion/commit phase, the files are moved from tmp to final dir.
 # Snapshot delete and snapshot commit operations can cause corruption by leaving incomplete metadata:

 * [Snapshot commit] create "/hbase/.hbase-snapshot/T1_1691888405683_1691888440827_1/.snapshotinfo"
 * [Snapshot delete from client]  delete "/hbase/.hbase-snapshot/T1_1691888405683_1691888440827_1/.snapshotinfo"
 * [Snapshot commit]  create "/hbase/.hbase-snapshot/T1_1691888405683_1691888440827_1/data-manifest"

 

The changes introduced by HBASE-21098 performs atomic rename for hbase 1 but not for hbase 2
{code:java}
  public static void completeSnapshot(Path snapshotDir, Path workingDir, FileSystem fs,
    FileSystem workingDirFs, final Configuration conf)
    throws SnapshotCreationException, IOException {
    LOG.debug(
      "Sentinel is done, just moving the snapshot from " + workingDir + " to " + snapshotDir);
    URI workingURI = workingDirFs.getUri();
    URI rootURI = fs.getUri();
    if (
      (!workingURI.getScheme().equals(rootURI.getScheme()) || workingURI.getAuthority() == null
        || !workingURI.getAuthority().equals(rootURI.getAuthority())
        || workingURI.getUserInfo() == null //always true for hdfs://{cluster}
        || !workingURI.getUserInfo().equals(rootURI.getUserInfo())
        || !fs.rename(workingDir, snapshotDir)) //this condition isn't even evaluated due to short circuit above
        && !FileUtil.copy(workingDirFs, workingDir, fs, snapshotDir, true, true, conf) // non-atomic rename operation
    ) {
      throw new SnapshotCreationException("Failed to copy working directory(" + workingDir
        + ") to completed directory(" + snapshotDir + ").");
    }
  } {code}
whereas for hbase 1
{code:java}
// check UGI/userInfo
if (workingURI.getUserInfo() == null && rootURI.getUserInfo() != null) {
  return true;
}
if (workingURI.getUserInfo() != null &&
    !workingURI.getUserInfo().equals(rootURI.getUserInfo())) {
  return true;
}
 {code}
this causes shouldSkipRenameSnapshotDirectories() to return false if workingURI and rootURI share the same filesystem, which would always lead to atomic rename:
{code:java}
if ((shouldSkipRenameSnapshotDirectories(workingURI, rootURI)
    || !fs.rename(workingDir, snapshotDir))
     && !FileUtil.copy(workingDirFs, workingDir, fs, snapshotDir, true, true, conf)) {
  throw new SnapshotCreationException("Failed to copy working directory(" + workingDir
      + ") to completed directory(" + snapshotDir + ").");
} {code}
Bug: Compaction queue should dedupe or consolidate queued requests
Status: Open | Priority: Major | Resolution: 
Created: 25/Sep/23 19:04

We had a compaction storm due to a huge bulkload. We ended up in a state where each regionserver had hundreds of compactions queued, despite only serving < 100 regions each. As writes continued, flushes happened, triggering minor compactions that got added to the queue as well. This results in a situation where we can never catch up on compactions because we keep compacting the same regions over and over.

It would be good if the compaction queue could automatically consolidate or deduplicate requests. For example, if a region has 5 compactions queued, just run 1 major compaction and clear the other 5 from the queue.
Bug: Upgrade BouncyCastle to fix CVE-2023-33201
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 16/Sep/23 19:38

HBase has a dependency on BouncyCastle 1.70 which is vulnerable with [CVE-2023-33201|https://nvd.nist.gov/vuln/detail/CVE-2023-33201]

Advisory: [https://github.com/bcgit/bc-java/wiki/CVE-2023-33201]

This JIRA's goal is to fix the following:
 * Upgrade to v1.76, the latest version.
 ** This requires  bcprov-jdk15on to be replaced with bcprov-jdk18on
 ** See [https://www.bouncycastle.org/latest_releases.html]
 *** 
{quote}*Java Version Details* With the arrival of Java 15. jdk15 is not quite as unambiguous as it was. The *jdk18on* jars are compiled to work with *anything* from Java 1.8 up. They are also multi-release jars so do support some features that were introduced in Java 9, Java 11, and Java 15. If you have issues with multi-release jars see the jdk15to18 release jars below.

*Packaging Change (users of 1.70 or earlier):* BC 1.71 changed the jdk15on jars to jdk18on so the base has now moved to Java 8. For earlier JVMs, or containers/applications that cannot cope with multi-release jars, you should now use the jdk15to18 jars.
{quote}
 * Exclude bcprov-jdk15on from everywhere else to avoid conflicts with bcprov-jdk18on
Bug: NPE in QuotaCache if Table is dropped from cluster
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 13/Jul/23 16:33

{code:java}
java.lang.NullPointerException: Cannot invoke "org.apache.hadoop.hbase.client.RegionStatesCount.getOpenRegions()" because the return value of "java.util.Map.get(Object)" is null        
  at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.updateQuotaFactors(QuotaCache.java:378) ~[hbase.jar]        
  at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:224) ~[hbase.jar]        
  at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) ~[hbase.jar]        
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]        
  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?]        
  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?]        
  at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) ~[hbase.jar]        
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]        
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]        
  at java.lang.Thread.run(Thread.java:833) ~[?:?] {code}
 

This is on 2.5.2, so the line numbers don't perfectly match with current branch-2. But the problem is [here|https://github.com/apache/hbase/blob/branch-2/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaCache.java#L376]. We may have cached a TableName in tableQuotaCache, and the table may have since been dropped from the cluster. In which case, tableRegionStatesCount which comes from ClusterMetrics, will not have that table.

We need to protect the lookup with a containsKey or null check, rather than directly call getOpenRegions() on the result. It probably makes sense to remove the tableName from the tableQuotaCache in this case.

While here, I think catching IOException in that block no longer makes sense. We can probably remove after verifying.
Bug: protobuf-java core and lite versions prior to 3.21.7, 3.20.3, 3.19.6 and 3.16.3 is vulnerable to CVE-2022-3509
Status: Open | Priority: Major | Resolution: 
Created: 20/Sep/23 04:57
Affects Version/s: 2.4.13

protobuf-java 3.23.2  causing increasing disk usage. Because of this we are not in position to upgrade protobuf to latest version.

we are currently using hbase 2.4.13 which uses protobuf version older than 3.23.2 which is vulnerable to cve.

is there any latest hbase and hadoop version available which is using protobuf version 3.23.2?
Bug:  The size that is checked against the maxfilesize threshold is the uncompressed size of the HFile
Status: Open | Priority: Major | Resolution: 
Created: 20/Sep/23 05:32

HBase server is configured to use Snappy compression.when doing bulkload in HBase, the size that is checked against the maxfilesize threshold is the uncompressed size of the HFile, not the compressed size.

HFileOutputFormat2.class

 
{code:java}
//代码占位符
new RecordWriter<ImmutableBytesWritable, V>() {
....
@Override
public void write(ImmutableBytesWritable row, V cell) throws IOException {
...


}

}
 {code}
!image-2023-09-20-12-09-49-959.png!
Bug: Clarify scanner timeout docs in book
Status: Open | Priority: Major | Resolution: 
Created: 19/Sep/23 18:37

There is a bit more nuance to scanner timeouts than is covered in [https://hbase.apache.org/book.html#config_timeouts]. As of HBASE-28085, one can enable a config to have similar semantics between AsyncTable and Table, and prepare for 3.0 release. We should improve these docs a bit.
Bug: Too long namespace/table/column name cause CreateTableProcedure stuck
Status: Open | Priority: Major | Resolution: 
Created: 18/Sep/23 16:06
Affects Version/s: 2.4.13

Too long namespace/table/column name cause CreateTableProcedure stuck

Due to the limitations of DFSConfigKeys.DFS_NAMENODE_MAX_COMPONENT_LENGTH_KEY, we cannot create directory if the the length of compoent exceeds this value.

1. Too long table name cause CreateTableProcedure is Running always and it would not end
2. We can create namespace if namespace is too long, however we will encounter the same problem as situation 1 if we create table with this namespace
3. we will also encounter the same problem as situation 1 if we create table with long column family name. Although HBase requires that the length of column family cannot exceed the Byte.MAX_VALUE, We may still fail if we set the value of DFSConfigKeys.DFS_NAMENODE_MAX_COMPONENT_LENGTH_KEY to be less than Byte.MAX_VALUE
Bug: Debian hbase package
Status: Open | Priority: Major | Resolution: 
Created: 14/Sep/23 19:37

Hi team,

There doesn't seem to be an hbase package in debian any longer.  Our instructions on how to use bigtable on our product depend on hbase.  I am build-from-source logic into the installer, but would prefer to install a package with apt-get instead.  Can assist with generating debian/rules

C.J.
Bug: Unhandled TableExistsException and NamespaceExistException in BackupSystemTable
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 12/Sep/23 19:36

Most backup-related operations instantiate a BackupSystemTable object. The first thing it does is check that the {{backup}} namespace and backup-related tables exist, and create them if not. One way in which this can be triggered is via LoadIncrementalHFiles, which can load files with a configurable number of threads. When bulkLoadHFile is called, a preCommitHFile hook is triggered which does the above. With a certain level of concurrency, one can easily trigger a race condition on creating the namespace or tables. This exception bubbles up and fails the bulkload.

We should catch TableExistsException and NamespaceExistException in BackupSystemTable initialization so that these do not cause client pain.