av9ash/GitBugs huggingface dataset:
Bug: Marker interface for Cells which may have backing ByteBuffers
Status: Open | Priority: Major | Resolution: 
Created: 17/Jun/24 06:10

We often need to treat cells that may have backing ByteBuffers differently.
The easy thing to would be checking if they are ByteBufferExtendedCell , but CellWrapper , OnHepDecodedCell and TagRewriteCell may also delegate to a ByteBufferExtendedCell.

Having a marker interface that indicates that a cell is guaranteed to be on-heap, or that it may not be fully on-heap would make it easier and faster to check whether we need to clone them. (We only need one, it may be either negative or positive)
Bug: Support non-SPNEGO authentication methods and implement session handling in REST java client library
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 08/Apr/24 08:39

The current java client only supports the SPENGO authentication method.

This does not support the case when an application proxy like Apache Knox performs AAA conversion from BASIC/DIGEST to kerberos authentication.

Add support for BASIC username/password auth the client.

Generally, the authentication code in the client looks quite backwards, it seems that most of the kerberos / auth cookie code duplicates HttpClient functionality. AFAICT setting HttpClient up (or letting user set it up) , and letting it handle authentication by itself would be a better and more generic solution.

-Also add support for specifying a prefix for the URL path.-
Bug: Make shell commands support column qualifiers with colons
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 24/Apr/24 03:38

Revisiting abandonded HBASE-13788.

h2. Problem

Shell commands do not support column qualifiers with colons (which are actually quite common in practice) because the part after the first colon is always treated as a converter expression.

This can be too restrictive because:
# Converters are only used for {{get}} and {{scan}} commands. They are not supported anyway for other mutating commands such as {{put}}, {{delete}}, {{incr}}, etc.
# Converter expression should follow a specific pattern. It should either be a method name of the {{Bytes}} class, or should be in {{c(CLASS).METHOD}} format. We ignore the part after the first colon even if it does not follow the pattern.

h2. Suggested solution

I suggest applying two approaches to make the commands support column qualifiers with colons.

# Do not interpret column qualifiers when using commands that don't support converters.
# If the part after the first colon does not follow the pattern, treat it as a part of the column qualifier

h2. Counterargument

Depending on how you see it, this makes the commands inconsistent in how they handle column qualifiers. For example, a user may want to use the same column expression throughout the commands.

{code}
create 't1', 'cf'

col = 'cf:cq:toLong'

# Expecting incr command to automatically ignore :toLong part
incr 't1', 'r1', col, 1

get 't1', 'r1', COLUMNS => [col]
{code}

However, if we see the converter as an option that is supported by only a few commands, passing it to a command that doesn't support it can be considered to be a user error. {{help 'put'}} or {{help 'delete'}} don't mention anything about converters.

h2. Alternative solution

An alternative solution would be to add a global switch that disables the converter interpretation altogether.
Bug: Merge of incremental backups fails if backups are on a separate FileSystem
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 22/Apr/24 07:30
Affects Version/s: 2.6.0

When the backups are stored on a location that is not the DistributedFilesystem underpinning HBase itself merging of incremental backups fails. Detected with backups stored on S3A, but can be reproduced with any other (like LocalFilesystem).

Attached is a patch with a proposed fix and a unit test that reproduces the issue.
Bug: FilterListWithAND can swallow SEEK_NEXT_USING_HINT
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/May/24 18:01

org.apache.hadoop.hbase.filter.FilterListWithAND.filterRowKey(Cell) will return true if ANY of the filters returns true for Filter#filterRowKey().

However, the SEEK_NEXT_USING_HINT mechanism relies on filterRowKey() returning false, so that filterCell() can return SEEK_NEXT_USING_HINT.

If none of the filters matches, but one of them returns true for filterRowKey(), then the  filter(s) that returned false, so that they can return SEEK_NEXT_USING_HINT in filterCell() never get a chance to return SEEK_NEXT_USING_HINT, and instead of seeking to the hint, FilterListWithAND will do very slow full scan.
Bug: BackupHFileCleaner.loadHFileRefs is very expensive
Status: Open | Priority: Major | Resolution: 
Created: 19/Apr/24 21:31

I noticed some odd CPU spikes on the hmasters of one of our clusters. Turns out it had been getting lots of bulkoads (30k) and processing them was expensive. The method scans hbase and then parses the paths. Surprisingly the parsing is more expensive than the reading hbase, with the vast majority of time spent in org/apache/hadoop/fs/Path.<init>.

We should see if this is possible to be optimized. Attaching profile.
Bug: Use Base64.getUrlEncoder().withoutPadding() in REST tests
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 30/May/24 04:42

The encoder returned by java.util.Base64.getUrlEncoder() is unsuitable for the purpose.

To get an encoder that is actually usable in URLs, java.util.Base64.getUrlEncoder().withoutPadding() must be used.

The relevant Java bug is https://bugs.openjdk.org/browse/JDK-8026330 , however instead of fixing the encoder, Java has decided to keep the broken default, and add the .withoutPadding()  method as a way to get a working one.

Due to sheer luck (or rather bad luck), this is not triggered in our tests, but anyone using them as a template will be in for a ride when hit by this problem.
Bug: Using JDK17 resulted in regionserver reportForDuty failing
Status: Resolved | Priority: Major | Resolution: Information Provided
Created: 30/May/24 08:23
Affects Version/s: 2.1.1

I am currently using HBASE-2.1.1 version to adapt to JDK17 and have encountered some issues: 1) Java. lang. NoSuchFiledException: modifiers; 2) Unable to make static boolean Java.nio Bits.unaligned() accessible: module java.base does not "opens java.nio" to unnamed moudle； 3) Regionserver HRegionServer: error telling master we are up。 Problem 1 is solved through HBASE-25516 [JDK17] reflective access Field. class. getDeclaredField ("modifiers") not supported - ASF JIRA (apache. org). Problem 2 is solved by adding – add open Java. base/Java. lang=ALL-UNNAMED – add open Java. base/Java. lang. reflect=ALL-UNNAMED – add open Java. base/Java. nio=ALL-UNNAMED parameter. However, there is currently no idea for problem 3. How can I handle this. now master is running  normally.

regionserver:

!image-2024-05-30-16-23-34-561.png!

master:

!image-2024-05-30-17-02-18-965.png!
Bug: Add ErrorProne ban on Hadoop shaded thirdparty jars
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/May/24 15:09

Over on HBASE-28568 we got tripped up because we pulled in the shaded Guava provided by Hadoop. This wasn't noticed until the backport to branch-2, which builds against hadoop-2. We should make this a compile time failure.
Bug: [hbase-operator-tools] setRegionState log should match actual region state
Status: Open | Priority: Major | Resolution: 
Created: 21/May/24 14:18

 
{code:java}
hbase:007:0> list_regions 'ns1:ot1'
                                  SERVER_NAME |                                                REGION_NAME |  START_KEY |    END_KEY |  SIZE |   REQ |   LOCALITY |
 -------------------------------------------- | ---------------------------------------------------------- | ---------- | ---------- | ----- | ----- | ---------- |
<rshost>,16020,1716198252550 |   ns1:ot1,,1716198398262.4fee86bfd0585b9e8431bc2f8148b11b. |            |         10 |     0 |     0 |        0.0 |  
<rshost>,16020,1716198252550 | ns1:ot1,10,1716198398262.ebec0c22088e9486c044458389de5e16. |         10 |         20 |     0 |     0 |        0.0 | 
<rshost>,16020,1716198252550 | ns1:ot1,20,1716198398262.fbd628f29f99d5e1ccc73abbbafc5f62. |         20 |         30 |     0 |     0 |        0.0 | 
<rshost>,16020,1716198252550 | ns1:ot1,30,1716198398262.621f63c9aebf16614bdaa4c8b22bd681. |         30 |         40 |     0 |     0 |        0.0 | 
<rshost>,16020,1716198252550 | ns1:ot1,40,1716198398262.4a3d75653f14bbbf90dda0db1a1eddcc. |         40 |            |     0 |     0 |        0.0 |{code}
{code:java}
hbase hbck -j <hbase-hbck2-jar> setRegionState 4a3d75653f14bbbf90dda0db1a1eddcc CLOSED {code}
With this command, CLI shows the state is changed to CLOSED

 

 
{code:java}
Changed region 4a3d75653f14bbbf90dda0db1a1eddcc STATE from OPEN to CLOSED {code}
However same region state in Hbase UI is still OPEN.

 

Next time when I run same command again, it changes from CLOSED to CLOSED

 
{code:java}
Changed region 4a3d75653f14bbbf90dda0db1a1eddcc STATE from CLOSED to CLOSED{code}
Bug: RowTooBigException is thrown when duplicate increment RPC call is attempted
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 16/May/24 11:45
Affects Version/s: 2.5.5

*Issue:*
`RowTooBigException` is thrown when a duplicate increment RPC call is attempted.

*Expected Behavior:*
1. The initial RPC increment call should time out for some reason.
2. The duplicate RPC call should be converted to a GET request and fetch the result that I am trying to increment.
3. The result should contain only the qualifier that I am attempting to increment.

*Actual Behavior:*
1. The initial RPC increment call timed out, which is expected.
2. The duplicate RPC call is converted to a GET request but fails to clone the qualifier into the GET request.
3. Hence, the GET request attempts to retrieve all qualifiers for the given row and columnfamily, resulting in a `RowTooBigException`.

*Steps to Reproduce:*
1. Ensure a row with a total value size exceeding `hbase.table.max.rowsize` (default = 1073741824) exists.
2. Nonce property should be enabled `hbase.client.nonces.enabled` which is actually defaulted to true.
3. Attempt to increment a qualifier against the same row.
4. In my case, I am using a postIncrement co-processor which may cause a delay (longer than the RPC timeout property).
5. A duplicate increment call should be triggered, which tries to get the value rather than increment it.
6. The GET request actually tries to retrieve all the qualifiers for the row, resulting in a `RowTooBigException`.

*Insights:*
Upon further debugging, I found that qualifiers are not cloned into the GET instance due to incorrect usage of [CellScanner.advance|https://github.com/apache/hbase/blob/7ebd4381261fefd78fc2acf258a95184f4147cee/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3833]

*Fix Suggestion:*
Removing the `!` operation from `while (!CellScanner.advance)` may resolve the issue.

Attached Exception Stack Trace for reference.
Bug: Finish 2.6.0 release
Status: Resolved | Priority: Major | Resolution: Done
Created: 17/May/24 14:00

# Release the artifacts on repository.apache.org
 # Move the binaries from dist-dev to dist-release
 # Add xml to download page (via HBASE-28236)
 # Push tag 2.6.0RC4 as tag rel/2.6.0
 # Release 2.6.0 on JIRA [https://issues.apache.org/jira/projects/HBASE/versions/12353291]
 # Add release data on [https://reporter.apache.org/addrelease.html?hbase]
 # Send announcement email
Bug: Incremental backup set does not correctly shrink
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 06/May/24 12:39
Affects Version/s: 2.6.0

The logic in BackupAdminImpl#finalizeDelete does not properly clean up tables from the incrementalBackupTableSet (= the set of backups to include in every incremental backup).

This can lead to backups failing.

 

Minimal example to reproduce from source:
 * Add following to `conf/hbase-site.xml` to enable backups:

{code:java}
<property>
    <name>hbase.backup.enable</name>
    <value>true</value>
  </property>
  <property>
    <name>hbase.master.logcleaner.plugins</name>
    <value>org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner,org.apache.hadoop.hbase.backup.master.BackupLogCleaner</value>
  </property>
  <property>
    <name>hbase.procedure.master.classes</name>
    <value>org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager</value>
  </property>
  <property>
    <name>hbase.procedure.regionserver.classes</name>
    <value>org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager</value>
  </property>
  <property>
  <name>hbase.coprocessor.region.classes</name>
  <value>org.apache.hadoop.hbase.backup.BackupObserver</value>
</property>
  <property>
    <name>hbase.fs.tmp.dir</name>
    <value>file:/tmp/hbase-tmp</value>
  </property> {code}
 * Start HBase: {{bin/start-hbase.sh}}
 * 
{code:java}
echo "create 'table1', 'cf'" | bin/hbase shell -n
echo "create 'table2', 'cf'" | bin/hbase shell -nbin/hbase backup create full file:/tmp/hbasebackups -t table1
bin/hbase backup create full file:/tmp/hbasebackups -t table2
bin/hbase backup create incremental file:/tmp/hbasebackups
# Deletes the 2 most recent backups
bin/hbase backup delete -l $(bin/hbase backup history | head -n1  | tail -n -1 | grep -o -P "backup_\d+"),$(bin/hbase backup history | head -n2  | tail -n -1 | grep -o -P "backup_\d+")
bin/hbase backup create incremental file:/tmp/hbasebackups -t table1
bin/hbase backup history{code}

 * Output shows the incremental backup still includes table2, this should only be table1:

{code:java}
{ID=backup_1715000053763,Type=INCREMENTAL,Tables={table2,table1},State=COMPLETE,Start time=Mon May 06 14:54:14 CEST 2024,End time=Mon May 06 14:54:16 CEST 2024,Progress=100%}
{ID=backup_1715000031407,Type=FULL,Tables={table1},State=COMPLETE,Start time=Mon May 06 14:53:52 CEST 2024,End time=Mon May 06 14:53:54 CEST 2024,Progress=100%}
{code}

PR will follow soon.

(Edited: my original ticket included a stacktrace of an IllegalStateException from a PR for HBASE-28562)
Bug: NPE for writer object access in AsyncFSWAL#closeWriter
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 16/May/24 06:34
Affects Version/s: 2.4.18

Observed NPE during execution of some of the UT cases.


Exception is happening in AbstractFSWAL#closeWriter when trying to put null writer object in inflightWALClosures map.

Need to add null check for writer object in AsyncFSWAL#doShutdown function before its usage.
Bug: Support native Cell format in REST server and client
Status: Open | Priority: Major | Resolution: 
Created: 16/May/24 06:18

REST currently uses its own (outdated) CellSetModel format for transferring cells.

This is fine for XML and JSON, which are slow anyway and even slower handling byte arrays, and is expected to be used in cases where a simple  client code which does not depend on the hbase java libraries is more important than raw performance.

However, we perform the same marshalling and unmarshalling when we are using protobuf, which doesn't really add value, but eats up resources.

We could add a new encoding for Results which uses the native cell format, by simply dumping the binary cell bytestreams into the REST response body.

This should save a lot of resources on the server side, and would be either faster, or the same speed on the client.

As an additional advantage, the resulting Cells would be of native HBase Cell type instead of the REST Cell type.
Bug: SSLContext not used for Kerberos auth negotiation in rest client
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 26/Apr/24 05:09

The included REST client now supports specifying a Trust store for SSL connections.
However, the configured SSL library is not used when the Kerberos negotation is performed by the Hadoop library, which uses its own client.

We need to set up the Hadoop auth process to use the same SSLContext.
Bug: Backup manifest of full backup contains incomplete table list
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 08/Apr/24 10:55
Affects Version/s: 2.6.0

Noticed that the {{BackupManifest#getTableNames}} is returning only a single table instead of the complete list of tables that were requested for the backup in case of a full backup, in case of an incremental backup the manifest table list seem complete.

Checking the {{TableBackupClient#addManifest}} method shows why:
 * While looping over the included tables the manifest is stored per table and the comment mentions something about storing the manifest with the table directory:

{code:java}
// Since we have each table's backup in its own directory structure,
// we'll store its manifest with the table directory.
for (TableName table : backupInfo.getTables()) {
  manifest = new BackupManifest(backupInfo, table);
  ArrayList<BackupImage> ancestors = backupManager.getAncestors(backupInfo, table);
  for (BackupImage image : ancestors) {
    manifest.addDependentImage(image);
  }

  if (type == BackupType.INCREMENTAL) {
    // We'll store the log timestamps for this table only in its manifest.
    Map<TableName, Map<String, Long>> tableTimestampMap = new HashMap<>();
    tableTimestampMap.put(table, backupInfo.getIncrTimestampMap().get(table));
    manifest.setIncrTimestampMap(tableTimestampMap);
    ArrayList<BackupImage> ancestorss = backupManager.getAncestors(backupInfo);
    for (BackupImage image : ancestorss) {
      manifest.addDependentImage(image);
    }
  }
  manifest.store(conf);
}{code}
 * but the manifest path is based on the backup root dir and backup id, so it is not on a table directory level: 
{code:java}
Path manifestFilePath = new Path(HBackupFileSystem.getBackupPath(backupImage.getRootDir(), backupImage.getBackupId()), MANIFEST_FILE_NAME);{code}

 * so each call to {{manifest.store(conf)}} is just overwriting the same manifest file
 * for incremental backups the "complete" manifest is stored as well with {{manifest.store(conf)}} and using the exact same path, so that explains why it is correct for incremental backups:

{code:java}
// For incremental backup, we store a overall manifest in
// <backup-root-dir>/WALs/<backup-id>
// This is used when created the next incremental backup
if (type == BackupType.INCREMENTAL) {
manifest = new BackupManifest(backupInfo);
// set the table region server start and end timestamps for incremental backup
manifest.setIncrTimestampMap(backupInfo.getIncrTimestampMap());
ArrayList<BackupImage> ancestors = backupManager.getAncestors(backupInfo);
for (BackupImage image : ancestors) {
manifest.addDependentImage(image);
}
manifest.store(conf);
}{code}
 * the comment related to the manifest path being {{<backup-root-dir>/WALs/<backup-id>}} is incorrect

 
I created a simple test that verifies this issue [^TestBackupManifest.java] , but no idea how to fix this. Perhaps only the overall manifest file should be stored on \{{<backup-root-dir>/<backup-id> }}level, but that goes against the comments here so not sure.
Bug: Update compatibility report generator to ignore o.a.h.hbase.shaded packages
Status: Open | Priority: Major | Resolution: 
Created: 07/May/24 08:47

This is a small change that will make reviewing release candidates a little easier. Right now that compatibility report includes classes that we shade. So when we shaded upgrade 3rd party dependencies, they show up in this report as an incompatible change. Changes to these classes do not affect users so there's no reason to consider them wrt compatibility. We should update the reporting tool to exclude this package.

For example, https://dist.apache.org/repos/dist/dev/hbase/2.6.0RC4/api_compare_2.5.0_to_2.6.0RC4.html#Binary_Removed
Bug: Race condition causes MetaRegionLocationCache to never set watcher to populate meta location
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 06/May/24 00:57
Affects Version/s: 2.5.8

{{ZKWatcher#getMetaReplicaNodesAndWatchChildren()}} attempts to set a a watch on the base /hbase znode children using {{ZKUtil.listChildrenAndWatchForNewChildren()}}, but if the node does not exist, no watch gets set.

We've seen this in the test container Trino uses over at [trino/21569|https://github.com/trinodb/trino/pull/21569] , where ZK, master, and RS are all run in the same container.
The fix is to throw if the node does not exist so that {{MetaRegionLocationCache}} can retry until the node gets created.
Bug: Change the deprecation cycle for HasMasterServices and HasRegionServerServices
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 01/Apr/24 09:32

The plan is to integration all core coprocessors into core but actually the work is not fully done in 3.0.0, like AccessController, quotas, visibility labels, bulk load replication, TokenProvider, etc.

So we are not likely to remove them in 3.0.0, and since these are all internal, we are free to remove it any releases.
Bug: Use standard ConnectionRegistry and Client API to get region server list in in replication
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 14/Apr/24 11:46

This is for allowing specify cluster key without zookeeper in replication peer config.

For now, we will set a watcher on zookeeper for fetching the region server list for the remote cluster, this means we must know the zookeeper address of the remote cluster. This should be fixed as we do not want to leak the zookeeper outside the cluster itself.
Bug: Implement a region server level configuration to enable/disable data-tiering
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 18/Apr/24 12:43

Provide the user with the ability to enable and disable the data tiering feature. The time-based data tiering is applicable to a specific set of use cases which write date based records and access to recently written data.
The feature, in general, should be avoided for use cases which are not dependent on the date-based reads and writes as the code flows which enable data temperature checks can induce performance regressions.

This Jira is added to track the functionality to optionally enable region-server wide configuration to disable or enable the feature.

Thanks,

Janardhan