av9ash/GitBugs huggingface dataset:
Bug: Clear CellScanner when replay
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 20/Oct/21 12:14
Affects Version/s: 2.4.7

RSRpcServices#mutate and RSRpcServices#multi we clear CellScanner immediately when get CellScanner from  HBaseRpcController，so need it in RSRpcServices#replay and  RSRpcServices#replicateWALEntry also.
{code}
    // rpc controller is how we bring in data via the back door;  it is unprotobuf'ed data.
    // It is also the conduit via which we pass back data.
    HBaseRpcController controller = (HBaseRpcController)rpcc;
    CellScanner cellScanner = controller != null ? controller.cellScanner(): null;
    if (controller != null) {
      controller.setCellScanner(null);
    }
{code}

if not we will see the error like the following
{code}
2021-10-18 12:04:13,971 WARN  [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16022] ipc.RpcServer: RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16022: caught: java.lang.IllegalArgumentException: ReferenceCount : 0 (expected: > 0)
	at org.apache.hbase.thirdparty.io.netty.util.internal.ObjectUtil.checkPositive(ObjectUtil.java:80)
	at org.apache.hadoop.hbase.nio.ByteBuff.checkRefCount(ByteBuff.java:70)
	at org.apache.hadoop.hbase.nio.SingleByteBuff.hasRemaining(SingleByteBuff.java:155)
	at org.apache.hadoop.hbase.codec.KeyValueCodec$ByteBuffKeyValueDecoder.advance(KeyValueCodec.java:90)
	at org.apache.hadoop.hbase.ipc.CellBlockBuilder.encodeCellsTo(CellBlockBuilder.java:191)
	at org.apache.hadoop.hbase.ipc.CellBlockBuilder.buildCellBlockStream(CellBlockBuilder.java:228)
	at org.apache.hadoop.hbase.ipc.ServerCall.setResponse(ServerCall.java:243)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:168)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
{code}
Bug: Upload mvn site artifacts for nightly build to nightlies
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 15/Oct/21 11:15

It is also pretty big and due to the strict CSP rules, the site is not fully functional on jenkins.

Sometimes we also have site build for pre commit, but it is rare and we include the url in yetus vote table, where it is not easy to replace the url with nightlies. So this issue only focus on nightly job.
Bug: Release 1.4.14
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 13/Oct/21 05:38

Per the end of this thread:

https://lists.apache.org/thread.html/r34d3fb86d667f8b3e58cbba78655733ac76e10f5883650f4910adc5c%40%3Cdev.hbase.apache.org%3E

Let's do a final 1.4.14 release and mark branch-1.4 as EOL.
Bug: Fix Bug for MultiByteBuff.put(int, byte)
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 10/Oct/21 12:18
Affects Version/s: 2.4.7

MultiByteBuff. put(int index, byte b)  has some obvious bugs, following line 732 should be getItemIndex(*index*), not getItemIndex(*limit*);
{code:java}
730    public MultiByteBuff put(int index, byte b) {
731       checkRefCount();
732       int itemIndex = getItemIndex(limit);
733       ByteBuffer item = items[itemIndex];
734       item.put(index - itemBeginPos[itemIndex], b);
735       return this;
736  }
{code}
Bug: verify hbase testing infra is up to date wrt H node renaming
Status: Open | Priority: Major | Resolution: 
Created: 06/Oct/21 13:55

Infra is renaming all the H* nodes.

https://cwiki.apache.org/confluence/display/INFRA/H+node+renaming+Proposal

do a pass through the Jenkins scripts and manually configured jobs to make sure we either do not directly name H* nodes or to update such for the rename.
Bug: Add Admin or HBaseHbck method waiting for a procedure by pid
Status: Open | Priority: Minor | Resolution: 
Created: 06/Dec/21 22:14

HBaseHbck methods like unassigns and assigns return a list of pids. It's hard to do anything with these pids because the GetProcedureResultRequest used by HBaseAdmin is not publicly accessible. It would be very helpful to expose the waitProcedureResult method used by ProcedureFuture in HBaseAdmin, or something else like that.

The use-case I have in mind is for a stuck region, where you might want to use HBaseHbck.unassigns followed by assigns. But you should not do assigns until the unassigns has finished, so we need a way to wait for the returned pids.
Bug: Update ref guide section for IT and Chaos Monkey to explain the additions from HBASE-26556
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 14/Dec/21 21:49

HBASE-26556 introduced a customisable monkey factory for the slow deterministic policy, as well as made possible for pluggable implementations of the hbase remote shell commands. This is to document how these new features can be used.
Bug: IT and Chaos Monkey improvements
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 10/Dec/21 00:17
Affects Version/s: 2.3.7

While running the IntegrationTestIngest with slowDeterministic chaos monkey in our environment, we faced some issues with some of the chaos actions that required run some linux shell commands, due to the way we set hbase environment variables and start/stop scripts.

This patch allows for defining extensions of HBaseShellCommandProvider in the configuration, so that custom commands can be used. It also extends SlowDeterministicMonkeyFactory to provide the ability to configure the destructive actions to be executed.
Bug: ExportSnapshot tool failed to copy data files for tables with merge region
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 24/Dec/21 09:16

When export snapshot for a table with merge regions, we found following exceptions:
{code:java}
2021-12-24 17:14:41,563 INFO  [main] snapshot.ExportSnapshot: Finalize the Snapshot Export
2021-12-24 17:14:41,589 INFO  [main] snapshot.ExportSnapshot: Verify snapshot integrity
2021-12-24 17:14:41,683 ERROR [main] snapshot.ExportSnapshot: Snapshot export failed
org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Missing parent hfile for: 043a9fe8aa7c469d8324956a57849db5.8e935527eb39a2cf9bf0f596754b5853 path=A/a=t42=8e935527eb39a2cf9bf0f596754b5853-043a9fe8aa7c469d8324956a57849db5
    at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.concurrentVisitReferencedFiles(SnapshotReferenceUtil.java:232)
    at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.concurrentVisitReferencedFiles(SnapshotReferenceUtil.java:195)
    at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.verifySnapshot(SnapshotReferenceUtil.java:172)
    at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.verifySnapshot(SnapshotReferenceUtil.java:156)
    at org.apache.hadoop.hbase.snapshot.ExportSnapshot.verifySnapshot(ExportSnapshot.java:851)
    at org.apache.hadoop.hbase.snapshot.ExportSnapshot.doWork(ExportSnapshot.java:1096)
    at org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:154)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.hbase.util.AbstractHBaseTool.doStaticMain(AbstractHBaseTool.java:280)
    at org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:1144) {code}
Bug: Add rsgroup support for Backup
Status: In Progress | Priority: Minor | Resolution: 
Created: 02/Oct/21 14:19
Affects Version/s: 3.0.0-alpha-2

There are some places where backup needs some changes with respect to rsgroup. Some of them being addressed here are 
 # Incremental backup wal backup should happen only for servers which are part of a particular rsgroup under which namespace is configured for table backup under consideration
 # BackupLogCleaner should keep references only from those servers which are part of a particular rsgroup under which namesapce is configured for table backup under consideration
Bug: Update dependencies in hbase-connectors: HBase version to 2.4.8, and make Hadoop 3 and Spark 3 defaults
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 02/Dec/21 15:32
Affects Version/s: hbase-connectors-1.0.1

This proposes to bump up the default versions used by hbase-connectors to recent versions, in particular for HBase, Spark, Hadoop.
HBase is upgraded to 2.4.8, Spark to 3.1.2, Hadoop to 3.2.0.

This proposes also to make Spark 3 and Hadoop 3 default version (as opposed to Spark 2 and Hadoop 2).
Bug: TableSnapshotInputFormat terminates in the middle
Status: Resolved | Priority: Minor | Resolution: Duplicate
Created: 28/Dec/21 06:46
Affects Version/s: 2.2.3

I am using by adding SingleColumnValueFilter in TableSnapshotInputFormat. In fact, there is a lot of data in the snapshot, but it was found that the mapper was completed in the middle.

 

There was a problem in the next method of ClientSideRegionScanner.

 
{code:java}
 public Result next() throws IOException {
    values.clear();
    scanner.nextRaw(values);
    if (values.isEmpty()) {
      //we are done
      return null;
    }

    Result result = Result.create(values);
    if (this.scanMetrics != null) {
      long resultSize = 0;
      for (Cell cell : values) {
        resultSize += PrivateCellUtil.estimatedSerializedSizeOf(cell);
      }
      this.scanMetrics.countOfBytesInResults.addAndGet(resultSize);
      this.scanMetrics.countOfRowsScanned.incrementAndGet();
    }

    return result;
  } {code}
values is empty, but scanner.nextRaw(values) returned true.

I modified it as follows and it worked normally.
{code:java}
public Result next() throws IOException {
        values.clear();
        boolean moreValues;
        do {
            moreValues = scanner.nextRaw(values);
        } while (values.isEmpty() && moreValues);

        if (!moreValues) {
            return null;
        }

        Result result = Result.create(values);
        if (this.scanMetrics != null) {
            long resultSize = 0;
            for (Cell cell : values) {
                resultSize += PrivateCellUtil.estimatedSerializedSizeOf(cell);
            }
            this.scanMetrics.countOfBytesInResults.addAndGet(resultSize);
            this.scanMetrics.countOfRowsScanned.incrementAndGet();
        }

        return result;
    } {code}
Please check this.
Bug: [HBOSS] add lock around openFile operation
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 23/Nov/21 17:31

The HBoss FS wrapper doesn't wrap the openFile(path) call with a lock, which means anything using that builder isn't going to have access synchronized.

adding a wrapper for this method will allow hbase to use the api call and so request different read policies on different files, or other options
Bug: I can't read/write a Hbase table by spark-hbase connector when the table is  in non-default namespace
Status: Resolved | Priority: Minor | Resolution: Duplicate
Created: 26/Nov/21 09:39
Affects Version/s: 1.0.0

I found I can't read/write a Hbase table by spark-hbase connector when the hbase table is  in a non-default namespace.

 

Because when spark opens a table(related to a hbase table), it creates a HBaseRelation instance first, and initializes a HBaseTableCatalog from the table definition saved in spark catalog. But in the function 'convert' the field 'tableCatalog' is constructed from a string template, in which the namespace is set as 'default', leading to a wrong namespace. This namespace is not  the one defined when user created the table before.

 

Pls have a look:

!image-2021-11-26-17-32-53-507.png!
Bug: KeyValueScanner might not be properly closed when using InternalScan.checkOnlyMemStore()
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 02/Dec/21 08:41
Affects Version/s: 2.3.6

While writing a custom RegionObserver and using InternalScan.checkOnlyMemStore() I stumbled upon an issue. The number of files opened by region servers would grow steadily and eventually region servers would crash with
{code:java}
2021-11-15 00:54:34,290 ERROR [MemStoreFlusher.1] regionserver.HStore: Failed to commit store file hdfs://<...removed...>:8020/hbase/data/default/<...removed...>/743071139057c819d7e6f7b59f065152/.tmp/f/394ba71102ec401d8779aa5f45819f84
java.io.IOException: Failed on local exception: java.io.IOException: Too many open files; Host Details : local host is: "<...removed...>/<...removed...>"; destination host is: "<...removed...>":8020;
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:805)
        at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1544)
        at org.apache.hadoop.ipc.Client.call(Client.java:1486)
        at org.apache.hadoop.ipc.Client.call(Client.java:1385)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
        at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:800)

{code}
Another symptom is the following messages in region server logs:
{code:java}
2021-11-18 13:41:29,539 INFO [RS_COMPACTED_FILES_DISCHARGER-regionserver/<...removed...>:16020-7] regionserver.HStore: Can't archive compacted file hdfs://<...removed...>:8020/hbase/data/default/<...removed...>/ce6f08fdd82967df94d1c83e289d3142/f/9b01b8bdea324c22a92f1ba6b386e050.6261a24c6a689d5406ae1ea87dc9bb9f because of either isCompactedAway=true or file has reference, isReferencedInReads=true, refCount=7, skipping for now.
{code}
The culprit is KeyValueScanner not being closed in StoreScanner.selectScannersFrom() before `continue`

[https://github.com/apache/hbase/blame/f000b775320330eb2f426f6b2a3b5e27a794a707/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java#L467]
Bug: Prune use of Random and SecureRandom objects
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 15/Dec/21 18:24

While looking over Findbugs/Spotbugs findings on HBASE-26555 I noticed we have some violations of Spotbugs rules regarding lack of Random object re-use, and generally have a lot of static Random and SecureRandom objects in our code, especially in tests, where ThreadLocalRandom would be more than sufficient. 

This is nitty stuff but let's clean it up. It will reduce noise on the analysis reports and SecureRandom is unnecessarily slow when used wherever we don't need it.
Bug: Allow Pluggable Queue to belong to FastPath or normal Balanced Executor
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 14/Dec/21 20:45

The PluggableQueue implementation added in [https://github.com/apache/hbase/pull/3522] for https://issues.apache.org/jira/browse/HBASE-6908 is only wired up with the BalancedQueueRpcExecutor. Users should be able to decide their queue operates with the FastPathBalancedQueueRpcExecutor optionally via configuration.

I plan to submit a PR for this update soon.
Bug: IntegrationTestLoadSmallValues
Status: Resolved | Priority: Minor | Resolution: Later
Created: 27/Oct/21 15:49

This integration test emulates a use case that stores a lot of small values into a table that would likely be heavily indexed (ROW_INDEX_V1, small blocks, etc.), an application that crowdsources weather (temperature) observation data. This IT can be used to test and optimize compression settings for such cases. It comes with a companion utility, HFileBlockExtracter, which extracts block data from HFiles into a set of individual files for each block's data, for use in training external compression dictionaries, perhaps with ZStandard's 'zstd' utility. 

See javadoc on class IntegrationTestLoadSmallValues for usage.

This was used to test the changes on HBASE-26353.
Bug: Fix findbugs/spotbugs findings
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 09/Dec/21 22:12

Clean up findbugs/spotbugs findings on branch HEADs (master, branch-2, branch-2.5, branch-2.4, branch-2.3) 

For example, from recent precommits: 

Possible null pointer dereference in org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opMeterName(Object) due to return value of called method Dereferenced at MetaTableMetrics.java:org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opMeterName(Object) due to return value of called method Dereferenced at MetaTableMetrics.java:[line 230]

	Possible null pointer dereference in org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opWithClientMeterName(Object) due to return value of called method Dereferenced at MetaTableMetrics.java:org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opWithClientMeterName(Object) due to return value of called method Dereferenced at MetaTableMetrics.java:[line 210]

	Possible null pointer dereference in org.apache.hadoop.hbase.io.hfile.LruCachedBlockQueue.add(LruCachedBlock) due to return value of called method Method invoked at LruCachedBlockQueue.java:org.apache.hadoop.hbase.io.hfile.LruCachedBlockQueue.add(LruCachedBlock) due to return value of called method Method invoked at LruCachedBlockQueue.java:[line 73]

	Possible null pointer dereference in org.apache.hadoop.hbase.io.hfile.bucket.CachedEntryQueue.add(Map$Entry) due to return value of called method Dereferenced at CachedEntryQueue.java:org.apache.hadoop.hbase.io.hfile.bucket.CachedEntryQueue.add(Map$Entry) due to return value of called method Dereferenced at CachedEntryQueue.java:[line 79]
Bug: [aarch64] TestAsyncNonMetaRegionLocator is flaky
Status: Resolved | Priority: Minor | Resolution: Cannot Reproduce
Created: 18/Dec/21 02:19
Affects Version/s: 2.4.9

There was an earlier issue where the problem was diagnosed as a race (see HBASE-20822). The test is flaky again now on MacOS aarch64, which is likely a timing/race issue again. The underlying hardware memory model is different so the JVM will be using memory barriers in more places on aarch64 than x86_64 etc. and multithreaded timings and interactions can be expected to be slightly different.

org.apache.hadoop.hbase.client.TestAsyncNonMetaRegionLocator.testMultiRegionTable[1]
[ERROR]   Run 1: TestAsyncNonMetaRegionLocator.testMultiRegionTable:251->lambda$testMultiRegionTable$5:251->lambda$null$4:253->assertLocEquals:196 array lengths differed, expected.length=3 actual.length=0; arrays first differed at element [0]; expected:<49> but was:<end of array>
[INFO]   Run 2: PASS

org.apache.hadoop.hbase.client.TestAsyncNonMetaRegionLocator.testRegionMove[1]
[ERROR]   Run 1: TestAsyncNonMetaRegionLocator.testRegionMove:292->assertLocEquals:197 expected:<192.168.1.25,64070,1639793242964> but was:<192.168.1.25,64053,1639793242927>
[INFO]   Run 2: PASS

org.apache.hadoop.hbase.client.TestAsyncNonMetaRegionLocator.testRegionReplicas[1]
[ERROR]   Run 1: TestAsyncNonMetaRegionLocator.testRegionReplicas:433 expected:<3> but was:<1>
[INFO]   Run 2: PASS

org.apache.hadoop.hbase.client.TestAsyncNonMetaRegionLocator.testReload[1]
[ERROR]   Run 1: TestAsyncNonMetaRegionLocator.testReload:365->assertLocEquals:197 expected:<192.168.1.25,64083,1639793243015> but was:<192.168.1.25,64053,1639793242927>
[INFO]   Run 2: PASS

org.apache.hadoop.hbase.client.TestAsyncNonMetaRegionLocator.testSingleRegionTable[1]
[ERROR]   Run 1: TestAsyncNonMetaRegionLocator.testSingleRegionTable:205->assertLocEquals:197 expected:<192.168.1.25,64070,1639793242964> but was:<192.168.1.25,64053,1639793242927>
[INFO]   Run 2: PASS
Bug: Add metrics around failed replication edits
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 15/Dec/21 15:01

We ran into HBASE-26575 recently, but had a hard time tracking down where the edits were coming from. The only real evidence of a failure in replication was the ageOfLastShipped metric and some warnings in the logs. It would be nice to add an explicit metric around source.failedBatches and sink.failedBatches, incremented whenever an exception is encountered.
Bug: Involving primary meta region in meta scan with CatalogReplicaLoadBalanceSimpleSelector
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 23/Dec/21 01:06
Affects Version/s: 2.4.9

In the current release with Meta replica LoadBalance mode, the primary meta region is not serving the meta scan (only meta replica region serves the read). When the result from meta replica region is stale, it will go to primary meta region for up-to-date location. 

From our experience, the primary meta region serves very less read traffic, so it will be better to load balance read traffic across the primary meta region as well.
Bug: Improve documentation of hbase 1.x to 2.x potential incompatibilities
Status: Open | Priority: Minor | Resolution: 
Created: 30/Nov/21 22:46

We're working on a major upgrade of almost 900 tables across 100 production clusters (and corresponding QA environment clusters). We've upgraded about 25% of our QA environment and run into a series of incompatibilities along the way. Most of them have been easy to get around, but I wanted to create this Jira to collect them so that we can make an update to the docs for future upgraders.

My plan is to periodically edit this description to add to the list. If anyone else has anything to contribute, feel free to edit as well or add a comment. 

Incompatibilities to document:
 -  HBASE-15676 changed the serialized byte string used for the fuzzy mask. FuzzyRowFilters created by older clients will not match any rows in an hbase2 cluster. This was fixed in HBASE-26537 but should be documented in our upgrade guide.
 - CDH5 try/catches bad HTableDescriptor.getDurability calls and returns USE_DEFAULT. In hbase2, if someone creates a table with a bad durability (i.e. DEFAULT instead of USE_DEFAULT), it results in a failure which causes the CreateTableProcedure to infinitely retries with no backoff. This rapid retry caused a bunch of pain on the cluster that encountered it, backing up datanode's ability to keep up with the millions of calls to create and delete .regioninfo files.
 - This isn't quite an incompatibility, but HBASE-19389 introduced a concurrency mitigation which may have surprising results coming from older versions. The defaults are pretty conservative – when writing more than 100 columns, no more than 10 concurrent writes or 20 pending writes at once.
 - Increments sent from branch-1 clients may get erroneously stored with a timestamp of 0 on hbase2+ clusters: HBASE-26713
 - CheckAndMutate with a "null" compare value used to ignore CompareOp. Fixed in HBASE-26742, checkAndMutate affects may change between versions.
 - client will not know how to handle dangling rep_barrier rows in meta: HBASE-26797
 - the default hbase split policy is SteppingSplitPolicy. This is overall a good policy which is more likely to split small tables to ensure they are spread across more servers. If you upgrade, you may notice your tables suddenly getting split more than you're used to. This may be an issue if you use a row key prefix, because hbase isn't aware of your prefix and may mess up your splits. You can get around this by defining a RegionSplitRestriction. See HBASE-25766
 - Regression in meta requests may impact replication on clusters with many regions. Fixed in 2.4.10+, per HBASE-26590
Bug: Checkstyle: permit single-line if-then statements
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 03/Dec/21 00:15

Change the [LeftCurly|https://checkstyle.sourceforge.io/config_blocks.html#LeftCurly] module [LeftCurlyOption|https://checkstyle.sourceforge.io/property_types.html#LeftCurlyOption] from the default {{eol}} to {{nlow}}. This will permit single-line code blocks like

{noformat}
    if (foo == null) { return null; }
{noformat}
Bug: Adjust log level when looking for .filelist/{f1,f2}
Status: Resolved | Priority: Minor | Resolution: Implemented
Created: 20/Dec/21 21:59

Currently, we get a really big exception in the RegionServer log under normal assignment conditions when we are currently using .filelist/f2 as the tracker file.

We should move this to debug/trace to avoid making operators think there is a problem when there isn't one.