av9ash/GitBugs huggingface dataset:
Bug: [Flakey Test] TestQuotaObserverChoreRegionReports.testReportExpiration
Status: Open | Priority: Major | Resolution: 
Created: 27/Feb/20 17:03
Affects Version/s: 3.0.0-alpha-1

{noformat}
at org.apache.hadoop.hbase.quotas.TestQuotaObserverChoreRegionReports.testReportExpiration(TestQuotaObserverChoreRegionReports.java:118)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException: 
org.apache.hadoop.hbase.TableNotFoundException: hbase:quota
	at java.lang.Thread.getStackTrace(Thread.java:1559)
	at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:130)
	at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:145)
	at org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:168)
	at org.apache.hadoop.hbase.client.TableOverAsyncTable.get(TableOverAsyncTable.java:180)
	at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:843)
	at org.apache.hadoop.hbase.quotas.QuotaTableUtil.getQuotas(QuotaTableUtil.java:162)
	at org.apache.hadoop.hbase.quotas.QuotaTableUtil.getQuotas(QuotaTableUtil.java:150)
	at org.apache.hadoop.hbase.quotas.QuotaTableUtil.getTableQuota(QuotaTableUtil.java:124)
	at org.apache.hadoop.hbase.quotas.MasterQuotaManager$4.fetch(MasterQuotaManager.java:276)
	at org.apache.hadoop.hbase.quotas.MasterQuotaManager.setQuota(MasterQuotaManager.java:467)
	at org.apache.hadoop.hbase.quotas.MasterQuotaManager.setTableQuota(MasterQuotaManager.java:272)
	at org.apache.hadoop.hbase.quotas.MasterQuotaManager.setQuota(MasterQuotaManager.java:160)
	at org.apache.hadoop.hbase.master.MasterRpcServices.setQuota(MasterRpcServices.java:1720)
	at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:374)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
{noformat}
Bug: No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 18/Feb/20 06:54

Now even the split/merge is disabled, master will submit a SplitTableRegionProcedure, too. And rollback it when execute failed. I thought the split/merge switch is a cluster level swtich. Master can check it early and no need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge switch is disabled.
Bug: Update GitHub PreCommit to post back incremental progress updates to PR
Status: Open | Priority: Major | Resolution: 
Created: 26/Feb/20 18:02

This is "merely" a quality of life issue, but I'm filing it as a "Major" because having it would increase contributor confidence in CI automation. GitHub provides some mechanism for build-bots to post details of the work they're performing on an PR. I've never used these APIs, but they appear to allow:
# posting an initial update listing the checking being run
# asynchronous updates of each check status as it completes
# reporting the final check result as a vote on the PR.

Let's see if we can wire these up, either as part of the pipeline or as part of Yetus. Ideally we can also hide the detailed results of these checks behind a UI that consolidates them behind a single summary, rather than having CI spam comments on issues over and over.
Bug: Fix flaky TestSnapshotFromMaster in 1.x versions
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/Feb/20 20:37

HBASE-23658 was about fixing this flakey ut in branch-2+. I think it can be backported to branch-1s as well. 18/30 times this test has failed in branch-1, 10/30 in 1.4 and 3/30 in 1.3.
Bug: [Flakey Test] TestStochasticLoadBalancerRegionReplicaHighReplication "Two or more region replicas are hosted on the same host after balance"
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 20/Feb/20 00:05

Make the calculation less ambitious so we pass more regularly. This is doing same trick as was done in parent and it seems good  now.. hopefully this one too.
Bug: [Flakey Test] TestGenerateDelegationToken; Master not initialized after 200000ms
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 19/Feb/20 18:17

This one was complaint about Master not initializing. Happened twice in a row on local run. Looking the in the .txt and -output.txt was no help but when I looked in the xml, I found this complaint:

{code}
org.apache.hadoop.hbase.ClockOutOfSyncException: Server localhost,53153,1582096705773 has been rejected; Reported time is too far out of sync with master.  Time difference of 44964ms > max allowed of 30000ms
{code}

Indeed, RS is initialized then does its kerberos stuff and there is a big pause of almost 45 seconds.

For now, let me up tolerance for all tests derived from SecureTestCluster
Bug: Update CI Nightly "packaging and integration" to use the new RM tools
Status: Open | Priority: Major | Resolution: 
Created: 19/Feb/20 17:08

Our nightly build does some validations designed to "mimic the actions of a release manager". Since the "actions of a release manager" are codified in a utility, let's use that utility (and it's isolated docker environment) instead of replicating part of the logic in the CI script.
Bug: Fix flaky TestSnapshotFromMaster
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 08/Jan/20 00:57

testAsyncSnapshotWillNotBlockSnapshotHFileCleaner is flaky.  The assert may fail.
{code:java}
assertTrue(master.getSnapshotManager().isTakingAnySnapshot());
future.get(); // in branch-2.2, here is Thread.sleep
assertFalse(master.getSnapshotManager().isTakingAnySnapshot());
{code}
See [https://builds.apache.org/job/HBase-Flaky-Tests/job/master/5227/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testAsyncSnapshotWillNotBlockSnapshotHFileCleaner/]

 

[https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/lastSuccessfulBuild/artifact/dashboard.html]
Bug: [Flakey Test] TestExportSnapshotNoCluster.testSnapshotV2WithRefsExportFileSystemState(TestExportSnapshotNoCluster.java:91) Wrong FS!
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 08/Feb/20 19:06

Can't repro locally w/o running with lots of parallelism. Follow-on from parent issue.

Here is error:

{code}
 [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 115.741 s <<< FAILURE! - in org.apache.hadoop.hbase.snapshot.TestExportSnapshotNoCluster
 [ERROR] org.apache.hadoop.hbase.snapshot.TestExportSnapshotNoCluster.testSnapshotV2WithRefsExportFileSystemState  Time elapsed: 74.459 s  <<< ERROR!
 java.lang.IllegalArgumentException: Wrong FS: file:/Users/stack/checkouts/hbase.apache.git/hbase-mapreduce/target/test-data/fad01338-65ef-e973-2412-05de114016fe/.hbase-snapshot/tableWithRefsV2, expected: hdfs://localhost:58051
   at org.apache.hadoop.hbase.snapshot.TestExportSnapshotNoCluster.testSnapshotWithRefsExportFileSystemState(TestExportSnapshotNoCluster.java:107)
   at org.apache.hadoop.hbase.snapshot.TestExportSnapshotNoCluster.testSnapshotV2WithRefsExportFileSystemState(TestExportSnapshotNoCluster.java:91)
{code}

With v1 had just run find which does same thing.
Bug: Remove unused hadoop.guava.version from pom.xml
Status: In Progress | Priority: Major | Resolution: 
Created: 11/Feb/20 12:34

<hadoop.guava.version>11.0.2</hadoop.guava.version> 
is still used in hbase-backup, I missed it at first...
So, this should be either updated or removed. 
Checking which is feasible...

Update:
So even if I remove hadoop.guava.version, with hadoop-2 profile, 11.0.2 will be used during the build, and it will be on the classpath. 
Since hadoop only upgraded to guava 27.0 in hadoop-3, I'm not sure what we can do here. hadoop-2 is incompatible with guava 2x.x versions.
Bug: Fix NPE when disable DeadServerMetricRegionChore 
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 13/Jan/20 02:23
Affects Version/s: 2.2.3

set hbase.assignment.dead.region.metric.chore.interval.msec = -1

{code}
2020-01-13 10:35:46,247 ERROR [master/10.89.25.45:16000:becomeActiveMaster] master.HMaster: Failed to become active master
java.lang.NullPointerException
    at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.addChore(ProcedureExecutor.java:736)
    at org.apache.hadoop.hbase.master.assignment.AssignmentManager.joinCluster(AssignmentManager.java:1381)
    at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1101)
    at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2223)
    at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:605)
    at java.lang.Thread.run(Thread.java:748)
{code}
Bug: [Flakey Tests] TestSplitTransactionOnCluster
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 24/Jan/20 18:05

TestSplitTransactionOnCluster fails 80% of time in the GCE run up on https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html

We want to run a compaction to clear up references but logic doesn't handle case where compaction already running.
Bug: Add CellComparator to HFileContext
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/Jan/20 20:00

The HFileContext is present when reading and writing files. It is populated at read time using HFile trailer content and file metadata. At write time, we create it up front.

Interesting is that though CellComparator is written to the HFile trailer, and parse of the Trailer creates an HFileInfo which builds the HFileContext at read time, the HFileContext does not expose what CellComparator to use decoding and seeking. Around the codebase there are various compensations made for this lack with decoders that actually have a decoding context (with a reference to the hfilecontext), hard-coding use of the default CellComparator. StoreFileInfo will use default if not passed a comparator (even though we'd just read the trailer and even though it has reference to filecontext) and HFile does similar. What CellComparator to use in a given context is confused.

Let me fix this situation removing ambiguity. It will also fix bugs in parent issue where UTs are failing because wrong CellComparator is being used.
Bug: Support Point-In-Time Queries
Status: Open | Priority: Major | Resolution: 
Created: 29/Jan/20 18:51

HBase currently offers a snapshot feature which allows operators to capture the state of a table at a point in time in a way that can be cloned or queried in the future. It's quite useful in some circumstances, but limited because it's a heavyweight operation, and because it requires prior knowledge of the time you want to capture. 

Phoenix currently offers a feature called "SCN", which uses the max timestamp on Scans to provide the illusion of a "lookback" query at a point in time. It's imperfect, however, because of HBase's filtering and cleanup logic for deletes, max versions and TTLs can prevent users from seeing certain Cells they would have been able to see at a previous point in time. Even PHOENIX-5645, and the equivalent HBASE-23602, which try to control major compaction cleanup, don't cover all edge cases completely. (For example, you can't see rows whose TTL has expired now but hadn't back then. Same with max versions.) 

There are useful non-Phoenix applications as well, such as a change stream that shows before/after images, as DynamoDB offers. 

Since full support will require new configuration options added not just to major compaction, but also to the read pipeline, I'm filing this as an umbrella JIRA so we can have smaller sub-tasks, rather than trying to cram everything into HBASE-23602.
Bug: fix flaky TestSplitTransactionOnCluster
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 09/Jan/20 12:32

testShutDownFixupWhenDaughterHasSplit is flaky. 

I had a look at it, it fails because the compaction isn't ready when the assertion is called. The function HRegion.compact() has a comment that it can block for a long time, do not use it for a time-sensitive thread. 

 

edit: also found a weird debug message, it can also be a problem: 
{{compactions.SortedCompactionPolicy(231): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate.}}
Bug: [Flakey Tests] TestFavoredNodeTableImport fails 30% of the time
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 25/Jan/20 01:38

Spent time on TestFavoredNodeTableImport. It fails w/ an NPE when we go to get favorednodes for one of the regions. It is sporadic. Fails for me locally too about 30% of the time.

I tried to study where we are going wrong. The balancer is disabled when we start the cluster up again on FN balancer... but this don't seem to be the problem.

Looks like laggard Regions taking their time to open means they don't show in the global list of favored nodes when the checking runs. Adding a wait till no RIT seems to stabilize the test.
Bug: Optimize Memstore Flush for Hbase on S3(Object Store)
Status: Open | Priority: Major | Resolution: 
Created: 24/Jan/20 01:04

The current Memstore Flush Process is divided into 2 stages:
 # Flushcache: In this stage, a “.tmp” region file is written in S3/HDFS for the memstore;
 # Commit: In this stage, the “.tmp” file created in the stage 1 is renamed to final destination of HBase region file.

The above design(flush and commit) is OK for HDFS because “rename” is light opertion(only metadata operation). However, for storage like S3 or other object store, rename is “copy” and “delete” operation.

We can follow the same pattern from V2 of  “FileOutputCommitter” in MapReduce. That means, we can write hfile directly to the S3 destination directory without “copy” and “paste”. So that we can have less S3 operations and the HBase memstore flush is more efficient.
Bug: Make HBaseInterClusterReplicationEndpoint more extensible
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 13/Jan/20 11:47

*HBaseInterClusterReplicationEndpoint* currently creates the cluster connection and sink manager instances inside its _init_ method and assigns those to private class variables. Then any potential custom extension of *HBaseInterClusterReplicationEndpoint* that requires custom implementations of connection and/or sink manager would need to resort to _java reflection_ for effectively replace those instances, such as below:

{noformat}
...
    ClusterConnection conn = (ClusterConnection)ConnectionFactory.
      createConnection(context.getConfiguration(), User.create(replicationUgi));
    ReplicationSinkManager sinkManager = new ReplicationSinkManager(conn, ctx.getPeerId(),
      this, context.getConfiguration());
    try {
      Field field = this.getClass().getSuperclass().getDeclaredField("conn");
      field.setAccessible(true);
      field.set(this, conn);
      field = this.getClass().getSuperclass().getDeclaredField("replicationSinkMgr");
      field.setAccessible(true);
      field.set(this, sinkManager);
    } catch (Exception e) {
      throw new IOException(e);
    }
...
{noformat}
Bug: Add further constraint on edits to hbase:meta to prevent possibly damaging edits
Status: Open | Priority: Major | Resolution: 
Created: 23/Jan/20 02:17

HBASE-23055 Alter hbase:meta enables editing hbase:meta in hbase-2.3.0. The patch includes constraint that prevents an operator accidentally deleting a critical column family. This issue is about adding further constraints to prevent possibly damaging configs. Per [~zhangduo], operator "...should not set TTL on the column families, and also, you should not change the versions of REPLICATION_BARRIER_FAMILY..."
Bug: RegionProcedureStore missing cleaning of hfile archive
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 11/Jan/20 00:13
Affects Version/s: 2.3.0

See tail of parent issue. The new RegionProcedureStore accumulates deleted hfiles in its local archive dir. Needs a cleaner like the one that watches over /hbase/archive.

Is there a problem clearning the new $masterproc$ files from the oldWALs too? These seem to stick around also.
Bug: ByteBuff may not recycle correctly when NettyRpcServer enabled and client disconnect
Status: Open | Priority: Major | Resolution: 
Created: 16/Jan/20 12:21

Code in NettyRpcServerResponseEncoder
{code:java}
@Override
public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
  if (msg instanceof RpcResponse) {
    RpcResponse resp = (RpcResponse) msg;
    BufferChain buf = resp.getResponse();
    ctx.write(Unpooled.wrappedBuffer(buf.getBuffers()), promise).addListener(f -> {
      resp.done();
      if (f.isSuccess()) {
        metrics.sentBytes(buf.size());
      }
    });
  } else {
    ctx.write(msg, promise);
  }
}
{code}
If exception occurs during ChannelHandlerContext#write (such as Client disconnected), ByteBuff may not be recycled correctly.
Bug: Update checkstyle plugin (and update checkstyle.xml to match)
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 17/Jan/20 22:54

In parent issue, its suggested we update our checkstyle plugin to match of intellij plugin default at least.  Will need checkstyle.xml changes else it fails parse (See notes in parent by [~bharathv] on what needs doing).

For extra points, do the [~ndimiduk] suggestion: "...It would be nice also if we could commit the .idea/checkstyle-idea.xml file with the checkstyle version used buy the plugin pinned to the same version as we're using in maven."
Bug: Replace HColumnDescriptor(String cf) with ColumnFamilyDescriptor
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 08/Jan/20 10:47
Affects Version/s: 3.0.0-alpha-1

We have deprecated HColumnDescriptor and HTableDescriptor some time back in 2.0.0. Although there are many use-cases still using them, we can start replacing them with builder pattern: ColumnFamilyDescriptorBuilder and TableDescriptorBuilder.

In an attempt to do so, starting with replacing specific constructor of HColumnDescriptor:
{code:java}
public HTableDescriptor(final TableName name) {
  this(new ModifyableTableDescriptor(name));
}
{code}
Bug: indicates a last flushed sequence id ... that is less than the previous last flushed sequence id
Status: Open | Priority: Major | Resolution: 
Created: 14/Jan/20 00:11
Affects Version/s: 2.3.0

I'm getting loads of the below in Master log running tests against branch-2. It is heavily loaded but generally keeping up though there is backlog in WAL files.... > 32 ... around the cluster so forced flushes happening. I'll see the below for a Region even though it seems like we've since flushed out a sequenceid on the RS-side that is larger than what the Master is seeing. Two column family table.

{code}
 2020-01-13 23:33:18,455 WARN org.apache.hadoop.hbase.master.ServerManager:      RegionServer hbasedn030.sp07.siri.apple.com,16020,1578934813139 indicates a     last flushed sequence id (1593644) that is less than the previous last flushed  sequence id (1593649) for region t1,f9371d5,1576227377175.                      3e41deae849d25f0a2f1d654f482d73a. Ignoring.
{code}
Bug: BaseLoadBalancer#wouldLowerAvailability should consider region replicas
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 08/Jan/20 08:30

Found this issue when try to fix the flaky unit test  TestRegionReplicaSplit. It may fail as 

java.lang.AssertionError: Splitted regions should not be assigned to same region server.

See [https://builds.apache.org/job/HBase-Flaky-Tests/job/master/5227/testReport/junit/org.apache.hadoop.hbase.master.assignment/TestRegionReplicaSplit/testRegionReplicaSplitRegionAssignment/].

 

Now  wouldLowerAvailability method only consider the primary regions. The replica region can't assign to same server with primary region. But can be assigned to same server with other replica regions.
Bug: Master log start filling with "Flush journal status" messages
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 09/Jan/20 07:27

Takes a while to get into this condition. Not each to tell how because all logs have rolled off and I only have logs filled w/ below:

{code}
2020-01-09 07:01:01,723 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush status journal:
	Acquiring readlock on region at 1578553261723
	Flush successful flush result:CANNOT_FLUSH_MEMSTORE_EMPTY, failureReason:Nothing to flush,flush seq id45226854 at 1578553261723
2020-01-09 07:01:01,723 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush status journal:
	Acquiring readlock on region at 1578553261723
	Flush successful flush result:CANNOT_FLUSH_MEMSTORE_EMPTY, failureReason:Nothing to flush,flush seq id45226855 at 1578553261723
2020-01-09 07:01:01,723 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush status journal:
	Acquiring readlock on region at 1578553261723
	Flush successful flush result:CANNOT_FLUSH_MEMSTORE_EMPTY, failureReason:Nothing to flush,flush seq id45226856 at 1578553261723
2020-01-09 07:01:01,723 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush status journal:
	Acquiring readlock on region at 1578553261723
	Flush successful flush result:CANNOT_FLUSH_MEMSTORE_EMPTY, failureReason:Nothing to flush,flush seq id45226857 at 1578553261723
{code}

... I added the printing of flushresult... i.e. cannot flush because store is empty.

Digging.
Bug: Please add Apache Trafodion and EsgynDB to "Powered by Apache HBase" page
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 06/Jan/20 22:40

Please add the following two entries to the web page "Powered By Apache HBase™" ([https://hbase.apache.org/poweredbyhbase.html]):

Trafodion (link [https://trafodion.apache.org/]) 

Apache Trafodion(TM) is a webscale SQL-on-Hadoop solution enabling transactional or operational workloads. It uses HBase as its storage engine for SQL tables.

EsgynDB (link [https://esgyn.com/])

EsgynDB, powered by Apache Trafodion(TM), provides enterprise SQL on Hadoop. It includes full ACID transactions, online transaction processing and online analytic processing, along with enterprise features such as disaster recovery and full backup/restore. Native tables are stored in HBase, but read and write access to various other file formats such as Apache Parquet and ORC is also supported.
Bug: org.apache.hadoop.hbase.ipc.RpcServer.Listener#closeCurrentConnection did not close the connection properly
Status: Open | Priority: Major | Resolution: 
Created: 10/Jan/20 06:54
Affects Version/s: 1.4.12

The connection is attached to readKey, but the parameter passed into the closeCurrentConnection is acceptedKey. branch-2 is fixed in HBASE-15948, though 2.0+ already uses netty.