av9ash/GitBugs huggingface dataset:
Bug: Fix flaky TestWALEntryStream
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 07/Jul/20 01:17

[https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/lastSuccessfulBuild/artifact/dashboard.html]

 

Failed 100.0% (13 / 13) recently.
Bug: Add Bulk load param details into its responseTooSlow log
Status: Open | Priority: Major | Resolution: 
Created: 04/Jul/20 06:59

Right now the log will come like
{code}
(responseTooSlow): {"call":"BulkLoadHFile(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$BulkLoadHFileRequest)","starttimems":1593820455043,"responsesize":2,"method":"BulkLoadHFile","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$BulkLoadHFileRequest",......}
{code}
Bug: [Flakey] TestVerifyReplication.testVerifyRepJob & TestVerifyReplicationAdjunct.testVerifyReplicationWithSnapshotSupport
Status: Open | Priority: Major | Resolution: 
Created: 03/Jul/20 14:17
Affects Version/s: 2.3.0

Attaching sunfire reports xml for details.

For both reports, on high level, I see some hanging threads for ReadOnlyZKClient and Replica not found for a datablock related Exceptions.

[^TEST-org.apache.hadoop.hbase.replication.TestVerifyReplication.xml]
Bug: When processing a procedure, the process doing the work should log the pid
Status: Open | Priority: Major | Resolution: 
Created: 01/Jul/20 23:37

One thing I'm observing while trying to debug ProcedureV2 work is it's difficult to track an procedure as it's execution moves around the cluster. It would be nice if workers would include the PID (and maybe PPID) in the work they're performing. This probably means including this information as potential metadata in all of our RPC requests, and finding a way to log it uniformly.
Bug: DIGEST based auth broken for rpc based ConnectionRegistry
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 16/Sep/20 16:13
Affects Version/s: 1.7.0

DIGEST-MD5 based sasl auth depends on cluster-ID to obtain tokens. With master registry, we have a circular dependency here because master registry needs an rpcClient to talk to masters (and to get cluster ID) and rpc-Client needs a clusterId if DIGEST based auth is configured. Earlier, there was a ZK client that has its own authentication mechanism to fetch the cluster ID.

HBASE-23330, I think doesn't fully fix the problem. It depends on an active connection to fetch delegation tokens for the MR job and that inherently assumes that the active connection does not use a DIGEST auth.

It is not clear to me how common it is to use DIGEST based auth in connections.
Bug: Unable to call HBaseContext From PySpark
Status: Open | Priority: Minor | Resolution: 
Created: 06/Aug/20 17:46

I'm able to get the JavaHBaseContext Object From PySpark but not HBaseContext 
{code:java}
temp = sc._jvm.org.apache.hadoop.hbase.HBaseConfiguration
conf = temp.create()
hbaseCon = sc._jvm.org.apache.hadoop.hbase.spark.HBaseContext(sc, conf){code}
Running the above Code gives me this error

 
{code:java}
AttributeError: 'SparkContext' object has no attribute '_get_object_id'
 AttributeError Traceback (most recent call last)
 in engine
 ----> 1 hbaseCon = sc._jvm.org.apache.hadoop.hbase.spark.HBaseContext(sc, conf)
/usr/local/lib/python3.6/site-packages/py4j/java_gateway.py in _call_(self, *args)
 1543 
 1544 args_command = "".join(
 -> 1545 [get_command_part(arg, self._pool) for arg in new_args])
 1546 
 1547 command = proto.CONSTRUCTOR_COMMAND_NAME +\
/usr/local/lib/python3.6/site-packages/py4j/java_gateway.py in <listcomp>(.0)
 1543 
 1544 args_command = "".join(
 -> 1545 [get_command_part(arg, self._pool) for arg in new_args])
 1546 
 1547 command = proto.CONSTRUCTOR_COMMAND_NAME +\
/usr/local/lib/python3.6/site-packages/py4j/protocol.py in get_command_part(parameter, python_proxy_pool)
 296 command_part += ";" + interface
 297 else:
 --> 298 command_part = REFERENCE_TYPE + parameter._get_object_id()
 299 
 300 command_part += "\n"
AttributeError: 'SparkContext' object has no attribute '_get_object_id'
{code}
Bug: return value of LogRoller.walRollFinished may be wrong
Status: Open | Priority: Minor | Resolution: 
Created: 29/Jul/20 07:18
Affects Version/s: 2.2.6

The LogRoller.walRollFinished impl is already buggy. This just checks the status of the boolean. Once we start a roll on a WAL, we reset the boolean (Even before patch HBASE-24665). So it is not clearly telling anything abt the roll status. This can return true even while an active wal roll is going on.

We might need another boolean in Controller which clearly tracks whether we are ongoing a roll.
Bug: Wait for connection attempt to succeed before performing operations on ZK
Status: In Progress | Priority: Minor | Resolution: 
Created: 31/Aug/20 18:01

{color:#1d1c1d}Creating the connection with ZK  is asynchronous and notified via the passed in watcher about the  successful connection event. When we attempt any operations, we try to create a connection and then perform a read/write ({color}{color:#1d1c1d}[https://github.com/apache/hbase/blob/979edfe72046b2075adcc869c65ae820e6f3ec2d/hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java#L323]{color}{color:#1d1c1d}) without really waiting for the notification event ([https://github.com/apache/hbase/blob/979edfe72046b2075adcc869c65ae820e6f3ec2d/hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKWatcher.java#L582)]{color}

 

{color:#1d1c1d}It is possible we get ConnectionLoss errors when we perform operations on ZK without waiting for the connection attempt to succeed{color}
Bug: Remove duplicate code appearing continuously in method ReplicationPeerManager.updatePeerConfig
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 20/Aug/20 12:30
Affects Version/s: 3.0.0-alpha-1

In [ReplicationPeerManager.updatePeerConfig|https://github.com/apache/hbase/blob/1164531d5ab519ab58af82ba3849f8fcded3453f/hbase-server/src/main/java/org/apache/hadoop/hbase/master/replication/ReplicationPeerManager.java#L272], I found duplicate code appearing twice continuously, so remove once.
{code:java}
newPeerConfigBuilder.putAllConfiguration(oldPeerConfig.getConfiguration());
newPeerConfigBuilder.putAllConfiguration(peerConfig.getConfiguration());
newPeerConfigBuilder.putAllConfiguration(oldPeerConfig.getConfiguration());
newPeerConfigBuilder.putAllConfiguration(peerConfig.getConfiguration());
{code}
Bug: [Flake Test][branch-1] TestClientOperationInterrupt#testInterrupt50Percent
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 14/Sep/20 07:54

Sometimes threads could finish faster before interrupt() gets called. 
{code}
      // ...
      t.start();
    }
    int expectedNoExNum = nbThread / 2;

    for (int i = 0; i < nbThread / 2; i++) {
      if (threads.get(i).getState().equals(Thread.State.TERMINATED)) {
        expectedNoExNum--;
      }
      threads.get(i).interrupt();
    }
{code}

So this test could get failed sometimes.
Bug: Add hedgedReadOpsInCurThread metric
Status: Closed | Priority: Minor | Resolution: Fixed
Created: 08/Sep/20 03:21

When adding Hedged Reads metrics at https://issues.apache.org/jira/browse/HBASE-12220, hedgedReadOpsInCurThread was not included.

I wonder if there was any reason to not include them. If not, I will provide pull request to include them soon.

 

Hedged reads use same thread pool for the original read:
 [https://github.com/apache/hadoop/blob/0b8464d75227fcee2c6e7f2410377b3d53d3d5f8/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java#L1349-L1357]
 When the executor is full and rejects the task, it will be executed in current thread, increasing this metric:
 [https://github.com/apache/hadoop/blob/0b8464d75227fcee2c6e7f2410377b3d53d3d5f8/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java#L2913-L2918]

This metric is very useful to determine if the hedged reads thread pool (dfs.client.hedged.read.threadpool.size) is undersized.
Bug: Pass WALFactory to Replication so it knows of all WALProviders, not just default/user-space
Status: Closed | Priority: Minor | Resolution: Fixed
Created: 19/Sep/20 00:41

Small change that passes all WALProviders to ReplicationService rather than just the default/user-space WALProvider. It does this using the WALFactory vessel since it holds all Providers. This change is to be exploited by adjacent sub-task HBASE-25055 in follow-on. This sub-task also exists to make the HBASE-25055 patch smaller and more focused, easier to review.
Bug: WebUI header bar overlaps page content when window is too narrow
Status: Open | Priority: Minor | Resolution: 
Created: 07/Jul/20 19:38
Affects Version/s: 2.3.0

It seems the CSS on our WebUI is such that the header will expand down vertically as the content wraps dynamically. However, the page content does not shift down along with it, resulting in the header overlapping the page content.
Bug: check if we're impacted by MASSEMBLY-941 and mitigate if needed
Status: Open | Priority: Minor | Resolution: 
Created: 13/Sep/20 04:51
Affects Version/s: 1.8.0

MASSEMBLY-941 notes a bug starting in version 3.2.0 of the assembly plugin where scripts lose their executable bit.

We've had this version since updating to the apache parent pom version 22 in HBASE-23675. We should check our release artifacts to see if we are impacted and if so downgrade the assembly plugin in our poms to 3.1.1.
Bug: Use GetoptLong or OptionParser in hbase-shell
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 24/Jul/20 19:23
Affects Version/s: 2.3.0

Currently, our hbase-shell command line argument parser is custom-rolled. It would be awesome to instead use Ruby's GetoptLong or OptionParser.
 * [https://ruby-doc.org/stdlib-2.3.0/libdoc/getoptlong/rdoc/GetoptLong.html]
 * [https://ruby-doc.org/stdlib-2.3.0/libdoc/optparse/rdoc/OptionParser.html]

There is a long-standing {{FIXME}} comment in {{bin/hirb.rb}} to address this: [https://github.com/apache/hbase/blob/975cdf7b88f001aa51d83395c663892f3b5ffbdb/bin/hirb.rb#L50]
Bug: table.rb use LocalDateTime to replace Instant
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 24/Aug/20 01:20
Affects Version/s: 2.2.3

https://github.com/apache/hbase/blob/9f62a82334574b135f8e220b024981df64fab811/hbase-shell/src/main/ruby/hbase/table.rb#L754

we can use timeZone to improve readability.

{code:java}
return java.time.LocalDateTime.ofInstant(instant, java.time.ZoneId.systemDefault()).toString
{code}
Bug: `assert` on allocated buffer sizes before making calls to `Unsafe.copyMemory`
Status: Open | Priority: Minor | Resolution: 
Created: 10/Sep/20 20:15
Affects Version/s: 2.3.0

While investigating a SIGSEGV, I stumbled into our use of {{Unsafe.copyMemory}} for moving data between {{byte[]}}'s and/or {{DirectByteBuffer}}'s. Calls to {{copyMemory}} use  an Object, offset API that performs no validation what so ever. I'd like to add {{assert}} statements (disabled by default) before these memory copies.
Bug: Candidate Generator helper getAction method ignoring 0th index region
Status: Resolved | Priority: Minor | Resolution: Duplicate
Created: 23/Aug/20 09:08

Balance Candidate generators such as `LocalityBasedCandidateGenerator`, `RegionReplicaCandidateGenerator`,  `RegionReplicaRackCandidateGenerator`, etc uses helper method `getAction` to generate action is ignoring 0th index of `fromRegion` and `toRegion`. 
{code:java}
protected BaseLoadBalancer.Cluster.Action getAction(int fromServer, int fromRegion,
    int toServer, int toRegion) {
  if (fromServer < 0 || toServer < 0) {
    return BaseLoadBalancer.Cluster.NullAction;
  }
  if (fromRegion > 0 && toRegion > 0) {
    return new BaseLoadBalancer.Cluster.SwapRegionsAction(fromServer, fromRegion,
      toServer, toRegion);
  } else if (fromRegion > 0) {
    return new BaseLoadBalancer.Cluster.MoveRegionAction(fromRegion, fromServer, toServer);
  } else if (toRegion > 0) {
    return new BaseLoadBalancer.Cluster.MoveRegionAction(toRegion, toServer, fromServer);
  } else {
    return BaseLoadBalancer.Cluster.NullAction;
  }
}
{code}
Is this unintentional or there is some particular reason?
Bug: Add possibility to set different types of L1 cache
Status: Resolved | Priority: Minor | Resolution: Implemented
Created: 30/Sep/20 10:41

The feature HBASE-23887 allow speed up to 3 times read performance but maybe it is too complicated. So there are proposals give users possibility to choose type of cache L1. Looks like it needs to change few classes (CombinedBlockCache, InclusiveCombinedBlockCache, CacheConfig) if somebody can code this it would be cool.
Bug: RegionMonitor support RegionTask count normalize
Status: Open | Priority: Minor | Resolution: 
Created: 29/Sep/20 09:01

large count of region task from canary may affect user normal request, meanwhile if region task is few, the  availability monitoring may shake for occasional exception.

so , if the task count is large , we will randomly trim tasks for each table, according to the raito of the table region count in whole tasks region count. If the task count is few,  we will repeat tasks
Bug: Add support of adding base peer configs via hbase-site.xml for all replication peers.
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 23/Jul/20 19:27
Affects Version/s: 1.7.0

Today, if a user needs to apply some common base peer configs to all the replication peers on a given cluster, the only way is to execute update_peer_config via CLI which requires manual intervention and can be tedious in case of large deployment fleet.

As part of this JIRA, we plan to add the support to have base replication peer configs as part of hbase-site.xml like hbase.replication.peer.base.config="k1=v1;k2=v2.." which can be easily updated and applied as part of a rolling restart. Example below:
 <property>
 <name>hbase.replication.peer.base.configs</name>
 <value>hbase.replication.source.custom.walentryfilters=x,y,z;hbase.rpc.protection=abc;hbase.xxx.custom_property=123</value>
 </property>

This will be empty by default, but one can override to have base configs in place.

The final peer configuration would be a merge of this newly added base config + whatever users override during the peer creation/update (if any).

Related Jira: https://issues.apache.org/jira/browse/HBASE-17543.  HBASE-17543 added the support to add the WALEntryFilters to default endpoint via peer configuration. By this new Jira we are extending the support to update peer configs via hbase-site.xml.
Bug: Display Task completion time and/or processing duration on Web UI
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 21/Jul/20 18:53
Affects Version/s: 2.3.0

Our web UI's list "Tasks", some of which are really important things an operator wants to know about. We have columns Start Time, Description, State, and Status (which looks more or less identical to Description). We don't have Completion Time or Duration columns. I think it would be nice to see at a glance how long various Tasks are taking.
Bug: We initialize Filesystems more than once.
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 16/Sep/20 13:24
Affects Version/s: 2.2.6

In HFileSystem
{code}
    // Create the default filesystem with checksum verification switched on.
    // By default, any operation to this FilterFileSystem occurs on
    // the underlying filesystem that has checksums switched on.
    this.fs = FileSystem.get(conf);
    this.useHBaseChecksum = useHBaseChecksum;

    fs.initialize(getDefaultUri(conf), conf);
{code}
We call fs.initialize(). Generally the FS would have been created and inited either in the FileSystem.get() call above or even when we try to check 
{code}
  FileSystem fs = p.getFileSystem(c);
{code}
The FS that gets cached in the hadoop-common layer does the init for us. So we doing it again is redundant.
Bug: Create a metric to track full region scans RPCs
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 15/Sep/20 05:48
Affects Version/s: 3.0.0-alpha-1

A metric that indicates how many of the scan requests were without start row and/or stop row. Generally such queries may be wrongly written or may require better schema design and those may be some queries doing some sanity check to verify if their actual application logic has done the necessary updates and the all that expected rows are processed. 

We do have some logs at the RPC layer to see what queries take time but nothing as a metric.
Bug: Improve insight into replication WAL readers hung on checkQuota
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 27/Jul/20 16:00

Helped a customer this past weekend who, with a large number of RegionServers, has some RegionServers which replicated data to a peer without issues while other RegionServers did not.

The number of queue logs varied over the past 24hrs in the same manner. Some spikes in queued logs into 100's of logs, but other times, only 1's-10's of logs were queued.

We were able to validate that there were "good" and "bad" RegionServers by creating a test table, assigning it to a regionserver, enabling replication on that table, and validating if the local puts were replicated to a peer. On a good RS, data was replicated immediately. On a bad RS, data was never replicated (at least, on the order of 10's of minutes which we waited).

On the "bad RS", we were able to observe that the \{{wal-reader}} thread(s) on that RS were spending time in a Thread.sleep() in a different location than the other. Specifically it was sitting in the {{ReplicationSourceWALReader#checkQuota()}}'s sleep call, _not_ the {{handleEmptyWALBatch()}} method on the same class.

My only assumption is that, somehow, these RegionServers got into a situation where they "allocated" memory from the quota but never freed it. Then, because the WAL reader thinks it has no free memory, it blocks indefinitely and there are no pending edits to ship and (thus) free that memory. A cursory glance at the code gives me a _lot_ of anxiety around places where we don't properly clean it up (e.g. batches that fail to ship, dropping a peer). As a first stab, let me add some more debugging so we can actually track this state properly for the operators and their sanity.
Bug: CompactionConfiguration logs unrealistic store file sizes
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 23/Sep/20 18:10
Affects Version/s: 2.3.2

I noticed this in the debug logging of a unit test that makes use of a MiniCluster. I'm pretty sure {{8.00 EB}} -- exabytes?? -- is not an accurate reporting of the size of these HFiles.

{noformat}
2020-09-22 16:20:57,996 INFO  [StoreOpener-5e0b426a952230abb65623fc54869833-1] compactions.CompactionConfiguration(177): size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ...
{noformat}
Bug: Invalid addFsRegionsMissingInMeta -d
Status: Open | Priority: Minor | Resolution: 
Created: 24/Aug/20 03:15
Affects Version/s: 2.2.3

[https://github.com/apache/hbase-operator-tools/blob/87878aada3354514050f5a2df11f27b317efd42d/hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java#L436]

-d is invalid. addFsRegionsMissingInMeta does not use -d
Bug: 'Failed report transition' logs too often
Status: Open | Priority: Minor | Resolution: 
Created: 16/Jul/20 22:03
Affects Version/s: 2.3.0

The parent issue fixed a backoff that was too aggressive. Now I notice we try too much. Saw 9k logs in 17 seconds of the below type...
{code:java}
2020-07-15 14:36:23,104 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Failed report transition server { host_name: "X.example.org" port: 16020 start_code: 1594823099666 } transition { transition_     code: CLOSED region_info { region_id: 1594814749475 table_name { namespace: "default" qualifier: "IntegrationTestBigLinkedList" } start_key: "\"\"\"\"\"\"\" " end_key: "#Q\352\f\003" offline: false split: false replica_id:      0 } proc_id: 81545 }; retry (#8888) after 200805ms delay (Master is coming online...).
 {code}
The delay doesn't seem correct or respected.
Bug: Move LogComparator from ReplicationSource to AbstractFSWALProvider#.WALsStartTimeComparator
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 23/Sep/20 18:11

Minor cleanup item noticed playing over in HBASE-18070.

ReplicationSource has an inner class named LogComparator which is a pretty generic name for the comparator only it just compares on WAL start time and nothing else.

Also, messing in HBASE-18070 I ran into compares that included user-space WALs and hbase:meta WALs. The LogComparator as is barfed on meta WALs.

This ticket moves the comparator to AbstractFSWALProvider, where folks will go looking if they need WAL comparators, and it also renames it to more clearly explain what it does (and makes it so it can compare start times even if it a meta WAL).
Bug: Fix property name added as part of HBASE-24764 in branch-1
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 30/Sep/20 01:07

A new property was added as part of this https://issues.apache.org/jira/browse/HBASE-24764. 

Master branch has the name as "hbase.replication.peer.base.config"

while branch-1 has the name as "hbase.replication.peer.default.config"

 

As part of this Jira fix the name in branch-1 to be consistent.
Bug: QuotaRefresherChore doesn't update QuotaCache for user effectively
Status: Open | Priority: Minor | Resolution: 
Created: 21/Sep/20 18:28

Currently RegionServer record a user on their first RPC and trigger the QuotaCache refresh,


 https://github.com/apache/hbase/blob/8a6299bb387aaedcdab05224a462e63cd121fbb0/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaCache.java#L136


And QuotaRefresherChore refresh quota periodically for those users who are recorded by RegionServer on their first RPC, 


https://github.com/apache/hbase/blob/8a6299bb387aaedcdab05224a462e63cd121fbb0/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaCache.java#L284

 

So even though user quota is set but it will be updated when user send RPC. Here each user will get 1 extra RPC in each RegionServer life and same will continue when got restarted. 

 

We can enhance this by doing narrow scan on quota table to get the user specific quota
Bug:  Display region name instead of encoded region name in HBCK report page.
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 19/Sep/20 06:09
Affects Version/s: 2.3.1

In HMaster UI, in HBCK report, holes, overlaps and unknown server display only encoded region names. 

In display region encode name does not give any information. like which table, what's the start key etc. 

I think it is better to display the region name instead of region encoded name.

!image-2020-09-19-11-39-01-755.png!
Bug: Hbck chore logs wrong message when loading regions from RS report
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 10/Sep/20 18:56
Affects Version/s: 2.3.1

{code:java}
LOG.info("Loaded {} regions from {} regionservers' reports and found {} orphan regions",
        numRegions, rsReports.size(), orphanRegionsOnFS.size());
{code}
In above log message orphanRegionsOnFS.size() should be replaced with orphanRegionsOnRS.size() as the regions are loaded from RS not form FS
Bug:  Fix several problems when starting webUI
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 11/Aug/20 14:40
Affects Version/s: 2.3.0

As discussed in the parent issue. There are three problems to be solved：
 # If we put up webUI while starting Canary, there throw a ClassNotFoundException that shaded class cannot be found
 # webUI should be showing up only in daemon mode
 # ServletException throwing when visiting the web page of Canary webUI
Bug: Update default URL to KEYS file in `hbase-vote.sh`
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 17/Sep/20 20:58
Affects Version/s: 3.0.0-alpha-1

{{hbase-vote.sh}} goes to {{https://www.apache.org/dist/hbase/KEYS}} to find the KEYS file. That URL now gives a redirect to {{https://downloads.apache.org/hbase/KEYS}}. The script doesn't follow the redirect.

Let's update the script to point to the new URL.