av9ash/GitBugs huggingface dataset:
Bug: RegionNormalizer spliting region should not be limited by hbase.normalizer.min.region.count
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 22/May/20 04:03
Affects Version/s: 3.0.0-alpha-1

In method computePlanForTable of SimpleRegionNormalizer: 

we will skip spliting region if the number of regions in the table is less than hbase.normalizer.min.region.count, even if there is a huge region in the table.
{code:java}
...
if (tableRegions == null || tableRegions.size() < minRegionCount) {
  ...
  return null;
}

...
// get region split plan
if (splitEnabled) {
  List<NormalizationPlan> splitPlans = getSplitNormalizationPlan(table);
  if (splitPlans != null) {
    plans.addAll(splitPlans);
  }
}
{code}
Bug: Consolidate Normalizer implementations
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 22/May/20 16:04
Affects Version/s: 2.3.0

After HBASE-22285, we have two implementations of {{RegionNormalizer}}, that have different feature sets and different configurations. I think these can be combined into a single implementation, with clear, decoupled configuration parameters. At least on branch-2.3, there's too many subsequent changes for HBASE-22285 to revert cleanly, so I'll use this ticket to consolidate the implementations.

If you have issues with the current normalizer, speak up here and we can include them, or add them as sub-tasks.
Bug: ServerName#getHostname() is case sensitive
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 18/May/20 09:43
Affects Version/s: 1.3.1

ServerName calss，the getServerName(String hostName, int port, long startcode),equals and compareTo are case insensitive, but getHostname() is case sensitive.
if hostName is HOSTNAME1, ServerName is hostname1,11111,1589615319931, and getHostname() returns HOSTNAME1.

and then BaseLoadBalancer#retainAssignment() uses ServerName#getHostname(), all keys of serversByHostname are upperCase(HOSTNAME1,HOSTNAME2,HOSTNAME3,HOSTNAME4...) from ServerManager#createDestinationServersList, but oldServerName.getHostname() is lowerCase(hostname1,hostname2,hostname3...) from walLog dir.

!image-2020-05-18-17-42-57-119.png!

and finally...all region of old ServerName will be assigned to random hosts
Bug: HBCK2 should accept "hbase:meta" as an alias for the encoded region name
Status: Open | Priority: Major | Resolution: 
Created: 30/Apr/20 17:54
Affects Version/s: 2.3.0

Simple one: hbck2 should accept the alias "meta" in place of the encoded region name. It's a little silly to have to say {{hbck2 assigns 1588230740}}
Bug: DateTiered Compaction should disallow 'hbase.rs.cachecompactedblocksonwrite' if the compacted file is from older window
Status: Open | Priority: Major | Resolution: 
Created: 15/May/20 11:06
Affects Version/s: 3.0.0-alpha-1

This issue is related to https://issues.apache.org/jira/browse/HBASE-23066.
In Date tiered compaction once we move to old window - we should ideally not cache the resultant file because generally those files are not read (as it is not the window of our interest). We can provide a switch to either turn it ON or OFF. Preferrably for Data tiered compaction this switch should be OFF.
Bug: Improvement update_config for PressureAwareCompactionThroughputController
Status: In Progress | Priority: Major | Resolution: 
Created: 25/Apr/20 06:03

If we start major compaction on a big table first,

then we modified compaction limit configuration and execute 'update_config' command,

it will not effective immediately.

The compaction which is running is still use the old Controller object.

The higher.bound,lower.bound,offPeakHours is still the old value.

 

I think we can make the modification effective immediately.

There is 2 points about it:

1.keep there is always only one PressureAwareCompactionThroughputController object in RS.

2.add a updateConfig method in Controller.

when we use 'update_config' command to update compact pressure conf , it just change the value of  higher.bound,lower.bound,offPeakHours ,instead of create a new Controller
Bug: HBCK2 fix addFsRegionsMissingInMeta to add regions in CLOSED state again
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 16/Jun/20 13:24

Originally, _addFsRegionsMissingInMeta_ relied on _MetaTableAccessor_ to add new regions, which correctly added new regions in CLOSED state. HBASE-23791 removed dependency on _MetaTableAccessor_ as to avoid relying on HBase IA private methods. _MetaTableAccessor_ functions needed by HBCK2 were then re-implemented on _HBCKMetaTableAccessor_. Implementation for _addRegionToMeta_, though, was not fully ported, and it was not setting newly added regions state. 

This would cause attempts to assign the newly added regions to fail, as AM ignore regions without state defined in meta, requiring extra calls for hbck2 setRegionState command for each of the newly added regions.

Kudos for [~brfrn169] who identified this problem while troubleshooting/fixing regions assignments issues with hbck2.

This PR fix implementation of _HBCKMetaTableAccessor_._addRegionToMeta_ to insert new regions with _CLOSED_ state, and also adds extra checks for this behaviour on both _TestHBCKMetaTableAccessor_ and _TestFsRegionsMetaRecoverer_ UTs.
Bug: hbase-filesystem to use guava from hbase-thirdparty
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 06/Apr/20 18:42
Affects Version/s: hbase-filesystem-1.0.0-alpha1

hbase-filesystem repo is on guava23.0:

{noformat}
$ grep -r "guava" .
./pom.xml:    <guava.version>23.0</guava.version>
./hbase-oss/pom.xml:      <groupId>com.google.guava</groupId>
./hbase-oss/pom.xml:      <artifactId>guava</artifactId>
./hbase-oss/pom.xml:      <version>${guava.version}</version>
./hbase-oss/pom.xml:          <!-- this should have been marked optional by guava because
{noformat}

It should use the guava from hbase-thirdparty instead.
Bug: Create metric to know the number of reads that happens from memstore
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/Apr/20 10:33
Affects Version/s: 3.0.0-alpha-1

A metric to identify number of reads that were served from memstore (atleast if the gets can be accounted for) then it gives a value addition to know if  among the reads how much was targeted at the most recent data.
Currently the existing metric framework at region level should be enough but we can also add a metric per store level.  That will be more granular. 
We can also expose this via HbTop.
Bug: HMaster UI hangs when rsgorup is enabled and meta-region is not available
Status: Resolved | Priority: Major | Resolution: Not A Problem
Created: 17/Apr/20 23:19
Affects Version/s: 2.2.4

HMaster UI hangs when rsgroup is enabled and meta-region is not available.

Steps to reproduce:
 # Cluster: 1 Master, 3 RS
 # Create rsgroup r1 and r2
 # Move rs1 to r1 and rs2 to r2 then all the regions are online on rs3
 # Stop rs3
 # Now access URL hmaster:Host:infoPort/master-status The page will not open.
Bug: Add a ClusterManager which submits command to ZooKeeper and its Agent which picks and execute those Commands.
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 23/Jun/20 11:38

Right now all the destructive actions are done through ssh, I have been working on Cluster Manager which submits tasks(commands) through ZooKeeper and an Agent which picks those tasks and executes them blindly. With this method we can use any destructive action that is present in hbase-it.
Bug: Add a ClusterManager that issues commands via coprocessor
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 24/Apr/20 19:31

I have a need to run the chaos monkey in an environment where ssh access is restricted. I can get most of what I need via {{RESTApiClusterManager}}, however {{kill -9}} is a critical and unsupported command. I have a {{ClusterManager}} that implements {{kill}} via an executor coprocessor.
Bug: Implement meta split
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/May/20 15:16

Need to modify Split procedures to deal with meta, and also remove all the hard coded non-splittable check for meta region.

And we also need to think of the split policy for meta table, i.e, whether it is safe tosplit at any point. For safety, maybe we could first use a special split policy to make sure that all records for a table are in the same region.
Bug: Allow unset table's rsgroup
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 29/Jun/20 06:16
Affects Version/s: 3.0.0-alpha-1

In TableDescriptorBuilder, we have only one method to set rsgroup, but have no one to unset it. this unset method is necessary In some cases.

If the table had rsgroup config before, but now I want to use the namespace config. It doesn't work that I set table rsgroup config to default rsgroup, must remove rsgroup config.
Bug: HBase table level replication metrics for shippedBytes are always 0
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 11/May/20 15:58
Affects Version/s: 1.7.0

It was observed during some investigations that table level metrics for shippedBytes are always 0 consistently even though data is getting shipped.

There are two problems with table-level metrics:
 # There are no table-level metrics for shipped bytes.
 # Another problem is that it's using `MetricsReplicationSourceSourceImpl` which is creating all source-level metrics at table level as well but updated only ageOfLastShippedOp. This reports lot of false/incorrect replication metrics at table level.
Bug: Allow operator to limit total normalization work per invocation
Status: Resolved | Priority: Major | Resolution: Won't Fix
Created: 28/May/20 23:06

During review on HBASE-24418, we observed that there's no way for an operator to limit the total amount of work a normalizer invocation will do. One suggestion was allowing settings for maximum number of plan executions per table or maximum number of plan executions per run. I've seen other systems limit the amount of CPU time permitted for a given run.

At least we have a run lock that prevents multiple invocations from running concurrently, which should prevent one run from starting before the previous one finished.
Bug: Provide more information about  merged child regions in Hbck Overlaps section, which cannot be fixed immediately 
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 14/May/20 05:10
Affects Version/s: 2.3.0

Right now, in Master's hbck page, pairs of overlap regions are listed. 

with Hbck2 -fixMeta, it will merge these overlap regions. However, if one of the region is a newly merged child region and GC has not kicked in yet, they are not fixable (merge will fail).

Users get confused as why after HBCK2 -fixMeta and still there are overlaps. To avoid this confusion, propose a new subsection saying that master is doing GC on these regions and these regions cannot be fixed/merged at this moment, wait until these regions are ready (not showing up in this subsection) to start the fix.
Bug: Replica region needs to check if primary region directory exists at file system  in TransitRegionStateProcedure 
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 12/Jun/20 23:15
Affects Version/s: 2.3.0

In hbase-1, it always runs into the situation that primary region has been closed/removed and replica region still stays in master's in-memory db and open at one of the region servers. Balancer can move this replica region to a new region server. During the region open, replica region does not check if primary region has been removed and moves forward. During store open, it will recreates primary region directory at hdfs and caused inconsistency.

 

In hbase-2, things get much better. To prevent the above inconsistency from happening, it adds more checks for a replica region, i.e, if primary regions' directory exists and there is a .regioninfo under.
Bug: Region normalizer now respects a rate limit
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 24/Jun/20 18:53
Affects Version/s: 2.4.0

There's no limits on the normalizer right now. It will iterate through the tables one at a time until it's touched all tables. For any table, it generates a complete plan and executes that plan in totality.

It would be nice to allow operators to configure some limits here. Off the top of my head, the two metrics that might be interesting are {{split|merge actions / hour}} or {{hfile mb volume / hour}}. Either way, we'd need to track a little more metadata.
Bug: Optionally ignore edits for deleted CFs for replication.
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 12/May/20 09:45
Affects Version/s: 2.2.4

Replication will be stuck after we delete CFs from both the source and the sink, if the source still has outstanding edits that now it could not get rid of. Now all replication is backed up behind these unreplicatable edits.
We should have an option to ignore edits for deleted CFs at the source.

This issue is similar to [HBASE-12091|https://issues.apache.org/jira/browse/HBASE-12091]
Bug: Normalizer should have configuration for minimum region size
Status: Resolved | Priority: Major | Resolution: Won't Fix
Created: 28/May/20 23:16

Another issue raised in PR review on HBASE-24418. With the current implementation, the lower-bound on how small the normalizer will make a region is its lowest granularity of size measurement: 1mb. That means it will happily balance a table out to be 1000 * 1mb regions (probably not likely, but seems plausible). The proposal was to add configuration knob that specifies the lower bound on region size as a guard against excessive splitting, something of an analogue to {{hbase.hregion.max.filesize}}.

I'm not convinced this is really needed; I think the table would need to have been crafted intentionally to trigger this behavior. However, it seems like a reasonable guard rail to install. Discuss.
Bug: Increase default Normalizer interval
Status: Resolved | Priority: Major | Resolution: Won't Fix
Created: 25/Jun/20 19:52

Our current default interval for the normalizer chore is 5 minutes. I think that's super aggressive for a background process that's intended to nudge a cluster toward healthy state, considering there's no rate-limiting in place at all (HBASE-24628). IIRC, the default used to be 30 minutes. Maybe we decide on a new default after we decide on a rate limit.
Bug: Normalizer should consider region max file size when planning merges
Status: Resolved | Priority: Major | Resolution: Not A Problem
Created: 28/May/20 23:20

When the normalizer plans its actions, it does not consider {{hbase.hregion.max.filesize}}. This means it could get into a merge/split loop in collaboration with the region server: the normalizer sees two regions are smaller than the table average, it merges them. The resulting region is larger that this max file size, so the region server splits them. Repeat.
Bug: Up the container nproc uplimit from 10000 to 12500
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 06/Apr/20 20:43

Let me up the container nproc count from 10000. If many tests running in parallel, container could breach the 10000 limit: i.e. if a bunch of heavy-duty long tests run at same time (0.5C on a 16CPU machine means 8 possible concurrents) and we spin up 1-2000 threads in each test, then could hit the 10000 limit. Move up the limit some expecially after INFRA just upped our limit to 30000.
Bug: [Flakey Tests] branch-2 TestMasterNoCluster.testStopDuringStart
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 29/Jun/20 17:57

org.apache.hadoop.hbase.master.TestMasterNoCluster.testStopDuringStart is (only) flakey on branch-2 currently. Fails here:

Error Message

KeeperErrorCode = Directory not empty for /hbase/backup-masters

Stacktrace

org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /hbase/backup-masters
	at org.apache.hadoop.hbase.master.TestMasterNoCluster.tearDown(TestMasterNoCluster.java:121)



I can see the zk events in teardown as we purge children as part of cleanup. Can also see that the backup master registers later. Other than that, log is opaque on why the teardown is failing. This is just clean up so adding in retry to see if that helps.
Bug: Introduce a CatalogAccessor
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 21/Jun/20 12:44

Move some common methods for accessing root/meta to this util class.

Notice that, root table is stored in a master local region, so the access API is a bit different comparing to a table.

Maybe we could make use of the RegionAsTable class under the test code to unify the API?
Bug: Make sure we can rollingupgrade from a non-splittable meta to splittable meta
Status: Open | Priority: Major | Resolution: 
Created: 17/May/20 15:19

Maybe we should set a DisabledSplitPolicy on existing meta table to prevent it being split, so we could still store the location of meta on zk so old client can still communicate with us.

And later once all the old clients are also upgraded, we remove the policy from meta table to make it splittable.
Bug: 'scandetail' log message is missing when responseTooSlow happens on the first scan rpc call
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 29/Apr/20 08:51

When responseTooSlow happens, the 'scandetail' message is printed to the warn log. But when the call is the first scan rpc call, this message is missing. This is because we get the 'scandetail' message with scannerId, but the first scan rpc call doesn't have scannerId yet.
Bug: Normalize one table at a time
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 24/Jun/20 18:46
Affects Version/s: 2.4.0

Out API and shell command around Normalizer is an all-or-nothing invocation. We should support an operator requesting to normalize a table at a time.

One use-case is for someone wanting to enable the normalizer for the first time. It would be nice to do a controlled roll-out of the normalizer, keeping it disabled at first, calling normalize one table at a time, and then turning it on after all tables have been normalized.
Bug: Make hbck chore aware of replica region and check/fix replica region consistency
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 15/Jun/20 16:34
Affects Version/s: 2.3.0

Hbck1 checks/fix only primary region consistency and ignores replica region. In hbase 2, hbck chore needs to be aware of replica region and check its consistency as well. Hbck2 needs to fix replica region inconsistency.
Bug: Heterogeneous Storage for Date Tiered Compaction
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 30/Apr/20 04:48

Support DateTiredCompaction(HBASE-15181) for cold and hot data separation, support different storage policies for different time periods of data to get better performance, for example, we can configure the data of last 1 month in SSD, and 1 month ago data was in HDD.

design doc: 

https://docs.google.com/document/d/1fk_EWLNnxniwt3gDjUS_apQ3cPzn90AmvDT1wkirvKE/edit?usp=sharing
Bug: Allow module tests run in parallel
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 09/Apr/20 04:03

Parent issue is about upping forkcount. This one is about mildly letting modules run tests in parallel. Parent wants forkcount to be 0.5C so we use half the processors. This is about keeping forkcount at 0.25C but allowing two modules to run their tests at same time when mvn figures it is possible.

In a test on a local machine, using forkcount of 1.0C and -T == 1 -- default -- the full test suite took 1hr 15mins. With forkcount at 0.5 and T == 2, full suite took 53mins.
Bug: do-release need not wait for tag
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 16/Jun/20 03:36

Making release failed waiting for tag to propagate to GitHub. On inspection, it seems the GitHub url is missing host information.

{noformat}
Waiting up to 30 seconds for tag to propagate to github mirror...
+ sleep 30
+ max_propagation_time=0
+ check_for_tag 2.3.0RC0
+ curl -s --head --fail /releases/tag/2.3.0RC0
+ ((  max_propagation_time <= 0  ))
+ echo 'ERROR: Taking more than 5 minutes to propagate Release Tag 2.3.0RC0 to github mirror.'
ERROR: Taking more than 5 minutes to propagate Release Tag 2.3.0RC0 to github mirror.
{noformat}
Bug: Normalizer can't actually merge empty regions when neighbor is larger than average size
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/Jun/20 22:37
Affects Version/s: 2.3.0

There are plenty of cases where empty regions can accumulate -- incorrect guessing at split points, old data is automatically expiring off, &c. The normalizer stubbornly refuses to handle this case, despite this being an original feature it was intended to support (HBASE-6613).

Earlier discussion had concerns for a user pre-splitting a table and then the normalizer coming along and merging those splits away before they could be populated. Thus, right now, the default behavior via {{hbase.normalizer.merge.min_region_size.mb=1}} is to not split any region that's so small. Later, we added {{hbase.normalizer.merge.min_region_age.days=3}}, which prevents us from merging any region too young. So there's plenty of nobs for an operator to customize their behavior.

But when I set {{hbase.normalizer.merge.min_region_size.mb=0}}, I still end up with stubborn regions that won't merge away. Looks like a large neighbor will prevent a merge from going through.
Bug: Change the return types of the new checkAndMutate methods introduced in HBASE-8458
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 28/Jun/20 00:47

To support CheckAndMutate with Increment/Append, the new checkAndMutate methods introduced in HBASE-8458 need to return the result of the specified Increment/Append operation in addition to a boolean value represents whether it's successful or not. Currently, the methods return only boolean value(s), so we need to change the return types of the methods. The methods are unreleased yet currently, so I think it's no problem to change the return types of the methods.
Bug: Flakey Test: TestReplicationAdminWithClusters timeout 
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 06/Apr/20 06:56
Affects Version/s: 2.3.0

{code:java}
2020-04-05 23:36:53,092 ERROR [RS_REFRESH_PEER-regionserver/asf905:0-0.replicationSource,2.replicationSource.shipperasf905.gq1.ygridcore.net%2C42849%2C1586129728118,2] regionserver.ReplicationSource(397): Unexpected exception in RS_REFRESH_PEER-regionserver/asf905:0-0.replicationSource,2.replicationSource.shipperasf905.gq1.ygridcore.net%2C42849%2C1586129728118,2 currentPath=hdfs://localhost:34203/user/jenkins/test-data/03854f9d-2780-eeaa-9645-c341240b62bf/WALs/asf905.gq1.ygridcore.net,42849,1586129728118/asf905.gq1.ygridcore.net%2C42849%2C1586129728118.1586129730509
java.lang.RuntimeException: Thread is interrupted, the replication source may be terminated
	at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.interruptOrAbortWhenFail(ReplicationSourceManager.java:477)
	at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:519)
	at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.updateLogPosition(ReplicationSourceShipper.java:264)
	at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:160)
	at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:118) {code}
Bug: Add a new option of designatedfile in RegionMover
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 15/Jun/20 06:42
Affects Version/s: 2.2.5

We have "excludefile" option in RegionMover to exclude specific regionservers during unloading. This Jira adds a new option "designatedfile" to designated regionservers during unloading.