av9ash/GitBugs huggingface dataset:
Bug: Flaky tests due to ZK client name resolution delays
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 29/Jan/20 17:52
Affects Version/s: 3.0.0-alpha-1

[~ndimiduk] and I ran into this issue (separately) and we noticed that there are some performance issues with name resolution in the Zookeeper client. Since we use ZK heavily in the unit tests, this often manifests as the following issues 

1. Test time outs starting the mini cluster (Master failed to start....)
2. InterruptedException (because the tests timeout)
3. Flaky tests because a subset of the cluster fails to start for whatever reason (replication tests especially because they spawn multiple clusters).
4. ConnectionLoss to znode /hbase/xyzz.. JVM pause?

I have strong feeling that this is a possible cause for many of our flaky tests in Jenkins. Luckily, it looks like the following workaround to switch to an IP address instead of hostname makes it much quicker. There are some related discussions in the ZK community (ZOOKEEPER-1666 and related jiras).


{code:java}
--- a/hbase-common/src/main/resources/hbase-default.xml
+++ b/hbase-common/src/main/resources/hbase-default.xml
@@ -72,7 +72,7 @@ possible configurations would overwhelm and obscure the important.
   </property>
   <property>
     <name>hbase.zookeeper.quorum</name>
-    <value>localhost</value>
+    <value>127.0.0.1</value>
     <description>Comma separated list of servers in the ZooKeeper ensemble
     (This config. should have been named hbase.zookeeper.ensemble).
     For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
{code}


Until we figure out the actual root cause and a dependency upgrade (if needed), we should consider making this hostname to IP switch for more stable builds.
Bug: [Flakey Tests] TestRegionReplicasWithRestartScenarios#testWhenRestart
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 30/Jan/20 06:09

Fails about 35% of the time in the GCE build. Let me attach a picture from current flakies dashboard for branch-2.

The test starts a cluster of three RS w/ 3 region replicas. It then stops a server, starts a new one, and then expects that the remaining three nodes do not have instances where two region replicas have landed on a single server.

It fails sporadically (reproducible locally) because when the SCP runs its assign, sometimes timing has it so Master knows of two servers only.  Making the new start before the old one is stopped (instead of other way around) seems to fix the test -- there'll be three servers up when SCP runs.
Bug: Edit of test classifications
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 31/Jan/20 00:47

Our test classifications have drifted. You can see for yourself running each of the small/medium and large test suites. See test complete times. See how even some large tests should be small and vice versa.

The more small tests we can run inside the single JVM, the faster we'll get through the build. Tests that are Medium start their own JVM for each test. Tests that are Medium but only last a second or two are expensive and should be aggregated with other single, short tests to amortize the JVM startup.

Anyways, let me edit the test categories and try and clean them up some.
Bug:  [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 04/Feb/20 01:30

Interesting one. Fails only if Master gets chance to become active -- which doesn't happen when all is easy-going. If struggling under load, it can become active and then test asserting NO ChunkCreator instance in Master fails because we want ChunkCreator now since ProcedureRegionStore was added: i.e. "    // always initialize the MemStoreLAB as we use a region to store procedure now."

Here is error I've seen

{code}
 [ERROR] Failures:
 [ERROR]   TestMasterNotCarryTable.testMasterMemStoreLAB:94 expected null, but was:<org.apache.hadoop.hbase.regionserver.ChunkCreator@73dbba7a>
{code}
Bug: [Flakey Tests] ERROR [Time-limited test] balancer.HeterogeneousRegionCountCostFunction(199): cannot read rules file located at ' /tmp/hbase-balancer.rules '
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 04/Feb/20 05:49

We can't find the balancer rules we just read in the HeterogeneousRegionCountCostFunction test in high load conditions

{code}
 2020-02-03 20:51:00,774 ERROR [Time-limited test] balancer.HeterogeneousRegionCountCostFunction(199): cannot read rules file located at ' /tmp/hbase-balancer.rules ':File /tmp/hbase-balancer.rules does not exist
 2020-02-03 20:51:00,774 WARN  [Time-limited test] balancer.HeterogeneousRegionCountCostFunction(155): cannot load rules file, keeping latest rules file which has 1 rules
{code}

Test then goes on to fail with:
{code} org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerHeterogeneousCost.testOneGroup  Time elapsed: 15.223 s  <<< FAILURE!
 junit.framework.AssertionFailedError: Host rs0 should be below 0.0%
   at org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerHeterogeneousCost.testWithCluster(TestStochasticLoadBalancerHeterogeneousCost.java:209)
   at org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerHeterogeneousCost.testHeterogeneousWithCluster(TestStochasticLoadBalancerHeterogeneousCost.java:160)
   at org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerHeterogeneousCost.testOneGroup(TestStochasticLoadBalancerHeterogeneousCost.java:102)
{code}

Instead, have tests write rules to local test dir.
Bug: [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did not replicate enough rows expected:<10> but was:<7>
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 07/Feb/20 22:26

I've seen this fail. Here is the latest:
{code}
 [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 21.044 s <<< FAILURE! - in org.apache.hadoop.hbase.replication.regionserver.TestReplicator
 [ERROR] org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors  Time elapsed: 7.139 s  <<< FAILURE!
 java.lang.AssertionError: We did not replicate enough rows expected:<10> but was:<7>
   at org.apache.hadoop.hbase.replication.regionserver.TestReplicator.testReplicatorWithErrors(TestReplicator.java:158)
{code}

In test logs there are a few (in groups of three) of these:
{code}
 4247 2020-02-07 13:50:29,070 WARN  [RS_REFRESH_PEER-regionserver/localhost:0-1.replicationSource,testReplicatorWithErrors.replicationSource.shipperlocalhost%2C49531%2C1581112194639,testReplicatorWithErrors] regionserver.                       ReplicationSourceShipper(223): org.apache.hadoop.hbase.replication.regionserver.TestReplicator$FailureInjectingReplicationEndpointForTest threw unknown exception:
 4248 java.lang.ClassCastException: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException cannot be cast to java.io.IOException
 4249   at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.parallelReplicate(HBaseInterClusterReplicationEndpoint.java:361)
 4250   at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:404)
 4251   at org.apache.hadoop.hbase.replication.regionserver.TestReplicator$ReplicationEndpointForTest.replicate(TestReplicator.java:228)
 4252   at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:188)
 4253   at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:118)
{code}

Let me fix this and see if it makes TestReplicator stable again.
Bug: [Flakey Test] AbstractTestAsyncTableRegionReplicasRead family of tests fails with no breadcrumbs on why
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 08/Feb/20 17:44

Running locally w/ upped forkcount and maven threading > 1, the set of tests that are derived from AbstractTestAsyncTableRegionReplicasRead fail near each time often the only ones failed (Get and Scan tests). They don't give clues why died. The xml has no info in it and no -output.txt is dropped. The .txt just says failed startup.

I've tried giving the forked jvms more RAM, messed w/ file counts....  Removing all of the custom setup seems to be a breakthrough. Let me put up a patch.
Bug: TestSnapshotScannerHDFSAclController is flakey
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 10/Feb/20 10:04

See  [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html]
Bug: Reconcile Hadoop version
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/Feb/20 17:02
Affects Version/s: 3.0.0-alpha-1

I followed the HBase book http://hbase.apache.org/book.html#maven.build.hadoop and wanted to build HBase (master) on top of Hadoop 3.2/3.3 but tests failed right away.

Build: 
{code}
mvn clean install -Dhadoop.profile=3.0 -Dhadoop-three.version=3.2.1 -DskipTests
{code}

Test:
{code}
mvn test -Dtest=TestHelloHBase -Dhadoop.profile=3.0 -Dhadoop-three.version=3.2.1
{code}

{noformat}
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.296 s <<< FAILURE! - in org.apache.hbase.archetypes.exemplars.shaded_client.TestHelloHBase
[ERROR] org.apache.hbase.archetypes.exemplars.shaded_client.TestHelloHBase  Time elapsed: 1.284 s  <<< ERROR!
java.lang.NoClassDefFoundError: org/apache/hadoop/hdfs/protocol/HdfsConstants$StoragePolicySatisfierMode
at org.apache.hbase.archetypes.exemplars.shaded_client.TestHelloHBase.beforeClass(TestHelloHBase.java:54)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hdfs.protocol.HdfsConstants$StoragePolicySatisfierMode
at org.apache.hbase.archetypes.exemplars.shaded_client.TestHelloHBase.beforeClass(TestHelloHBase.java:54)
{noformat}

Adding mvn -X parameter, I was able to tell that it was because the hbase-server module includes hadoop-distcp and hadoop-dfs-client 3.1.2 (the default Hadoop 3 dependency version) while it uses version 3.2.1 of other hadoop jars . The classpath conflict (the storage policy satisfier is a new feature in Hadoop 3.2) failed the test.

This is reproducible on any Hadoop version 3.2 and above. It looks to me the version of hadoop-distcp and hadoop-hdfs-client should be specified at the top level pom (they are specified in hbase-server/pom.xml).
Bug: Add necessary validations to HRegion.checkAndMutate() and HRegion.checkAndRowMutate()
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 21/Mar/20 08:41

Need to check that the following are fulfilled:

# The mutation type should be Put or Delete in HRegion.checkAndMutate().
# The specified row should match the row of the Put/Delete/RowMutations in HRegion.checkAndMutate() and HRegion.checkAndRowMutate().
Bug: Fix missing jdk8 dependencies in hbase-assembly/hadoop-two-compat
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 31/Mar/20 00:15
Affects Version/s: 2.3.0

Trying to launch a standalone master out of binary tarball, doesn't start.

{noformat}
$ JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home ~/tmp/packaging_and_integration/hbase-install/bin/hbase master start
2020-03-30 17:14:21,831 INFO  [main] master.HMaster: STARTING service HMaster
2020-03-30 17:14:21,833 INFO  [main] util.VersionInfo: HBase 3.0.0-SNAPSHOT
2020-03-30 17:14:21,833 INFO  [main] util.VersionInfo: Source code repository file:///Users/ndimiduk/tmp/packaging_and_integration/unpacked_src_tarball revision=Unknown
2020-03-30 17:14:21,833 INFO  [main] util.VersionInfo: Compiled by ndimiduk on Mon Mar 30 14:59:17 PDT 2020
2020-03-30 17:14:21,833 INFO  [main] util.VersionInfo: From source with checksum c13d259fe1832de6fef3f5d192e6651d3ca0b52599c64ef19eaa87c6bdbd3956cc51fda3c3dc917b47add5a633c9058a6968df24ddce88f85c345681763cd568
Exception in thread "main" java.lang.NoClassDefFoundError: com/ctc/wstx/io/InputBootstrapper
        at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
        at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3062)
Caused by: java.lang.ClassNotFoundException: com.ctc.wstx.io.InputBootstrapper
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 2 more
{noformat}
Bug: [Flakey Test] TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 19/Feb/20 06:38

Test has this on it:

@Test // Test is flakey. TODO: Fix!

Let me cut it down. It runs for a while.

It fails nearly every run since     HBASE-23865 Up flakey history from 5 to 10 which upped the flakey test so it ran with more fork count.
Bug: [Flakey Tests] Disable TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 25/Mar/20 22:40

Fails sporadically with relogin in progress complaints. See https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html

FYI [~elserj]

i.e. 

org.apache.hadoop.hbase.security.provider.TestCustomSaslAuthenticationProvider.testNegativeAuthentication[2: rpcClientImpl=org.apache.hadoop.hbase.ipc.NettyRpcClient, rpcServerImpl=org.apache.hadoop.hbase.ipc.SimpleRpcServer]

{code}
Failed after attempts=4, exceptions:
2020-03-25T21:25:05.239Z, RpcRetryingCaller{globalStartTime=1585171502143, pause=100, maxAttempts=4}, org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=4, exceptions:
2020-03-25T21:25:04.624Z, RpcRetryingCaller{globalStartTime=1585171504623, pause=100, maxAttempts=4}, java.io.IOException: Call to asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception: java.io.IOException: Can not send request because relogin is in progress.
2020-03-25T21:25:04.726Z, RpcRetryingCaller{globalStartTime=1585171504623, pause=100, maxAttempts=4}, java.io.IOException: Call to asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception: java.io.IOException: Can not send request because relogin is in progress.
2020-03-25T21:25:04.929Z, RpcRetryingCaller{globalStartTime=1585171504623, pause=100, maxAttempts=4}, java.io.IOException: Call to asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception: java.io.IOException: Can not send request because relogin is in progress.
2020-03-25T21:25:05.235Z, RpcRetryingCaller{globalStartTime=1585171504623, pause=100, maxAttempts=4}, java.io.IOException: Call to asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception: java.io.IOException: Can not send request because relogin is in progress.
{code}

Disabling rather than spending time in here.
Bug: Bump hadoop-two.version to 2.10.0 on master and branch-2
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 14/Mar/20 01:49

See this discussion on mailing list

https://lists.apache.org/thread.html/rbab2e79f25b38c85fe205390b3fdbc6711984881a994a499c54aee97%40%3Cdev.hbase.apache.org%3E
Bug: MapReduce on snapshot restores and opens all regions in each mapper
Status: Open | Priority: Major | Resolution: 
Created: 20/Mar/20 22:35
Affects Version/s: 1.6.0

Given this scenario: one MR job scans a table (with many regions). I will use 'RestoreSnapshotHelper' to restore snapshot for all regions in each mapper. 

In the code [https://github.com/apache/hbase/blob/branch-2.0/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java#L183]

Seems there is no way to only restore relevant regions from snapshot to region.

This leads to extreme slowness and waste of resource. 

 

 

One quick example I can show as below, in my test, there are 2 regions in a testing table, each mapper opens and iterates 2 regions.  (which is wrong IMO, each mapper should only touch 1 region based on the splits) I have checked the splits are correct, which have  correct startKey and endKey

2020-03-19 18:58:15,225 INFO [main] mapred.MapTask - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
 2020-03-19 18:58:15,285 INFO [main] snapshot.RestoreSnapshotHelper - region to add: *d7f85b4a9d3fa22a5e7b88bda39f6d50*
 2020-03-19 18:58:15,285 INFO [main] snapshot.RestoreSnapshotHelper - region to add: *69dd3fdba3698f827f8883ed911161ef*
 2020-03-19 18:58:15,286 INFO [main] snapshot.RestoreSnapshotHelper - clone region=d7f85b4a9d3fa22a5e7b88bda39f6d50 as d7f85b4a9d3fa22a5e7b88bda39f6d50

 

Please correct me if I am wrong or miss anything. thanks.

So if I misunderstood anything, can anyone point to me where in this class, can distinguish which region to go through for different mappers? 

 

btw the original implementation for MR on Snapshot is here, there weren't too many big changes after that HBASE-8369
Bug: [RSGroup] Assign created tables to respective rsgroup automatically instead of manual operations 
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 21/Mar/20 10:53

In prod practise, it's time consuming and tiring to assign tables to their rsgroups for HBase admin by shell, especially in 1.x rsgroup client is IA.private, we could not even write some programs to do that.

And I'm here proposing a solution to do the automated rsgroup assignment by allowing running a shell in postCreateTable to determine which table should go to which rsgroup (way in branch-1 not master). The benefit is obvious, admin can determine or change at any time the mapping rule in script without restarting or re-config anything. It's flexible, convenience and time saving.
Bug: /hadoop-3.1.3/libexec/hadoop-functions.sh:行2360: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_USER: 错误的替换
Status: Resolved | Priority: Major | Resolution: Invalid
Created: 03/Feb/20 07:34
Affects Version/s: 2.2.3

/hadoop-3.1.3/libexec/hadoop-functions.sh:行2360: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_USER: 错误的替换
/hadoop-3.1.3/libexec/hadoop-functions.sh:行2455: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_OPTS: 错误的替换

Hadoop版本3.1.3

分布式安装启动报错
Bug: shell fails to launch out of binary tarball
Status: Open | Priority: Major | Resolution: 
Created: 31/Mar/20 00:18
Affects Version/s: 3.0.0-alpha-1

Trying to launch the shell out of binary tarball, doesn't launch.

{noformat}
$ JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home ~/tmp/packaging_and_integration/hbase-install/bin/hbase shell
NameError: missing class name (`org.apache.hadoop.hbase.HColumnDescriptor')
           method_missing at org/jruby/javasupport/JavaPackage.java:259
  <module:HBaseConstants> at /Users/ndimiduk/tmp/packaging_and_integration/hbase-install/lib/ruby/hbase_constants.rb:42
                   <main> at /Users/ndimiduk/tmp/packaging_and_integration/hbase-install/lib/ruby/hbase_constants.rb:34
                  require at org/jruby/RubyKernel.java:956
                   (root) at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:1
                   <main> at /Users/ndimiduk/tmp/packaging_and_integration/hbase-install/bin/../bin/hirb.rb:131
{noformat}
Bug: [Flakey Tests] Reenable TestThriftHBaseServiceHandler#testLongLivedScan
Status: Open | Priority: Major | Resolution: 
Created: 30/Mar/20 21:46

Disabled by parent issue. Fails with

{code}
  //  Caused by: java.util.concurrent.RejectedExecutionException:
  //  Task org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@e385431
  //  rejected from java.util.concurrent.ThreadPoolExecutor@   52b027d[Terminated, pool size = 0,
  //  active threads = 0, queued tasks = 0, completed tasks = 1]
  //  at org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler.
  //  testLongLivedScan(TestThriftHBaseServiceHandler.java:804)
{code}

Reenable.
Bug: [Flakey Tests] Renenable TestBucketCacheRefCnt.testBlockInRAMCache
Status: Open | Priority: Major | Resolution: 
Created: 30/Mar/20 21:32

Disabled in parent issue.

Fails like this. Rare but enough times to mess up runs.

TestBucketCacheRefCnt.testBlockInRAMCache:121 expected:<3> but was:<2>
Bug: Enable SpotBugs in PreCommit
Status: Resolved | Priority: Major | Resolution: Not A Problem
Created: 26/Mar/20 17:28
Affects Version/s: 2.3.0

Yetus's SpotBugs module depends on {{maven_add_install}}. In {{Jenkinsfile_GitHub}}, I'm pretty strict on my module definitions, with

{noformat}
GENERAL_CHECK_PLUGINS = 'all,-compile,-javac,-javadoc,-jira,-shadedjars,-unit'
JDK_SPECIFIC_PLUGINS = 'compile,github,htmlout,javac,javadoc,maven,mvninstall,shadedjars,unit'
{noformat}

The general check requests {{all}} but omits {{compile}} and {{maven}}, which I think means the {{spotbugs}} check gets dropped. Before HBASE-23767, we had just a single yetus that did {{all}}, so {{spotbugs}} would have been run.
Bug: FileSystemUtilizationChore reduces unnecessary regions
Status: Open | Priority: Major | Resolution: 
Created: 23/Feb/20 05:59
Affects Version/s: 2.2.3

If the RS has 2000 reegions, and only one region1 has been disk space quota. 
FileSystemUtilizationChore can reduce unnecessary regions. 
 RS only need to collect the region1 size

!image-2020-02-23-13-59-32-894.png|width=608,height=138!
Bug: Snapshoting a splitting region results in corrupted snapshot
Status: Open | Priority: Major | Resolution: 
Created: 16/Mar/20 12:44
Affects Version/s: 2.0.2

The problem seems to originate from the fact that while the region split itself runs in a lock, the compactions following it run in separate threads. Alternatively the use of space quota policies can prevent compaction after a split and leads to the same issue.

In both cases the resulting snapshot will keep the split status of the parent region, but do not keep the references to the daughter regions, because they (splitA, splitB qualifiers) are stored separately in the meta table and do not propagate with the snapshot.

This is important because the in the freshly cloned table CatalogJanitor will find the parent region, realizes it is in split state, but because it can not find the daughter region references (haven't propagated) assumes parent could be cleaned up and deletes it. The archived region used in the snaphost only has back reference to the now also archived parent region and if the snapshot is deleted they both gets cleaned up. Unfortunately the daughter regions only contains hfile links, so at this point the data is lost.

How to reproduce:
{code:java}
hbase shell <<EOF
create 'test', 'cf'
(0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
flush 'test'
split 'test'
snapshot 'test', 'testshot'
EOF
{code}
This should make sure the snapshot is made before the compaction could be finished even with small amount of data.
{code:java}
sudo -u habse hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot testshot -copy-to hdfs://target:8020/apps/hbase/data/
{code}
I export the snapshot to make the usecase cleaner but deleting both the snapshot and the original table after the cloning should have the same effect.
{code:java}
clone_snapshot 'testshot', 'test2'
delete_snapshot "testshot"
{code}
I'm not sure what would be the best way to fix this. Preventing snapshots when a region is in split state, would make snapshot creation problematic. Forcing to run compaction as part of the split thread would make it rather slow. Propagating the daughter region references could prevent the deletion of the cloned parent region and the data would not be broken anymore but I'm not sure we have a logic in place that could pick up the pieces and finish the split process.