av9ash/GitBugs huggingface dataset:
Bug: hbase.rs.evictblocksonclose is not honored when removing compacted files and closing the storefiles
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 10/Jun/20 04:36

Currently, when removing compacted files and closing the storefiles, RS always does evict block caches for the store files. It should honor hbase.rs.evictblocksonclose:
https://github.com/apache/hbase/blob/7b396e9b8ca93361de6a6c4bc8a40442db77c4da/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L2744
https://github.com/apache/hbase/blob/7b396e9b8ca93361de6a6c4bc8a40442db77c4da/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L625
Bug: Hadoop2&Hadoop3 profiles are both active when pre-commit PR builds run
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 10/May/20 17:26

We need the magic done in the parent out in our precommit builds too. See how https://github.com/apache/hbase/pull/1664 fails in hbase-rest w/ complaint about jersey; this is a symptom of double hadoop2+hadoop3 profile activation.
Bug: Support deleteall with ts but without column in shell mode
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 06/May/20 09:30
Affects Version/s: 3.0.0-alpha-1

The position after rowkey is column, so we can't only specify ts now.
 My proposal is use a empty string to represent no column specified.
 useage: 
 deleteall 'test','r1','',1588765900000
 deleteall 'test', \{ROWPREFIXFILTER => 'prefix'}, '', 1588765900000
Bug: GCMulitpleMergedRegionsProcedure  is not idempotent
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 04/May/20 16:36
Affects Version/s: 2.3.0

Currently deleteMergeQualifiers() is not idempotent. If two GCMulitpleMergedRegionsProcedures are run for the same merged child region, the second run will delete the row for the merge region from meta table and leave a hole. It needs to make sure it only deletes columns with merge qualifiers.

[https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java#L1849]
Bug: [flakey test] regionserver.TestRegionMergeTransactionOnCluster
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 09/Apr/20 19:07
Affects Version/s: 2.3.0

As reported in [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.3/lastSuccessfulBuild/artifact/dashboard.html#job_1,] 

testCleanMergeReference fails when waiting for merge table in meta table to be cleaned.

Debugged a bit, found that this is case that state is polluted from other testing cases.

 

the meta table is polluted from the previous region merge test cases (tables are not deleted after the testing case finishes). While compaction still goes on and janitor kicks in, it will clean up the old table state and breaks the loop (cleaned > 0), so leave the state under test never cleaned up.The solution is to delete tables after each test case.
Bug: When fixOverlap hits the max region limit, it is possible to include the same region in multiple merge request
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 24/Apr/20 16:50
Affects Version/s: 2.3.0

If the same region is included in multiple merge requests, the first merge cleans up the region state, the second merge request will fail with error msg "No state for region ****". The algo needs to be improved to make sure the same region is not included in multi merge requests.
Bug: Avoid aggressive MergeRegion and GCMultipleMergedRegionsProcedure 
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 14/May/20 05:27

In [https://github.com/apache/hbase/blob/a40a0322a73add68d9cb0579abacdd6a2e41e8fb/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java#L478,  |https://github.com/apache/hbase/blob/a40a0322a73add68d9cb0579abacdd6a2e41e8fb/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java#L478]

prepareMergeRegion, it checks if one of merged parent regions is a merged child region and has not been GCed. If it is ready to GC, it will kick off a GCMultipleMergedRegionsProcedure and also start the MergeRegionProcedure. There is a race condition here. If MergeRegionProcedure finishes first, it will delete meta row for the merged child region. Then GCMultipleMergedRegionsProcedure runs, and because the newly added check, it thinks GC has been done and wont schedule GCRegionProcedure to clean up those merged parent regions. The end result is that these merged parent regions are left as orphans on Filesystem.

 

[https://github.com/apache/hbase/blob/a40a0322a73add68d9cb0579abacdd6a2e41e8fb/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java#L105]

 

The proposed solution is to avoid being so aggressive, if it needs to kick off GCMultipleMergedRegionsProcedure, then abort MergeRegionProcedure and user can try MergeRegionProcedure later.

[|https://github.com/apache/hbase/blob/a40a0322a73add68d9cb0579abacdd6a2e41e8fb/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java#L478]
Bug: Deflake TestCompaction#testStopStartCompaction
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 31/May/20 22:45
Affects Version/s: 1.7.0

Saw this in our internal test runs. 

{noformat}
org.apache.hadoop.hbase.regionserver.TestCompactionWithCoprocessor.testStopStartCompaction

Failing for the past 1 build (Since Unstable#10 )
Took 0.72 sec.
add description
Error Message
expected:<1> but was:<0>
Stacktrace
java.lang.AssertionError: expected:<1> but was:<0>
{noformat}

The test asserts can be tightened to improve this. Will follow up with a patch shortly.  HBASE-23899 added some debug logging to figure out why.
Bug: Introduce a split policy similar with SteppingSplitPolicy but count all store size
Status: Resolved | Priority: Major | Resolution: Won't Do
Created: 10/Jun/20 08:19

SteppingSplitPolicy decide whether split by max store size, if there are several column families and write evenly, then the region will grows very big.
  
 In this case, count by all store size would be a better choise.
Bug: Rewrite TestClusterRestartFailover.test since namespace table is gone on on master
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 02/Jun/20 03:50

We still have this
{code}
    // Find server that does not have hbase:namespace on it. This tests holds up SCPs. If it
    // holds up the server w/ hbase:namespace, the Master initialization will be held up
    // because this table is not online and test fails.
    for (JVMClusterUtil.RegionServerThread rst:
        UTIL.getHBaseCluster().getLiveRegionServerThreads()) {
      HRegionServer rs = rst.getRegionServer();
      if (rs.getRegions(TableName.NAMESPACE_TABLE_NAME).isEmpty()) {
        SERVER_FOR_TEST = rs.getServerName();
      }
    }
{code}
Bug: Stabilize master startup with meta replicas enabled
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 15/Jun/20 14:55
Affects Version/s: 2.2.5

This is related to HBASE-21624 . 

I created a separate ticket because in the original one a "complete solution for meta replicas" was requested and this is not one. I'm just trying to make master startup more stable by making assigning meta replicas asynchronous and preventing a potential assignment failure from crashing master.

The idea is that starting master with less or even no meta replicas assigned is preferable to not having a running master.
Bug: Priority compaction for recently split daughter regions
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 25/May/20 17:12

We observe that under hotspotting conditions that splitting will proceed very slowly and the "_Cannot split region due to reference files being there_" log line will be logged excessively. (branch-1 based production.) This is because after a region is split it must be compacted before it can be split again. Reference files must be replaced by real HFiles, normal housekeeping performed during compaction. However if the regionserver is under excessive load, its compaction queues may become deep. The daughters of a recently split hotspotting region may themselves continue to hotspot and will rapidly need to split again. If the scheduled compaction work to remove/replace reference files is queued hundreds or thousands of compaction queue elements behind current, the recently split daughter regions will not be able to split again for a long time and may grow very large, producing additional complications (very large regions, very deep replication queues).

To help avoid this condition we should prioritize the compaction of recently split daughter regions. Compaction requests include a {{priority}} field and CompactionRequest implements a comparator that sorts by this field. We already detect when a compaction request involves a region that has reference files, to ensure that it gets selected to be eligible for compaction, but we do not seem to prioritize the requests for post-split housekeeping. Split work should be placed at the top of the queue. Ensure that this is happening.
Bug: RegionMover should exclude draining/decommissioning nodes from target RSs
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 02/Apr/20 06:38

When using RegionMover tool to unload the regions from a given RS, it decides the list of destination RSs by 
{code}
List<ServerName> regionServers = new ArrayList<>();
        regionServers.addAll(admin.getRegionServers());
        // Remove the host Region server from target Region Servers list
        ServerName server = stripServer(regionServers, hostname, port);
        .....
        // Remove RS present in the exclude file
        stripExcludes(regionServers);
        stripMaster(regionServers);
{code}
Ya it is removing the RSs mentioned in the exclude file.  
Better when the RegionMover user is NOT mentioning any exclude list, we can exclude the draining/decommissioning RSs
Admin#listDecommissionedRegionServers()
Bug: Procedure V2 - Distributed WAL Splitting => LOGGING
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 16/Jun/20 18:37

The procedure-based distributed WAL splitter uses the old SplitLogManager class. It registers it as a chore even though the chore #run immediately skips out. Meantime we log SLM ran in #0ms frequently. For example...
{code}
...
2020-06-17 17:48:30,567 DEBUG [localhost:16000.splitLogManager..Chore.1] hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
2020-06-17 17:48:31,567 DEBUG [localhost:16000.splitLogManager..Chore.1] hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
2020-06-17 17:48:32,570 DEBUG [localhost:16000.splitLogManager..Chore.1] hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
2020-06-17 17:48:33,567 DEBUG [localhost:16000.splitLogManager..Chore.1] hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
2020-06-17 17:48:34,569 DEBUG [localhost:16000.splitLogManager..Chore.1] hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
2020-06-17 17:48:35,569 DEBUG [localhost:16000.splitLogManager..Chore.1] hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
2020-06-17 17:48:36,568 DEBUG [localhost:16000.splitLogManager..Chore.1] hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
...
{code}

The new pv2 WAL splitter introduces new procedures that log their class name only w/o detail on log that is being split or the server that is running the split.
Bug: Set the log level for ScheduledChore to INFO in HBTU
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Jun/20 02:29

Now we changed the thead waken interval to 100ms in tests, which causes the test log beng flooded with

{noformat}
2020-06-27 10:28:00,680 DEBUG [regionserver/zhangduo-ubuntu:0.Chore.1] hbase.ScheduledChore(192): MemstoreFlusherChore execution time: 0 ms.
2020-06-27 10:28:00,680 DEBUG [regionserver/zhangduo-ubuntu:0.Chore.1] hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms.
2020-06-27 10:28:00,684 DEBUG [regionserver/zhangduo-ubuntu:0.Chore.1] hbase.ScheduledChore(192): MemstoreFlusherChore execution time: 0 ms.
2020-06-27 10:28:00,684 DEBUG [regionserver/zhangduo-ubuntu:0.Chore.1] hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms.
2020-06-27 10:28:00,696 DEBUG [regionserver/zhangduo-ubuntu:0.Chore.1] hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms.
2020-06-27 10:28:00,696 DEBUG [regionserver/zhangduo-ubuntu:0.Chore.1] hbase.ScheduledChore(192): MemstoreFlusherChore execution time: 0 ms.
2020-06-27 10:28:00,782 DEBUG [regionserver/zhangduo-ubuntu:0.Chore.1] hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms.
2020-06-27 10:28:00,782 DEBUG [regionserver/zhangduo-ubuntu:0.Chore.1] hbase.ScheduledChore(192): MemstoreFlusherChore execution time: 0 ms.
2020-06-27 10:28:00,786 DEBUG [regionserver/zhangduo-ubuntu:0.Chore.1] hbase.ScheduledChore(192): CompactionChecker execution time: 0 ms.
{noformat}

We could set the log level to INFO when starting a mini cluster in HBTU.
Bug: TestHeapSize.testSizes  failure
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 30/Jun/20 22:17
Affects Version/s: 3.0.0-alpha-1

{code:java}
INFO] 
[INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @ hbase-server ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.hbase.io.TestHeapSize
[ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.884 s <<< FAILURE! - in org.apache.hadoop.hbase.io.TestHeapSize
[ERROR] org.apache.hadoop.hbase.io.TestHeapSize.testSizes  Time elapsed: 0.308 s  <<< FAILURE!
java.lang.AssertionError: expected:<368> but was:<360>
	at org.apache.hadoop.hbase.io.TestHeapSize.testSizes(TestHeapSize.java:493)


[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   TestHeapSize.testSizes:493 expected:<368> but was:<360>
[INFO] 
[ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0
[INFO] 
{code}
Bug: Job failed because of TableInputFormatBase HRegionLocation.getHostname NullPointerException
Status: Open | Priority: Major | Resolution: 
Created: 30/Jun/20 18:28
Affects Version/s: 2.3.0

I've not seen this before. Failed the job.

{code}
00:30:11  Caused by: java.lang.NullPointerException
00:30:11  	at org.apache.hadoop.hbase.HRegionLocation.getHostname(HRegionLocation.java:107)
00:30:11  	at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.oneInputSplitPerRegion(TableInputFormatBase.java:337)
00:30:11  	at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:257)
00:30:11  	at org.apache.hadoop.hbase.mapreduce.TableInputFormat.getSplits(TableInputFormat.java:254)
00:30:11  	at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:130)
00:30:11  	at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:253)
00:30:11  	at scala.Option.getOrElse(Option.scala:189)
00:30:11  	at org.apache.spark.rdd.RDD.partitions(RDD.scala:251)
00:30:11  	at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49)
00:30:11  	at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:253)
00:30:11  	at scala.Option.getOrElse(Option.scala:189)
00:30:11  	at org.apache.spark.rdd.RDD.partitions(RDD.scala:251)
00:30:11  	at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49)
00:30:11  	at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:253)
00:30:11  	at scala.Option.getOrElse(Option.scala:189)
00:30:11  	at org.apache.spark.rdd.RDD.partitions(RDD.scala:251)
00:30:11  	at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49)
00:30:11  	at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:253)
00:30:11  	at scala.Option.getOrElse(Option.scala:189)
00:30:11  	at org.apache.spark.rdd.RDD.partitions(RDD.scala:251)
00:30:11  	at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49)
00:30:11  	at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:253)
00:30:11  	at scala.Option.getOrElse(Option.scala:189)
00:30:11  	at org.apache.spark.rdd.RDD.partitions(RDD.scala:251)
00:30:11  	at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:159)
00:30:11  	... 31 more
00:30:11  
00:30:11  	 ApplicationMaster host: ps1331.example.org
00:30:11  	 ApplicationMaster RPC port: 27682
00:30:11  	 queue: root.adhoc
00:30:11  	 start time: 1593502056697
00:30:11  	 final status: FAILED
00:30:11  	 tracking URL: http://web2.example.org:8088/proxy/application_1593477746222_0014/
00:30:11  	 user: nobody
{code}
Bug: Include target hostname/ip/port in `ipc.ServerNotRunningYetException`
Status: Open | Priority: Major | Resolution: 
Created: 08/Jun/20 22:58
Affects Version/s: 2.3.0

We get almost 100 lines of exception backtrace along with this exception, but the identity of the server with whom the client is trying to communication is not included. For example,

{noformat}
2020-06-06 00:35:37,123 WARN  [ChaosMonkey] client.ConnectionImplementation: Checking master connection
org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet
        at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2901)
        at org.apache.hadoop.hbase.master.MasterRpcServices.isMasterRunning(MasterRpcServices.java:1178)
        at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:393)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
...
{noformat}
Bug: Address asciidoc warnings about images
Status: Open | Priority: Major | Resolution: 
Created: 29/Jun/20 17:20

Noticed this while tailing the log for 2.3.0rc1. I assume this is related to generating the PDF version of the book. Should verify these images do indeed make it into the pdf.

{noformat}
17:15:36 [INFO] asciidoctor: WARN: image to embed not found or not readable: /home/vagrant/hbase-rm/output/hbase/src/main/asciidoc/images/WAL_splitting.png
17:16:37 [INFO] asciidoctor: WARN: image to embed not found or not readable: /home/vagrant/hbase-rm/output/hbase/src/main/asciidoc/images/master-snapshot.png                                         
17:16:37 [INFO] asciidoctor: WARN: image to embed not found or not readable: /home/vagrant/hbase-rm/output/hbase/src/main/asciidoc/images/1-snapshot.png
17:16:37 [INFO] asciidoctor: WARN: image to embed not found or not readable: /home/vagrant/hbase-rm/output/hbase/src/main/asciidoc/images/2-snapshots.png
17:16:37 [INFO] asciidoctor: WARN: image to embed not found or not readable: /home/vagrant/hbase-rm/output/hbase/src/main/asciidoc/images/empty-snapshots.png
17:17:03 [INFO] asciidoctor: WARN: GIF image format not supported. Install the prawn-gmagick gem or convert https://hbase.apache.org/hbtop-images/top_screen.gif to PNG.                              
17:17:03 [INFO] asciidoctor: WARN: GIF image format not supported. Install the prawn-gmagick gem or convert https://hbase.apache.org/hbtop-images/scrolling_metric_records.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the prawn-gmagick gem or convert https://hbase.apache.org/hbtop-images/changing_mode.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the prawn-gmagick gem or convert https://hbase.apache.org/hbtop-images/changing_refresh_delay.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the prawn-gmagick gem or convert https://hbase.apache.org/hbtop-images/changing_displayed_fields.gif to PNG.               
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the prawn-gmagick gem or convert https://hbase.apache.org/hbtop-images/changing_sort_field.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the prawn-gmagick gem or convert https://hbase.apache.org/hbtop-images/changing_order_of_fields.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the prawn-gmagick gem or convert https://hbase.apache.org/hbtop-images/adding_filters.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the prawn-gmagick gem or convert https://hbase.apache.org/hbtop-images/showing_and_clearing_filters.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the prawn-gmagick gem or convert https://hbase.apache.org/hbtop-images/driling_down.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the prawn-gmagick gem or convert https://hbase.apache.org/hbtop-images/help_screen.gif to PNG.
{noformat}
Bug: Remove BoundedRecoveredHFilesOutputSink  dependency on a TableDescriptor
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 23/Jun/20 00:04

BoundedRecoveredHFilesOutputSink wants to read TableDescriptor so it writes the particular hfile format specified by a table's schema. Getting the table schema can be tough at various points of operation especially around startup. HBASE-23739 tried to read from the fs if unable to read TableDescriptor from Master. This approach works generally but fails in standalone mode as in standalone mode we will have given-up our start up attempt BEFORE the request to Master for TableDescriptor times out (the read from fs is never attempted).

The suggested patch here does away w/ reading TableDescriptor and just has BoundedRecoveredHFilesOutputSink write generic hfiles.
Bug: TestSimpleRegionNormalizerOnCluster is broken
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 27/Jun/20 13:30

It failed for all the recent pre commit build, and has started to show on the flaky page.

Spent some time when merging HBASE-24389, copy the comments from PR here

It seems the problem is that while issuing a split, the normalizer also issues a merge, which lead to an unchanged region count and make the tests timeout.

Not sure what is the root cause yet. HBASE-24588 maybe?
Bug: Normalizer plan execution is not consistent between plan types
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 18/Jun/20 23:50
Affects Version/s: 2.3.0

I left a comment on a merged [commit|https://github.com/apache/hbase/commit/5d0e0fc5fd09bddb2d766d1e24e28e472961f454#r39987289] where a little discussion has blossomed. Right now the normalizer produces two types of plans: "split" or "merge". The master receives the list of plans and executes them in a simple loop. The way it does the actual execution is delegated off to the plan implementation.

The bug I noticed is that the two implementations are subtly different. Both use async APIs to submit procedures, but "split" blocks on completion while "merge" does not. Furthermore, because "split" blocks, it's able to capture any exception that's thrown, while "merge" cannot.

These implementations should be made consistent. My thinking at the moment is this {{execute}} method should instead be named {{submit}}, creating and returning the {{Future}} that represents whatever work it submitted, and the calling context should handle the resolution of those futures in a single place.