Branch to migrate pipeline_resync changes to 1.3.0 base

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7608 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Add javadoc for possible metaBits reservation in order to stress allocation methods. 
Previously a method had been added to ensure reservation but because this had to be integrated with the commit protocol it felt like the wrong approach and was not committed once the original problem that prompted its development was solved.

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7613 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Add service specific methods for doPipelineRemove/doWithdrawVote.. etc to allow the forcing of zookeeper state changes by "controlling" services, where previously global zookeeper state changes were only triggered by the effected service.  This has been introduced specifically to support the forced removal of a service as a result of a sudden death or wire pull.

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7614 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Add sure kill tests, but with explicit invocation of forceRemoveService until we finalise the correct error propagation

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7615 8f7bc0f5-282e-0410-95e3-8d296e9bb460
forceRemoveServce tests

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7616 8f7bc0f5-282e-0410-95e3-8d296e9bb460
add variant to use forceRemoveService

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7617 8f7bc0f5-282e-0410-95e3-8d296e9bb460
sync to martyn.

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7618 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Modified forceRemoveService() to use the runActorTask() pattern for better code compatibility and forward support for timeouts on actor actors. See #779.

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7621 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Implement ForceRemoveService task

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7622 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Added unit tests to the HA CI test suite for forceRemoveService().  See #779.

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7624 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Removed the no longer used memberRemoveInterruptably() method.  This method used to be used in AbstractQuorum.terminate().  It was probably removed from use when we did the ZK disconnect refactor.


git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7625 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Socket tests to experiment/test socket connection errors

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7626 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Sync to martyn on low-level socket behavior test suite.

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7627 8f7bc0f5-282e-0410-95e3-8d296e9bb460
sync to martyn

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7628 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Sync to Martyn on #779.

I have added a listener protocol for the incremental write replication and sufficient hooks that we can force the failure of the write replication protocol by intercepting it at some desired point between the receive and the replicate of bytes in a payload.

I added a sure kill for A+B+C where B is killed during a large load when it has receive two chunks of data for the large load.  This test passes.

I did a similar test where C is killed.  This test fails - it gets to the end of the test and then discovers that the LOAD failed.



git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7630 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Modified forceRemoveService() to use the following order:

- serviceLeave
- withdrawVote
- pipelineRemove
- memberRemove

The is the reverse of the order in which the service establishes itself.  It is important to do the service leave first in case the remote service was not truely dead.

@see #724.

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7631 8f7bc0f5-282e-0410-95e3-8d296e9bb460
javadoc - linking the changes to ticket #724.

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7632 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Modified HAReceiveService to always drain down the payload for the current message if there is a problem with re-replicating the payload to the downstream service.

Added code to expose the HAGlue implementation object - this is the object that gets exported.  This makes it possible to reach into the HAGlueTestImpl object in some unit tests.

See #724

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7634 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Added unit tests:

  - Add test where we kill B after the KB create and then do the large load.

  - Add test where we kill C after the KB create and then do the large load.

See #724.

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7635 8f7bc0f5-282e-0410-95e3-8d296e9bb460
Sync to Martyn and CI on #724 (write replication pipeline resynchronization).

We have incorporated logic to drain to the marker in the replication protocol.

The marker concept has been refactored. There is now an IHASendState that captures original and potentially routing information for the payload.  This has been raised into the HAPipelineGlue interface.  

We still need to bring in the typed exception handling for forceRemoveService() invocations and examine QuorumPipelineImpl for possible lock contention issues around this resync protocol.

Removed dead test suite from build.xml

git-svn-id: https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/MGC_1_3_0@7636 8f7bc0f5-282e-0410-95e3-8d296e9bb460