HBASE-6765 'Take a snapshot' interface

Add interfaces taking a snapshot. This is in hopes of cutting down on the overhead involved in reviewing snapshots.
HBASE-6571 Generic multi-thread/cross-process error handling framework

The idea for a generic inter-process error-handling framework came
from working on HBASE-6055 (snapshots). Distributed snapshots require tight
time constraints in taking a snapshot to minimize offline time in face of
errors. However, we often need to coordinate errors between processes and
the current Abortable framework is not sufficiently flexible to handle the
multitude of situations that can occur when coordinating between all region
servers, the master and zookeeper. Using this framework error handling for
snapshots was a simple matter, amounting to maybe 200 LOC.

This seems to be a generally useful framework and can be used to easily add
inter-process error handling in HBase. The most obvious immediate usage is
as part of HBASE-5487 when coordinating multiple sub-tasks.
HBASE-6353: Snapshots shell (Matteo Bertozzi)
HBASE-7107: Snapshot References Utils (FileSystem Visitor) (Matteo Bertozzi)
HBASE-6765 'Take a snapshot' interface - ADDENDUM: Fix snasphot -> snapshot (Matteo Bertozzi)
HBASE-6836 Offline snapshots
HBASE-6865: Snapshot File Cleaners
HBASE-6777: Snapshot Restore Interface (Matteo Bertozzi)
HBASE-6230: Restore Snapshots for HBase 0.96 (Matteo Bertozzi)
HBASE-6802: Export Snapshot (Matteo Bertozzi)
HBASE-7353 [shell] have list and list_snapshot return jruby string arrays
HBASE-7339 Splitting an HFileLink causes region servers to go down.

This is a nasty HACK to allow references to read through HFileLinks.
HBASE-7206 ForeignException framework v2 (simplifies and replaces HBASE-6571)

ExternalException replaces previous ErrorHandling framework
- Barrier protos  (likely remove)
- Error Handling protos
- Prepping ExternalException for review

Remove Barrier protos - ExternalException
Removing Barrier proto - TestExernalException
Minor fixes to ExternalExceptions
Renamed StackTraceElement to StackElementTraceMessage
Folded ExternalExceptionSerializerUtil into ExternalException
ExternalException cleanup
- add serialize method (prevent exposure of ExternalExceptionMessge)
- change some publics to package exposure
Made ExternalException stack traces closer to normal stack trace convention
fixed  a minor import

[err] Renamed ExternalExceptions to ForeignExceptions
[err] changed helper.
[err] docs nits
[err] updated protobuf
[err] removed weak refernce junk.
[err] Added peering notes

[exn] Jesse review fixes.
[err] Remove TimoeutExceptioinMessage from protos
Update comments about foreign exceptions.
[err] remove last reference to TimeoutForeignException.
HBASE-7208 Transition Offline Snapshots to ForeignExceptions and Refactor for merge

DisabledTableSnapshot and Master refactors
  Updates to TestSnapshotManager
  SnapshotManager Updates for Offline Snapshot
  Base class for Master side Snapshot Request Handlers
  Fixes for TestSnapshotFromMaster
  Interfaces that IMO serve little purpose
  HBaseSnaphot Exceptions mods (do we need these?)
  Convert tasks to Runnables
  Snapshot Tasks
  Get rid of SnaphostExceptionSnare
  Refernce task and test
  Add touch to FSUtils
  Snapshot log utils (duplicate?)
  More Utils - snapshot reference, Take snapshot utils
  Touch in TestFSUtils
  Reference updates (duplicate?)
  new FS Filters and utils like touch.
  Fixes for breakup portion

Removing duplicated  code.
Remove old errorhandling framework
Removed Aborting (really this is just wrapping a boolean)
Merged Finish/Finishable with Stopped.
Removed Finish/Finishable.
Remove TestSnapshotUtils dupe
- Reference used to have content but now is just the file name.
Remove unused changes in Reference
Remove SnaphsotLogUtils (code is in TakeSnapshotUtils)
Remove touch since it is identical to fs.createNewFile
Remove Procedure coordinator references

Removed ExecutorService from consturctor list of SnapshotManager (part of MasterServices)
Back to getSnapshotManagerForTesting
Moved bulk of isSnapshotCompleted into SnapshotManager
Fixing nits
Renamed TableSnapshotHandler to TakeSnapshotHandler
Changed SnapshotTask to Callable<Void> from Runnable
Fixed TestSnapshotFromMaster archiver test
Cleanups of TableInfoCopyTask
Cleanup nits in exceptions
Minor nit fixes to TestsSnapshotFromMaster, TestSnapshotManager and TestCopyRecoveredEditsTask
Remove duplicate TestWALReferenceTask
Master side TS snapshot fixes from Matteo's review.
used SnapshotManager's isSnapshotCompleted method instead of having one in HMasteR
Move Snapshot FS code (into SnapshotManager).
- This really makes the exception handling story much simpler.
Move constant from SnapshotDescriptionUtil to SnapshotManager.

Cleanup refactor nits
Rely on Callable exception throwing in ReferenceServerWALsTask
Remove HBaseSnapshotException.getExceptionAsSnapshotException
Cleanup ExternalExcpetion refactor

Comments and nits to TakeSnapshotHandler refactor
Removed Stoppable interface from SnapshotSentinel.
Remove cancel from DisabledTableSnapshotHandler
Rename snapshot in TakeSnapshotHandler to snapshotRegions
Moved completeSnapshot from SnapshotManager to TakeSnapshotHandler
Removed test that broke because of completeSnapshot move. (was a new test anyway)
Merged stop and with cancel and added Throwable argument.

Fix Spacing nits
Cleaned up TODOs in TakeSnapshotHandler
Re-added restore handler cleanup

HBASE-7208 review cleanups from ted's review of v1.
[off] remove SnapshotManager from SnapshotManager
[offline] Remove snapshot manager form take snapshot handler
[offline] remove monitor argument in TakeSnapshotHandler.
[offline] Fix npe problem with take snapshothandler

[offline] added inheritDoc for a jesse's nit
Renamed timer timeoutinjector in DisabledTablesnapshotHandler
[offline] External -> Foreign
[offline] external -> Foreign: tasks.
[offline] external -> foreign - restoreSnapshotHelper

[offline] matteo review fixes.
[offline] ted hbase-7208 review part 3 fixes.
[offline] more cleanup from reviews.
- rename getSnapshots->getCompletedSnapshots
- updated some out of date comments.
- made isSnapshotDone more straight line.
loadTableDescriptor -> privatize make follow on.

removed bad too-implementation-specific part of a test.
improved SnapshotManager#isRestoringTable

[offline] add logging to disabled
[offline] updated verifier to handle splits in meta.

[offline] Improving error message in take snapshot handler

fix test wal ref task nit
HBASE-7212 Globally Barriered Procedure Mechanism

Removing Barrier proto - Procudure.
Fixed race condition in TestZKProcedure
Updates to procedure to take advantage of better method names.
Tighten up ProcedureMember's use of ExternalException
Update Procedure to not use ExternalExceptionMessage and to use updated ExternalException

Renamed ProcedureBuilder to ProcedureFactory Fixed spacing nits in Procedure code.
Cleanup for procedurebuilder->factory change.

[proc] remove releaseBarrier from Subprocedure.
Fix spacing nits
Repackage procedure to o.a.h.h.procedure.
Remove latches and cleanup method only used by testing from Procedure
Fixed TestProcedure positive cases so that they are not flakey
Renamed barrier* call back to barrier*ByMember
Cleaning up Procedure:
 Rename duringBarrierLatch -> acquiredBarrierLatch
 Simplify error handling.
Renamed Procedure getErrorCheckable to getErrorMonitor.
Remove wrap method from Procedure
Got rid of Procedure.getCompletedLatch
Removed silly helper methods with receiveError calls
Renamed sendSatisfiedBarrier -> sendReachedBarrier
Removed some methods that exposed latches and created new higher level methods.

Added isRemote to ExternalException.
Procedure is now clean.
[err] Procedure is now clean
tightened ProcedureCoordinator
Comment fix in ExternalException to remove a circular doc dependency on Procedure
ProcedureCoordinatorComms now clean.
Got rid of silly threads counting methods in ProcedureMember
Made ProcedureMember.getComms() package viz.
Cleanup ProcedureMember
- reorganized file for better readability.
- merged runSubprocedure into submitSubprocedure
Moved ExternalException serialization into ZKProcedureMemberComms
Trying to decide if I should cull this controllerConnection failure code.  (I think the answer is, if in doubt, remove).
Git rid of error helpers.
Remove Procedure import from ExternalExceptionDispatcher.
Going to tackel subprocedure local exception
Cleaned up TestProcedureMemeber and Subprocedure.cancel.
Make Subprocedure abstract
Cleaned up ProcedureMemberComms and Subprocedure
Cleaned up ZKProcedureUtil.
Cleanup of the ZK parts of procedure
Fix some nits in tests.
used import static for mockito in TestProcedure
Added TODO about unused ZKProcedureCoordintorComms#sendAbortToMemebers
Cleaned up TestProcedureMemeber.

hbase-7212 updates from ted's review.
After ted review, procedure clean.
Stack + Ted's review updates of Procedure
Added Comms->Rpcs RPC files.
I'm prove variable name for procedureDescription -> procedureType in ZKProcedureCoordinatorComms
Remove nodeName/coordName from ProcedureCoordinator.
handling controllerConnectionFailure wonkyness
Cleaned up nomenclature in ProcedureCoordinatorRpcs and ProcedureFactory
More nomenclature cleanup in procedure coordintator
cleanup procedure member
cleaned up nomencalture of subprocedure
Rename ZKProcedureCoordinatorComms -> Rpc fixed naming in zkprocedurecoordinatorrpcs
Fixed naming in ZKProcedureMemberRpcs
add files for zk rpcs rename
Fix spacing nits.

Updated procedure to use WeakValueMapping
Used WeakValueMapping in ProcedureMember
Added tests for Weak value mapping.

[proc] External -> Foreign

[proc] inline Subprocedure#setupTimer
[proc] ted review coments v3
[proc] most of jesse v3 review updates.

[proc] Add error handling to ZKProcedureMemberRpcs
[proc] better error message in proc

[proc] remove completed subprocedures from the member's proc list. (allow 'same' proc to run again, after previous completes)
HBASE-6864 Online snapshots scaffolding [rs]

Distributed Snapshot RegionServer Infrastructure

Changed to make TIMESTAMP fail
Get rid of o.a.h.h.regionserver.snapshot.operation package.
Cleanup RegionServerSnapshotManager constructors
[rs] first cut cleanup of of region server side of online snapshot.
[online] Remove snapshot subprocedure
[rs] Nits in SnapshotSubprocedurePool

[rs] const renames.

HBASE-6864 Online Snapshots scaffolding [hm]

Timestamp Snapshots [master side]

Master side online table snapshot plumbing
  Wire in and test online snapshots
  Wire in Enabled TableSnapshots
  Remove TestSnapshotFromServer -- requires injection framework
  Re add procedure stuff to SnapshotManager
  FS protos

Remove cancel from EnabledTableSnapshotHandler

[master] Repackage Procedure
[online] renamed Procedure getErrorCheckable -> getErrorMonitor

[hm] rename Comms -> RPC
[hm] remove node name from ProcedureCoordinator
[hm] comms -> rpc

[hm] Spacing fixes in EnabledTableSnapshotHandler
[hm] remove manager from inherititing constructor of TakeSnapshotHandler
[hm] spacing nits
[hm] handle changed constroctur (remove dispatcher argument)
[hm] removed unused newOnlineTableSnapshotHandler

[hm] external -> foreign

[online] review fixes matteo's from offline refactor review
[hm] ted hbase-7208 part 3 review fixes.
Hooray, Snapshot manager initialization gets even cleaner!
[hm] Cleaned up a comment
remove server from imports of SnapshotManager

[hm] add warning about race and verifier.
replace Hbasefsck.debuglsr with logFSTree.

[online] change default to LOGROLL snapshot (arg!)
[online] moved .tableinfo copy to master
this final broke some mockito tests.
[online] add more error handling messages
HBASE-7321 Simple flush online snapshot

[flush] Add simple flush snapshot.
[flush] bump upto 3 node minicluster
HBASE-7291 Log roll snapshot [rs]

- Logrollsnapshot uses external->foreign

logroll move due ot o.a.h.h.ss.operation rm
Add LOGROLL style snapshots to protobuf
Adding log roll snapshot test.
[lr] Remove unused field in SnapshotSubprocedure.
- noticed that SnapshotSubprocedure not used by TimestampSnapshotSubprocedure.
[lr] Update to remove dependency on SnapshotSubprocedure
Parallelize the logrollsnapshotssubprocedure.
End-to-end test showing that log roll works (and it is online)
logroll fixes

Fixed test broken due to new default snapshot type.

[lr] grr.. bad comments in log roll.
[lr] move .tableinfo copy to master
[lr] log roll log rolling timing problem fixed, tets pass.
Tweak to force -- had truncated hlogs on clone/restore.
[lr] imports nits
[lr] add debugging to logroll tracker.
[lr] fix cleaners on logroll tests.
[lr] add debugging to logroll snapshot
HBASE-6866 Timestamp snapshots [regionserver side]

Timestamp Snapshot on the region
  RegionServer side manager and plumbing
  SS Desc utils nits.
  Flush region ts snapshot regionserver side.
  Online SnapshotSubprocedure
  Fixes for making rs side compile

[rs] Remove releaseBarrier subproc no-op
[rs] repackage procedure
[rs] rename externalErrorMonitor -> monitor
[ts] renamed externalErrorMonitor -> monitor

[rs] procedure is now clean.
[rs] nit fix for online exception handling.
[rs] change from localOperationException -> cancel.
[rs] changed constructor args of Subprocedure

[rs] fix spacing nits.
[rs] Comms->Rpcs
[rs] clean up procedure member
[rs] Fixed naming.

[rs] Folded CheckForSwappedStores into FlushRegionAtTimestampTask

[rs] External -> foreign

Pullout rs infrastructure

[ts] get rid of o.a.h.h.s.operation Move Timestamp flush snapshot
[ts] Rename TimestampSnapshotOperation -> TimestampSnapshotSubprocedure
[ts] minor foling of a method in TimestampSnapshotSubprocedure
Enabled TimestanpSnapshotSubproc in RegionServerSnapshotManager
hregion Nit fix.

[ts] comment they goofiness of tableinfo copy.