Client exponential backoff

Added exponential backoff for master client.

Based off my testing, I found that the current 1 second sleep was taking 30% of my CPU time.  After switching to exponential backoff, the retry logic was no longer the top consumer (or even noticeable).

I made this change since it was effecting performance.  The current logic for worker doesn't have the same issue in my tests (prob cause the process doesnt get under load?), so rather not touch that code with this patch.
The original was 1 second, yes.

Felt that it was too long so capped at 500ms but can make this 1 second.

The third retry will be at around 400ms, so only the last retry is effected.
On Sep 1, 2014 8:48 PM, "Aaron Davidson" <notifications@github.com> wrote:

> In core/src/main/java/tachyon/master/MasterClient.java:
>
> >      Exception lastException = null;
> > -    while (tries ++ < MAX_CONNECT_TRY && !mIsShutdown) {
> > +    RetryPolicy retry = new ExponentialBackoffRetry(50, 500, MAX_CONNECT_TRY);
>
> Isn't this a max of 500ms? I thought the original was a flat 1 second.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/amplab/tachyon/pull/401/files#r16969349>.
>