Rebalance Cassandra and clean Zookeeper.

Added script to re-balance cassandra and cleanup zookeeper.
Ok. I did a new push that goes directly to each node instead of master. I
wonder if that does on large number of nodes. Would you mind to try it
out?

On Thu, Oct 30, 2014 at 09:33:30PM -0700, Meni Vaitsi wrote:
> > +                # If the key is already in use, we have to do nothing.
> > +                for y in $OLD_KEYS ; do
> > +                        if [ "$y" = "$key" ]; then
> > +                                echo "   $key already in use, skipping it"
> > +                                IN_USE="YES"
> > +                        fi
> > +                done
> > +                if [ "$IN_USE" = "NO" ]; then
> > +                        echo -n "   node $x gets token $key..."
> > +                        if [ "$PARALLEL" = "NO" ]; then
> > +                                if ! ssh $MASTER "$CMD -h $x move $key" > /dev/null ; then
> > +                                        echo "failed."
> > +                                        exit 1
> > +                                fi
> > +                        else
> > +                                ( ssh $MASTER "$CMD -h $x move $key" ) &
> 
> I actually tested this parallel process and it's not working. Cluster nodes are getting stuck for more time than it would take for them to move data around sequentially.
> 
> ---
> Reply to this email directly or view it on GitHub:
> https://github.com/AppScale/appscale/pull/1615/files#r19651803

-- 
Graziano Obertelli
AppScale Systems