Juniper/contrail-controller — pull request #634 by bailkeri (2014-04-20 11:09:11+00:00)
Title: zk fixes

Sanity Result: http://10.204.216.50/Docs/logs/0000_2014_04_20_12_27_13/test_report.html

1. Track the zookeeper state
2. Exit if the connection is LOST and
3. Pause the master if connection moves to SUSPENDED. (Since master could have switched)
A. Changed schema and svc mon to wait in main loop.
B. Discovery change is pending. To be handled in
C. API server doesn’t accept any request (PUT, POST and DELETE) if not connected to Zookeeper. Only accept GET request. In this case we don’t query zookeeper.
4. Connect to all zookeeper server instead of connecting to local zookeeper server.(Pass list of server while creating KazooClient)
5. Change the retry logic with KazooRetry. Kazoo client has logic to reconnect automatically till SessionExpires(LOST). Instead of reconnecting from our application logic, use KazooRetry to retry zookeeper operation.