coreos/etcd — pull request #1962 by xiang90 (2014-12-18 20:09:29+00:00)
Title: raft: leader waits for the reply of previous message 

It is reasonable for the leader to wait for the reply before sending out the next
msgApp or msgSnap when the follower is in bad path. Or the leader will send
out useless messages if the previous message is rejected or the previous message
is a snapshot. Especially for the snapshot case, the leader will be 100% to send out
duplicate message including the snapshot, which is a huge waste.

This commit implement a timeout based wait mechanism. The timeout for normal msgApp is a
heartbeatTimeout and the timeout for snapshot is electionTimeout(snapshot is larger). We
can implement a piggyback mechanism(application notifies the msg lost) in the future
if necessary.

@bdarnell @yichengq I am adding test for this.