fix(peer_server): exit all server goroutines in Stop()

The point that I am unsure about this PR is that
Is 'stop the timer each time' a overkill?

#645