server: fix timer leak

@xiangli-cmu @bmizerany
Good catch. @unihorn, can you take a look?