Make ReplyRunnable cancelable