Before revision:  In this paper, we discover that the weight initialization adopted by prior works, such as Xavier or orthogonal initialization, which was originally designed for standard network training, results in very loose certified bounds at initialization thus a longer warmup schedule must be used.
After revision:  In this paper, we identify two important issues in existing methods, namely exploded bounds at initialization thus a longer warmup schedule must be used.