Set retryDelay regardless of retryable status.

This allows subsequent handlers to set `retryable` to true and
use the predefined `retryDelay`.