I do not want it to be configurable.  

The step size initialization should be pretty conservative by default to avoid warmup that gets stuck in a bad neighborhood really early, and making it configurable would be too confusing to the user (sharing the same delta is bad, and having multiple deltas would be confusing).