The function is called `dtype_limits`, so I expect the limits to be related to the data-type, not the image's content.  As such, I may simply have it `clip_negative=True` and always clip to 0.