Do not return a larger minimum than the natural size.

This fixes a couple of warnings from GTK+ in some cases.