BUG: optimize: fix epsfcn choice in minpack routines

An additional sqrt() was introduced in 703dc23 --- minpack takes the
sqrt internally, so it should not be done again.

The routines can still fail if the problem is badly scaled. The test is
here changed to be less badly scaled.  The correct choice of epsfcn is
problem-dependent and in general needs to be left to the user.

Fixes gh-2639