Repository: scipy/scipy

Issue:
BUG: optimize: incorrect result from approx_fprime
This issue is a simple example of the problem with `approx_fprime` reported by @austinwiltshire in a comment in https://github.com/scipy/scipy/pull/10673.

Define the function `func` as

```
def func(x): 
    return -np.abs(x + 1)
```
The graph is

![Figure_1](https://user-images.githubusercontent.com/321463/86522304-ed284f80-be29-11ea-8c7d-17fac7d72ab5.png)

According to  the [docstring for `approx_fprime`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.approx_fprime.html), 

> the function gradient is determined by the forward finite difference formula:

```
         f(xk[i] + epsilon[i]) - f(xk[i])
f'[i] = ---------------------------------
                    epsilon[i]
```

If the finite differences used by `approx_fprime` are, in fact, forward finite difference as stated and shown in the formula in the docstring, `approx_fprime` shouldn't sample the function to the left of the discontinuity.  So `approx_fprime(-1, func, 1e-8)` should return -1.  In SciPy 1.5.0, we get 1:

```
In [112]: import scipy

In [113]: scipy.__version__
Out[113]: '1.5.0'

In [114]: from scipy.optimize import approx_fprime

In [115]: approx_fprime(-1, func, 1e-8)
Out[115]: array([1.])
```

With SciPy 1.4.1, we get the (approximately) correct value:

```
In [11]: import scipy                                                           

In [12]: scipy.__version__                                                      
Out[12]: '1.4.1'

In [13]: from scipy.optimize import approx_fprime                               
                                                                      
In [14]: approx_fprime(np.array([-1]), func, 1e-8)                              
Out[14]: array([-1.00000001])
```




Discussion:
Due to this [line](https://github.com/scipy/scipy/blob/master/scipy/optimize/_numdiff.py#L405). It should be as simple as removing that line, and adding some tests
Also, thank you to @austinwiltshire for picking this up.
Fix underway

Resolving patch:
diff --git a/scipy/optimize/_numdiff.py b/scipy/optimize/_numdiff.py
--- a/scipy/optimize/_numdiff.py
+++ b/scipy/optimize/_numdiff.py
@@ -403,7 +403,7 @@ def fun_wrapped(x):
         else:
             # user specifies an absolute step
             sign_x0 = (x0 >= 0).astype(float) * 2 - 1
-            h = abs_step * sign_x0
+            h = abs_step
 
             # cannot have a zero step. This might happen if x0 is very large
             # or small. In which case fall back to relative step.


Test patch:
diff --git a/scipy/optimize/tests/test__numdiff.py b/scipy/optimize/tests/test__numdiff.py
--- a/scipy/optimize/tests/test__numdiff.py
+++ b/scipy/optimize/tests/test__numdiff.py
@@ -622,3 +622,56 @@ def test_exception(self):
         assert_raises(ValueError, approx_derivative,
                       self.fun_vector_vector, x0,
                       method='2-point', bounds=(1, np.inf))
+
+
+def test_absolute_step():
+    # test for gh12487
+    # if an absolute step is specified for 2-point differences make sure that
+    # the side corresponds to the step. i.e. if step is positive then forward
+    # differences should be used, if step is negative then backwards
+    # differences should be used.
+
+    # function has double discontinuity at x = [-1, -1]
+    # first component is \/, second component is /\
+    def f(x):
+        return -np.abs(x[0] + 1) + np.abs(x[1] + 1)
+
+    # check that the forward difference is used
+    grad = approx_derivative(f, [-1, -1], method='2-point', abs_step=1e-8)
+    assert_allclose(grad, [-1.0, 1.0])
+
+    # check that the backwards difference is used
+    grad = approx_derivative(f, [-1, -1], method='2-point', abs_step=-1e-8)
+    assert_allclose(grad, [1.0, -1.0])
+
+    # check that the forwards difference is used with a step for both
+    # parameters
+    grad = approx_derivative(
+        f, [-1, -1], method='2-point', abs_step=[1e-8, 1e-8]
+    )
+    assert_allclose(grad, [-1.0, 1.0])
+
+    # check that we can mix forward/backwards steps.
+    grad = approx_derivative(
+        f, [-1, -1], method='2-point', abs_step=[1e-8, -1e-8]
+     )
+    assert_allclose(grad, [-1.0, -1.0])
+    grad = approx_derivative(
+        f, [-1, -1], method='2-point', abs_step=[-1e-8, 1e-8]
+    )
+    assert_allclose(grad, [1.0, 1.0])
+
+    # the forward step should reverse to a backwards step if it runs into a
+    # bound
+    # This is kind of tested in TestAdjustSchemeToBounds, but only for a lower level
+    # function.
+    grad = approx_derivative(
+        f, [-1, -1], method='2-point', abs_step=1e-8,
+        bounds=(-np.inf, -1)
+    )
+    assert_allclose(grad, [1.0, -1.0])
+
+    grad = approx_derivative(
+        f, [-1, -1], method='2-point', abs_step=-1e-8, bounds=(-1, np.inf)
+    )
+    assert_allclose(grad, [-1.0, 1.0])