Repository: scipy/scipy

Issue:
BUG: optimize.least_squares with method='trf' yields wrong result when initial guess is close to (lower) bound
### Describe your issue.

When least_squares is called with method='trf' and the initial guess for a fit parameter is very close (<1e-10) to the lower bound, the optimization does not work and 1e-10 is returned as the "optimized" parameter. This is very likely due to the call of the function `make_strictly_feasible` in that case. In this function, the initial guess appears to be set to lower bound + 1e-10 if it is closer than 1e-10 to the lower bound.

Would it make sense to call `make_strictly_feasible` only when the initial guess is actually not feasible? This is checked a couple of lines earlier (with the function `not_in_bounds`). 

### Reproducing Code Example

```python
import numpy as np
from scipy.optimize import least_squares

def chi2(x):
    return (x-1e-12)**2

print (least_squares(chi2, x0=1.1e-12, bounds=(0, np.inf)))
```


### Error message

```shell
message: `gtol` termination condition is satisfied.
     success: True
      status: 1
         fun: [ 9.801e-21]
           x: [ 1.000e-10]
        cost: 4.802980050000001e-41
         jac: [[ 1.510e-08]]
        grad: [ 1.480e-28]
  optimality: 1.479868788609009e-38
 active_mask: [-1]
        nfev: 1
        njev: 1
```


### SciPy/NumPy/Python version and system information

```shell
1.10.0 1.24.1 sys.version_info(major=3, minor=10, micro=6, releaselevel='final', serial=0)
blas_mkl_info:
  NOT AVAILABLE
blis_info:
  NOT AVAILABLE
openblas_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/usr/local/lib']
blas_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/usr/local/lib']
lapack_mkl_info:
  NOT AVAILABLE
openblas_lapack_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/usr/local/lib']
lapack_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
    runtime_library_dirs = ['/usr/local/lib']
Supported SIMD extensions in this NumPy install:
    baseline = SSE,SSE2,SSE3
    found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2
    not found = AVX512F,AVX512CD,AVX512_KNL,AVX512_KNM,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL
```



Discussion:
Hi Adrian!

I agree that x0, not lb+1e-10, should be returned if x0 is a satisfactory solution. I tried removing that initial make_strictly_feasible (line 828 in least_squares.py) and, after that, trf that behavior (x: [ 1.1e-12 ]). As far as I can tell the initial make_strictly_feasible isn't needed because scipy already throws an error if the initial condition is out of bounds. I will make a PR with this change.

Resolving patch:
diff --git a/scipy/optimize/_lsq/least_squares.py b/scipy/optimize/_lsq/least_squares.py
--- a/scipy/optimize/_lsq/least_squares.py
+++ b/scipy/optimize/_lsq/least_squares.py
@@ -12,7 +12,7 @@
 
 from .trf import trf
 from .dogbox import dogbox
-from .common import EPS, in_bounds, make_strictly_feasible
+from .common import EPS, in_bounds
 
 
 TERMINATION_MESSAGES = {
@@ -824,9 +824,6 @@ def least_squares(
     def fun_wrapped(x):
         return np.atleast_1d(fun(x, *args, **kwargs))
 
-    if method == 'trf':
-        x0 = make_strictly_feasible(x0, lb, ub)
-
     f0 = fun_wrapped(x0)
 
     if f0.ndim != 1:


Test patch:
diff --git a/scipy/optimize/tests/test_least_squares.py b/scipy/optimize/tests/test_least_squares.py
--- a/scipy/optimize/tests/test_least_squares.py
+++ b/scipy/optimize/tests/test_least_squares.py
@@ -809,3 +809,16 @@ def err(p, x, y):
     # used a step size for FP64 when the working space was FP32.
     assert res.nfev > 2
     assert_allclose(res.x, np.array([0.4082241, 0.15530563]), atol=5e-5)
+
+
+def test_gh_18793():
+    answer = 1e-12
+    initial_guess = 1.1e-12
+
+    def chi2(x):
+        return (x-answer)**2
+
+    res = least_squares(chi2, x0=initial_guess, bounds=(0, np.inf))
+    # if we choose an initial condition that is close to the solution
+    # we shouldn't return an answer that is further away from the solution
+    assert_allclose(res.x, answer, atol=initial_guess-answer)