## 2 Background
### Nelder-Mead Algorithm  
The Nelder-Mead algorithm [64], also known as the simplex method, is a popular optimization technique introduced by [PERSON] and [PERSON] in 1965. It is a direct search method that does not require derivative information and is capable of handling both smooth and non-smooth objective functions. The algorithm begins with an initial simplex, which is a geometric shape consisting of \(n+1\) vertices in an \(n\)-dimensional space. Each vertex represents a potential solution to the optimization problem. The Nelder-Mead algorithm iteratively modifies and explores the simplex to search for the optimal solution. At each iteration, the algorithm evaluates the objective function at each vertex of the simplex, identifying the best (lowest) function value (viz. \(f(\mathbf{x}_{1})\)), the worst (highest) function value (viz. \(f(\mathbf{x}_{n+1})\)), and the second-worst function value (viz. \(f(\mathbf{x}_{n})\))) among the vertices. Based on these evaluations, the algorithm performs various operations to update the simplex:  
1. **Reflection**: The worst vertex is reflected through the centroid of the remaining \(n\) vertices. If the reflected vertex yields a better function value than the second-worst vertex but worse than the best vertex, it replaces the worst vertex. The reflection operation helps the algorithm explore the search space in the direction of the reflected vertex. The reflection phase can be summarized as follows: 1. Compute the reflection vertex \(\mathbf{x}_{r}\) using Equation (5), where \(\overline{\mathbf{x}}\) is the centroid of the \(n\) first best points (i.e., \(\overline{\mathbf{x}}=\frac{1}{n}\sum_{i=1}^{n}\mathbf{x}_{i}\)) and \(\rho\) is the coefficient of reflection. \[\mathbf{x}_{r}=(1+\rho)\overline{\mathbf{x}}+\rho\mathbf{x}_{n+1}\] (5) 2. If \(f(\mathbf{x}_{1})\leq f(\mathbf{x}_{r})<f(\mathbf{x}_{n})\), accept the reflected point \(\mathbf{x}_{r}\) and terminate the iteration. 2. **Expansion:** If the reflected vertex has a better function value than the best vertex, the algorithm performs an expansion operation. It calculates a new point by extrapolating beyond the reflected vertex and evaluates the function at this new point. If the new point is better than the reflected vertex, it replaces the worst vertex. The expansion operation allows the simplex to grow in the direction of the reflected vertex, potentially discovering better solutions. The expansion phase can be recapitulated as follows: 1. If \(f(\mathbf{x}_{r})<f(\mathbf{x}_{1})\), compute the expansion vertex \(\mathbf{x}_{e}\) using Equation (6), where \(\chi\) is the coefficient of expansion. \[\mathbf{x}_{e}=(1+\rho\chi)\overline{\mathbf{x}}-\rho\chi\mathbf{x}_{n+1}\] (6) 2. If \(f(\mathbf{x}_{e})<f(\mathbf{x}_{r})\), accept \(\mathbf{x}_{e}\) and terminate the iteration; otherwise, accept \(\mathbf{x}_{r}\) and terminate the iteration. 3. **Contraction:** If the reflected vertex does not improve the function value compared to the second-worst vertex, the algorithm performs a contraction operation. It calculates a new point by contracting toward the best vertex from the reflected vertex and evaluates the function at this new point. If the new point yields a better function value than the reflected vertex, it replaces the worst vertex. The contraction operation helps the algorithm converge toward the best vertex. The contraction phase can be described as follows:  
1. [label=()]
2. If \(f(\mathbf{x}_{n})\leq f(\mathbf{x}_{r})<f(\mathbf{x}_{n+1})\), perform an outside contraction as follows: 1. Compute the outside contraction vertex \(\mathbf{x}_{oc}\) using Equation (7), where \(\gamma\) is the coefficient of contraction. \[\mathbf{x}_{oc}=(1+\rho\gamma)\overline{\mathbf{x}}-\rho\gamma\mathbf{x}_{n+1}\] (7) ii. If \(f(\mathbf{x}_{oc})<f(\mathbf{x}_{r})\), accept \(\mathbf{x}_{oc}\) and terminate the iteration; otherwise, go to step 4. 2. If \(f(\mathbf{x}_{r})\geq f(\mathbf{x}_{n+1})\), perform an inside contraction as follows: 1.