# The Nelder-Mead Simplex Algorithm Is Sixty Years Old: New Convergence Results and Open Questions
## 6 Convergence of the Simplex Sequences
For Algorithm 2, define the set \(\widehat{\mathcal{M}}_{2}=\left\{T(\alpha)P_{j}:\alpha=\pm\frac{1}{2},\;j=1,2 \right\}\cup\mathcal{M}_{2}\) and assume that * _there exists a matrix norm_ \(\left\|\cdot\right\|_{\phi}\) _(induced by a vector norm_ \(\left\|\cdot\right\|_{\phi}\)_) such that if_ \(T_{i}P^{(i)}\in\widehat{\mathcal{M}}_{2}\)_, then_ \(\left\|\mathcal{C}\big{(}T_{i}P^{(i)}\big{)}\right\|_{\phi}<1\)_._  
If (A) holds, then \(\widehat{\mathcal{M}}_{2}\) is an RCP set and every infinite product \(\prod_{i=1}^{\infty}T_{i}P^{(i)}\) (\(T_{i}P^{(i)}\in\widehat{\mathcal{M}}_{2}\)) has the form \(we^{T}\) for some \(w\). It follows from Theorem 1 that for \(T_{i}P^{(i)}\in\mathcal{T}\setminus\widehat{\mathcal{M}}_{2}\), \(\left\|C\big{(}T_{i}P^{(i)}\big{)}\right\|_{\phi}\geq 1\). The existence of such norm \(\left\|\cdot\right\|_{\phi}\) of the form \(\left\|A\right\|_{\phi}=\left\|S^{-1}AS\right\|_{2}\) was proved in [14] for \(n=1,2,3\) and in [15] for \(n=1,2,\ldots,8\). For the cases \(n=9,10\), the corresponding matrices \(S\), found experimentally by direct search methods, are given in Appendix A. Since the eigenvalues of \(T(\alpha)P_{j}\) (\(\left|\alpha\right|<1\), \(j=1,2\)) that lay inside the unit disk converge to \(1\) for \(n\to\infty\) (see Lemma 3 of [15]), it is hard to find a norm with property (A) for greater \(n\) values. The obtained RCP sets \(\mathcal{M}_{1}\) and \(\widehat{\mathcal{M}}_{2}\) are quite narrow. However, we can keep the convergence of any infinite product from \(\mathcal{M}_{1}\) or \(\widehat{\mathcal{M}}_{2}\) by inserting an infinite number of matrices from \(\mathbb{T}\) under proper conditions (for Algorithm 2, see [14; 15]). For Algorithm 2, it was also observed [16] that several fixed-length matrix products have spectral norms less than \(1\). Upon these observations, the convergence set of both algorithms can be significantly expanded. Assume again that \(\mathbb{T}=\widetilde{\mathcal{T}}\) or \(\mathbb{T}=\mathcal{T}\). Moreover, introduce the notation  
\[\mathbb{T}^{\ell}=\Big{\{}T_{i_{1}}\cdots T_{i_{\ell}}:T_{i_{j}}\in\mathbb{T},\,j=1,2,\ldots,\ell\Big{\}} \tag{31}\]  
and consider the product \(\prod_{j=1}^{m\ell}T_{i_{j}}=\prod_{j=1}^{m}\left(\prod_{t=1}^{\ell}T_{i_{(j- 1)\ell+t}}\right)\), where  
\[\widehat{T}_{j}=\prod_{t=1}^{\ell}T_{i_{(j-1)\ell+t}}\in\mathbb{T}^{\ell} \quad(j=1,\ldots,m).\]  
Let  
\[F^{-1}T_{i_{j}}F=\left[ \begin{array}{cc}1&0\\ b\Big{(}T_{i_{j}}\Big{)}&C\Big{(}T_{i_{j}}\Big{)}\end{array} \right]=\left[ \begin{array}{cc}1&0\\ b_{i_{j}}&C_{i_{j}}\end{array} \right].\]  
Then,  
\[F^{-1}\left(\prod_{j=1}^{m\ell}T_{i_{j}}\right)F=\left[ \begin{array}{cc}1&0 \\ \sum_{j=1}^{m\ell}\left(\prod_{t=1}^{j-1}C_{i_{t}}\right)b_{i_{j}}&\prod_{j=1}^ {m\ell}C_{i_{j}}\end{array} \right] \tag{32}\]  
and  
\[F^{-1}\widehat{T}_{j}F =\left[ \begin{array}{cc}1&0\\ \sum_{r=1}^{\ell}\left(\prod_{t=1}^{r-1}C_{i_{(j-1)\ell+t}}\right)b_{i_{(j-1) \ell+r}}&\prod_{\tau=1}^{\ell}C_{i_{(j-1)\ell+\tau}}\end{array} \right]\] \[=\left[ \begin{array}{cc}1&0\\ \widehat{b}_{j}&\widehat{C}_{j}\end{array} \right]=\left[ \begin{array}{cc}1&0 \\ b\Big{(}\widehat{T}_{j}\Big{)}&C\Big{(}\widehat{T}_{j}\Big{)}\end{array} \right],\]  
where \(\left\|\widehat{b}_{j}\right\|\leq\left(\sum_{r=1}^{\ell-1}\gamma_{C}^{r-1} \right)\gamma_{b}=:\widehat{\gamma}_{\widehat{b}}\) and \(\left\|C\left(\widehat{T}_{j}\right)\right\|\leq\gamma_{C}^{\ell}=:\widehat{ \gamma}_{\widehat{C}}\) are uniformly bounded. Assume that \(k=\ell m\) and consider the expression  
\[F^{-1}\left(\prod_{j=1}^{m}\widehat{T}_{j}\right)F=\left[ \begin{array}{cc}1& 0\\ \sum_{j=1}^{m}\left(\prod_{t=1}^{j-1}\widehat{C}_{t}\right)\widehat{b}_{j}& \prod_{j=1}^{m}\widehat{C}_{j}\end{array} \right]. \tag{33}\]  
which is clearly equal to (32).