# The Nelder-Mead Simplex Algorithm Is Sixty Years Old: New Convergence Results and Open Questions
## 6 Convergence of the Simplex Sequences
[PERSON]'s theorem on rank (see, e.g., [PERSON] [30]) implies that  
\[\text{rank}(S^{\infty})\geq\text{rank}\Big{(}S^{(0)}\Big{)}+\text{rank}(B^{ \infty})-(n+1).\]Since, by assumption, rank \(\left(S^{(0)}\right)=n\), we have \(n\geq\text{rank }(S^{\infty})\geq\text{rank }(B^{\infty})-1\). In Example 4 of [15], rank \(\left(S^{(0)}\right)=n\), rank \((B^{\infty})=n\) and rank \((S^{\infty})=n-1\). The following simple result characterizes the possible limits of infinite matrix products. **Lemma 12**. : _Assume that \(B_{k}=\prod_{i=0}^{k}X_{i}\to B^{\infty}\) (\(X_{i}\in\mathbb{T}\)) and \(X_{s}\) occurs infinitely often in the product \(\prod_{i=0}^{\infty}X_{i}\), then every nonzero row of \(B^{\infty}\) is a left eigenvector of \(X_{s}\) belonging to the eigenvalue \(\lambda=1\)._  
Proof. : Since \(X_{s}\) appears infinitely many times in the product \(\prod_{i=0}^{\infty}X_{i}\), there is a subsequence of \(\left\{B_{i_{j}}\right\}\) with rightmost factor \(X_{s}\), say  
\[B_{i_{1}}X_{s},B_{i_{2}}X_{s},\ldots,\]  
where the \(B_{i_{j}}\)s are products of \(X_{i}\)s. Since \(B_{i_{j}}\to B^{\infty}\), we have \(B_{i_{j}}X_{s}\to B^{\infty}X_{s}=B^{\infty}\). Define the (left) \(1\)-eigenspace of matrix \(A\) by \(E_{1}(A)=\left\{x:xA=x\right\}\). The rows of \(B^{\infty}\) belong to \(E_{1}(X_{s})\). If several \(X_{s}\), say \(X_{s_{i}}\) (\(i=1,\ldots,m\)), occur infinitely often in the product \(\prod_{i=0}^{\infty}X_{i}\), then the rows of \(B^{\infty}\) belong to \(\cap_{i=1}^{m}E_{1}(X_{s_{i}})\). Since \(e^{T}\in E_{1}\Big{(}T_{i}P^{(i)}\Big{)}\) for all \(T_{i}P^{(i)}\in\mathcal{T}\) and \(e^{T}\in E_{1}\Big{(}T^{(i)}\Big{)}\) for all \(T^{(i)}\in\widetilde{\mathcal{T}}\), we have \(e^{T}\in\cap_{i=1}^{m}E_{1}\Big{(}T_{s_{i}}P^{(s_{i})}\Big{)}\) and \(e^{T}\in\cap_{i=1}^{m}E_{1}\Big{(}T^{(s_{i})}\Big{)}\), respectively. If \(\cap_{i=1}^{m}E_{1}\Big{(}T_{s_{i}}P^{(s_{i})}\Big{)}=\big{\{}\lambda e^{T}: \lambda\in\mathbb{R}\big{\}}\) or \(\cap_{i=1}^{m}E_{1}\Big{(}T^{(s_{i})}\Big{)}=\big{\{}\lambda e^{T}:\lambda\in \mathbb{R}\big{\}}\), then \(B^{\infty}\) has the form \(we^{T}\) for some \(w\in\mathbb{R}^{n+1}\). Note that for any \(T\in\widetilde{\mathcal{T}}\) or \(T\in\mathcal{T}\), \(e^{T}T=e^{T}\). Hence, if \(B_{k}\to B^{\infty}\), then \(e^{T}B_{k}=e^{T}\to e^{T}B^{\infty}\), that is, \(e^{T}=e^{T}B^{\infty}\). If \(B^{\infty}=we^{T}\), then \(e^{T}B^{\infty}=e^{T}we^{T}=e^{T}\) implies that \(e^{T}w=1\). We recall the following definitions and results (see, e.g., [PERSON] [31; 32]). A right infinite matrix product is an expression \(A_{1}A_{2}\cdots A_{k}A_{k+1}\cdots\). A set \(\mathcal{M}\) of \(n\times n\) matrices has the right convergence property (RCP), if all possible right infinite products \(\prod_{i=1}^{\infty}A_{i}\) (\(A_{i}\in\mathcal{M}\)) converge. A set \(\mathcal{M}\) of \(n\times n\) matrices is product-bounded if there is a constant \(\beta>0\) such that \(\|A_{1}\cdots A_{k}\|\leq\beta\) for all \(k\) and all \(A_{1},\ldots,A_{k}\in\mathcal{M}\). If \(\mathcal{M}\) is an RCP set, then \(\mathcal{M}\) is also product-bounded. **Lemma 13**. : _If \(\mathcal{M}\) is an RCP set, \(A_{1},\ldots,A_{k}\in\mathcal{M}\) and \(\lambda\) is an eigenvalue of \(A_{1}A_{2}\cdots A_{k}\), then \(|\lambda|<1\) or \(\lambda=1\), and this eigenvalue is simple. Hence, each matrix of \(\mathcal{M}\) must satisfy this condition._  
The matrices \(T(2)P_{1}\) and \(T_{j}(2)\) have at least one eigenvalue greater than \(1\), and \(\|T(2)P_{1}\|>1\) and \(\left\|T_{j}(2)\right\|>1\) in any induced matrix norm. Hence, \(\Big{\{}[T(2)P_{1}]^{k}\Big{\}}\) and \(\Big{\{}T_{j}(2)^{k}\Big{\}}\) are unbounded. There are also examples when the Nelder-Mead algorithm produces unbounded simplex sequences \(S^{(k)}=S^{(0)}[T(2)P_{1}]^{k}\) or \(S^{(k)}=S^{(0)}[T(1)P_{1}]^{k}\) (see, e.g., [14; 15; 20]). It is clear that the whole matrix set \(\widetilde{\mathcal{T}}\) or \(\mathcal{T}\) is not an RCP set.