Rayleigh–Ritz method
The Rayleigh–Ritz method is a direct numerical method of approximating eigenvalue, originated in the context of solving physical boundary value problems and named after Lord Rayleigh and Walther Ritz. The name Rayleigh–Ritz is being debated It is used in all applications that involve approximating eigenvalues and eigenvectors, often under different names. In quantum mechanics, where a system of particles is described using a Hamiltonian, the Ritz method uses trial wave functions to approximate the ground state eigenfunction with the lowest energy. In the finite element method context, mathematically the same algorithm is commonly called the  Ritz-Galerkin method. The Rayleigh–Ritz method or Ritz method terminology is typical in mechanical and structural engineering to approximate the eigenmodes and resonant frequencies of a structure. For matrix eigenvalue problems. In numerical linear algebra, the Rayleigh–Ritz method is commonly applied to approximate an eigenvalue problem for the matrix formula_2 of size formula_3 using a projected matrix of a smaller size formula_4, generated from a given matrix formula_5 with orthonormal columns. The matrix version of the algorithm is the most simple: If the subspace with the orthonormal basis given by the columns of the matrix formula_5 contains formula_16  vectors that are close to eigenvectors of the matrix formula_14, the Rayleigh–Ritz method above finds formula_18 Ritz vectors that well approximate these eigenvectors. The easily computable quantity formula_19 determines the accuracy of such an approximation for every Ritz pair. In the easiest case formula_20, the formula_21 matrix formula_9 turns into a unit column-vector formula_23, the formula_6 matrix formula_7 is a scalar that is equal to the Rayleigh quotient formula_26, the only formula_27 solution to the eigenvalue problem is formula_28 and formula_29, and the only one Ritz vector is formula_23 itself. Thus, the Rayleigh–Ritz method turns into computing  of the Rayleigh quotient if formula_20. Another useful connection to the Rayleigh quotient is that formula_32 for every Ritz pair formula_13, allowing to derive some properties of Ritz values formula_34 from the corresponding theory for the Rayleigh quotient. For example, if formula_14 is a Hermitian matrix, its Rayleigh quotient (and thus its every Ritz value) is real and takes values within the closed interval of the smallest and largest eigenvalues of formula_14. Example. The matrix has eigenvalues formula_38 and the corresponding eigenvectors Let us take then with eigenvalues formula_42 and the corresponding eigenvectors so that the Ritz values are formula_42  and the Ritz vectors are We observe that each one of the Ritz vectors is exactly one of the eigenvectors of formula_14 for the given formula_9 as well as the Ritz values give exactly two of the three eigenvalues of formula_14. A mathematical explanation for the exact approximation is based on the fact that the column space of the matrix formula_9 happens to be exactly the same as the subspace spanned by the two eigenvectors formula_50 and formula_51 in this example. For matrix singular value problems. Truncated singular value decomposition (SVD) in numerical linear algebra can also use the Rayleigh –Ritz method to find approximations to left and right singular vectors of the matrix formula_52 of size formula_53-by-formula_3 in given subspaces by turning the singular value problem into an eigenvalue problem. Using the normal matrix. The definition of the singular value formula_55 and the corresponding left and right singular vectors is formula_56 and formula_57. Having found one set (left of right) of approximate singular vectors and singular values by applying naively the Rayleigh–Ritz method to the Hermitian normal matrix formula_58 or formula_59, whichever one is smaller size, one could determine the other set of left of right  singular vectors simply by dividing by the singular values. However, the division is unstable or fails for small or zero singular values. An alternative approach, e.g., defining the normal matrix as formula_60 of size formula_3-by-formula_3, takes advantage of the fact that for a given formula_3-by-formula_64 matrix formula_65 with orthonormal columns the eigenvalue problem of the Rayleigh–Ritz method for the formula_64-by-formula_64 matrix can be interpreted as a singular value problem for the formula_3-by-formula_64 matrix formula_71. This interpretation allows simple simultaneous calculation of both left and right approximate singular vectors as follows. The  algorithm can be used as a post-processing step where the matrix formula_88 is an output of an eigenvalue solver, e.g., such as LOBPCG, approximating numerically selected eigenvectors of the normal matrix formula_90. Example. The matrix has its normal matrix singular values formula_93 and the corresponding thin SVD where the columns of the first multiplier from the complete set of the left singular vectors of the matrix formula_14, the diagonal entries of the middle term are the singular values, and the columns of the last multiplier transposed (although the transposition does not change it) are the corresponding right singular vectors . Let us take with the column-space that is spanned by the two exact right singular vectors corresponding to the singular values 1 and 2. Following the algorithm step 1, we compute and on step 2 its thin SVD formula_100 with Thus we already obtain the singular values 2 and 1 from formula_102 and from formula_77 the corresponding two left singular vectors formula_104 as formula_105 and formula_106, which span the column-space of the matrix formula_88, explaining why the approximations are exact for the given formula_88. Finally, step 3 computes the matrix formula_85 recovering from its rows the two right  singular vectors formula_23 as formula_112 and formula_113. We validate the first vector: formula_114 and formula_57 Thus, for the given matrix formula_88 with its column-space that is spanned by two exact right singular vectors, we determine these right singular vectors, as well as the corresponding left singular vectors and the singular values, all exactly. For an arbitrary matrix formula_88, we obtain approximate singular triplets which are optimal given formula_88 in the sense of optimality of the Rayleigh–Ritz method.