  We consider the problem of minimizing a function over the manifold of
orthogonal matrices. The majority of algorithms for this problem compute a
direction in the tangent space, and then use a retraction to move in that
direction while staying on the manifold. Unfortunately, the numerical
computation of retractions on the orthogonal manifold always involves some
expensive linear algebra operation, such as matrix inversion or matrix
square-root. These operations quickly become expensive as the dimension of the
matrices grows. To bypass this limitation, we propose the landing algorithm
which does not involve retractions. The algorithm is not constrained to stay on
the manifold but its evolution is driven by a potential energy which
progressively attracts it towards the manifold. One iteration of the landing
algorithm only involves matrix multiplications, which makes it cheap compared
to its retraction counterparts. We provide an analysis of the convergence of
the algorithm, and demonstrate its promises on large-scale problems, where it
is faster and less prone to numerical errors than retraction-based methods.
