  In this note, we consider the optimization problem associated with computing
the rank decomposition of a symmetric tensor. We show that, in a well-defined
sense, minima in this highly nonconvex optimization problem break the symmetry
of the target tensor -- but not too much. This phenomenon of symmetry breaking
applies to various choices of tensor norms, and makes it possible to study the
optimization landscape using a set of recently-developed symmetry-based
analytical tools. The fact that the objective function under consideration is a
multivariate polynomial allows us to apply symbolic methods from computational
algebra to obtain more refined information on the symmetry breaking phenomenon.
