  Using first-principles calculations, we show that CsBX$_3$ halides with B=Sn
or Pb undergo octahedral rotation distortions, while for B=Ge and Si, they
undergo a ferro-electric rhombohedral distortion accompanied by a rhombohedral
stretching of the lattice. We show that these are mutually exclusive at their
equilibrium volume although different distortions may occur as function of
lattice expansion. The choice between the two distortion modes is in part
governed by the Goldschmidt tolerance factor. However, another factor
explaining the difference between Sn and Pb compared with Ge and Si is the
stronger lone-pair character of Ge and Si when forced to be divalent as is the
case in these structures. The lone-pair chemistry is related to the
off-centering. While the Si-based compounds have not yet been synthesized, the
Ge compounds have been established experimentally. As a final test of the
importance of the tolerance factor we consider RbGeX$_3$, which has smaller
tolerance factor than the corresponding CsGeX$_3$ because Rb is smaller than
Cs. We find that it can lower its energy by both rotations or rhombohedral
off-centering distortions but the latter lower the energy slightly more
efficiently.
