Basically this is a consequence of electrostatic repulsion between cations - they try to stay as far apart as possible. In crystals the number of structurally distinct sites tends to be small. This is the atomic structure of olivine. Iron or magnesium atoms are yellow and silicon is purple. The structure of olivine in a polyhedral format.
Every vertex represents an oxygen atom.
Iron or magnesium atoms are at the centers of the octahedra, silicon at the centers of the tetrahedra. Blue tetrahedra sit over openings between the octahedra and correspond to the purple tetrahedra in the next layer down. Two layers are shown and part of a third so that the layering sequence becomes apparent. All the cations occur at the centers of octahedra or tetrahedra. Although the oxygens are close-packed, olivine is orthorhombic because of the way the cations fill the voids between the oxygen sheets.