  Using a fast and accurate neural network potential we are able to
systematically explore the energy landscape of large unit cells of bulk
magnesium oxide with the minima hopping method. The potential is trained with a
focus on the near-stoichiometric compositions, in particular on suboxides,
i.e., Mg$_x$O$_{1-x}$ with $0.50<x<0.60$. Our extensive exploration
demonstrates that for bulk stoichiometric compounds, there are several new
low-energy rocksalt-like structures in which Mg atoms are octahedrally
six--coordinated and form trigonal prismatic motifs with different stacking
sequences. Furthermore, we find a dense spectrum of novel non-stoichiometric
crystal phases of Mg$_x$O$_{1-x}$ for each composition of $x$. These structures
are mostly similar to the rock salt structure with octahedral coordination and
five--coordinated Mg atoms. Due to the removal of one oxygen atom, the energy
landscape becomes more glass-like with oxygen-vacancy type structures that all
lie very close to each other energetically. For the same number of magnesium
and oxygen atoms our oxygen-deficient structures are lower in energy if the
vacancies are aligned along lines or planes than rock salt structures with
randomly distributed oxygen vacancies. We also found the putative global minima
configurations for each composition of the non-stoichiometric suboxide
structures. These structures are predominantly composed of (111) slabs of the
rock salt structure which are terminated with Mg atoms at the top and bottom,
and are stacked in different sequences along the $z$-direction. Like other
Magn\'eli-type phases, these structures have properties that differ
considerably from their stoichiometric counterparts such as low lattice thermal
conductivity and high electrical conductivity.
