  CeRh$_6$Ge$_4$ is an unusual example of a stoichiometric heavy fermion
ferromagnet, which can be cleanly tuned by hydrostatic pressure to a quantum
critical point. In order to understand the origin of this anomalous behavior,
we have characterized the magnetic ordering and crystalline electric field
(CEF) scheme of this system. While magnetic Bragg peaks are not resolved in
neutron powder diffraction, coherent oscillations are observed in zero-field
$\mu$SR below $T_{\rm C}$, which are consistent with in-plane ferromagnetic
ordering consisting of reduced Ce moments. From analyzing the magnetic
susceptibility and inelastic neutron scattering, we propose a CEF-level scheme
which accounts for the easy-plane magnetocrystalline anisotropy, and suggests
that the orbital anisotropy of the ground state and low lying excited state
doublets is an important factor giving rise to the observed anisotropic
hybridization.
