  Geometrical frustration among interacting spins combined with strong quantum
fluctuations destabilize long-range magnetic order in favor of more exotic
states such as spin liquids. By following this guiding principle, a number of
spin liquid candidate systems were identified in quasi-two-dimensional
(quasi-2D) systems. For 3D, however, the situation is less favorable as quantum
fluctuations are reduced and competing states become more relevant. Here we
report a comprehensive study of thermodynamic, magnetic and dielectric
properties on single crystalline and pressed-powder samples of PbCuTe$_2$O$_6$,
a candidate material for a 3D frustrated quantum spin liquid featuring a
hyperkagome lattice. Whereas the low-temperature properties of the powder
samples are consistent with the recently proposed quantum spin liquid state, an
even more exotic behaviour is revealed for the single crystals. These crystals
show ferroelectric order at $T_{\text{FE}} \approx 1\,\text{K}$, accompanied by
strong lattice distortions, and a modified magnetic response -- still
consistent with a quantum spin liquid -- but with clear indications for quantum
critical behaviour.
