  Non-Hermitian systems with parity-time ($\mathcal{PT}$) symmetry and
anti-$\mathcal{PT}$ symmetry give rise to exceptional points (EPs) with
important properties due to the coalescence of eigenvectors. Because of the
limitations on the Hermiticity of the Hamiltonian, these kinds of non-Hermitian
systems have recently attracted extensive attention and emerged as a new way to
engineer the response of open quantum systems. We propose an efficient and
easily computable tool, based on the Hilbert-Schmidt speed (HSS), which does
not require the diagonalization of the evolved density matrix, to characterize
non-Markovian effects in (anti-)$\mathcal{PT}\!-$symmetric systems. Our
proposed witness of non-Markovianity can also detect exactly the EPs and hence
the critical behavior of the non-Hermitian systems, especially high-dimensional
qudits. Our theoretical predictions are completely consistent with results
extracted from recent experiments studying the non-Markovianity and criticality
in (anti-)$\mathcal{PT}\!-$symmetric systems.
