  The complex structure of gas, metals, and dust in the interstellar and
circumgalactic medium (ISM and CGM, respectively) in star-forming galaxies can
be probed by Ly$\alpha$ emission and absorption, low-ionization interstellar
(LIS) metal absorption, and dust reddening $E(B-V)$. We present a statistical
analysis of the correlations among Ly$\alpha$ equivalent width
(EW$_{Ly\alpha}$), LIS equivalent width (EW$_{LIS}$), and $E(B-V)$ in a sample
of 159 star-forming galaxies at $z\sim2.3$. With measurements obtained from
individual, deep rest-UV spectra and spectral-energy distribution (SED)
modeling, we find that the tightest correlation exists between EW$_{LIS}$ and
$E(B-V)$, although correlations among all three parameters are statistically
significant. These results signal a direct connection between dust and
metal-enriched HI gas, and that they are likely co-spatial. By comparing our
results with the predictions of different ISM/CGM models, we favor a clumpy
ISM/CGM model where dust resides in neutral gas clumps and Ly$\alpha$ photons
escape through the low HI covering fraction/column density intra-clump medium.
Finally, we investigate the factors that potentially contribute to the
intrinsic scatter in the correlations studied in this work, including
metallicity, outflow kinematics, Ly$\alpha$ production efficiency, and slit
loss. Specifically, we find evidence that scatter in the relationship between
EW$_{Ly\alpha}$ and $E(B-V)$ reflects the variation in metal-to-HI covering
fraction ratio as a function of metallicity, and the effects of outflows on the
porosity of the ISM/CGM. Future simulations incorporating star-formation
feedback and the radiative transfer of Ly$\alpha$ photons will provide key
constraints on the spatial distributions of neutral gas and dust in the ISM/CGM
structure.
