  We consider Bernoulli distributions and their regularizations, which are
measures on the $p$-adic integers $\mathbb{Z}_p$. It is well known that their
Mellin transform can be used to define $p$-adic $L$-functions. We show that for
$p>2$ one of the regularized Bernoulli distributions is particularly simple and
equal to a measure on $\mathbb{Z}_p$ that takes the values $\pm \frac{1}{2}$ on
clopen balls. We apply this to $p$-adic $L$-functions for Dirichlet characters
of $p$-power conductor and obtain Dirichlet series expansions similar to the
complex case. Such expansions were studied by D. Delbourgo, and this
contribution provides an approach via $p$-adic measures.
