Yes, bitters typically contain alcohol. However, the amount of alcohol in a bottle of bitters varies by brand. Furthermore, because bitters are typically used in such small amounts, it's hard to estimate the amount of alcohol that they add to a drink.
Some companies are starting to roll out alcohol-free bitters, which are a good alternative if you're looking to avoid alcohol completely.