Well, the answer is “YES!”. It is ok to drink tap water in any city, town or village in Romania without worrying that you'll get sick.
Yet, Romanians use to drink bottled water whenever possible. It is not because the tap water wouldn’t be drinkable, but it is, we think because Romanians like to have a selection to pick up from.
Tap water in Bucharest is permanently analysed and the local water company publishes the results regularly.