Yes, the cold water is absolutely drinkable and delicious. You don’t need to buy water while you’re here! It is safe to drink tap water in Iceland.