Yes. Cape Town is one of the few places in Africa where you can drink tap water. However, South Africans regularly encounter water shortages, so you have to be responsible and not waste water. Come with a water bottle, or buy one as soon as you arrive.