It is not safe to drink tap water in Egypt as it's highly chlorinated. It is advisable to drink bottled water during your entire stay. You can use tap water to brush your teeth etc. with no worries.