Phosphate itself is not harmful to fish, but it will significantly boost your algae load. Most of the time, algae is an aesthetic nuisance, but at high levels, it can choke your corals and suck all the oxygen out of your water. Algae uses both phosphate and nitrate as a food source, so be sure to check your levels regularly and keep up with your regular maintenance to limit your supply on other remedies.