A tank-based electric water heater will cost you the least amount of money up-front, though yearly usage costs are slightly higher. The cost of replacing a gas water heater is also generally higher.