If your tank is healthy, your aquarium heater shouldn't really need cleaning, but if in doubt (or if you notice a buildup of algae), you can give it a wipe over – with water only – when you clean your tank.