It depends on what method you’re using. If you use a chemical algaecide, it would be a good idea to remove them just to be on the safe side.