Pool umbrellas can be cleaned with a mixture of soap and water. You can use a soft brush on them to get off any stains or imperfections that may have accumulated. If you want to wash your pool umbrella, do it on a sunny day so that it can dry completely.