While many bath pillows come with removable mesh covers that can be machine-washed, you can also scrub your pillow with a baking soda or diluted vinegar mixture to remove any bacteria that may cause mold or mildew. It's always important to let your pillow air dry after cleaning so that no excess moisture is trapped in the pillow.