You can, but the pillow is more likely to build up mildew and bacteria this way, because moisture may become trapped between the pillow and the tub.