Yes, a coffee filter will dissolve over time. Most paper coffee filters are made of cellulose, which is a type of plant fiber. Cellulose is relatively easy to dissolve in water and other liquids.