Mineral spirits usually don’t affect the polyurethane, but it might soften your brush, and some of its material may be wiped off due to a stronger mineral spirit effect.