Yes, a water filtration system can effectively remove the minerals and contaminants that may be causing the sweetness in your water. However, it is essential to choose the right type of filtration system and maintain it regularly.