No, it won't harm your GPU at all. Your GPU only does that to reduce overall noise. You can freely do so without much worry. Either way, your GPU will ramp up if it reaches a certain temperature.