Overstaying their welcome is typical behavior for red gnomes. Choose one with a few seconds left and move it to something that hasn't been built to get rid of it. All of their remaining time will be spent up, and then they'll leave.