Yes, it is. Just like humans, cats do not appreciate being scared or threatened for their safety. It activates an immediate adrenaline rush, stress, and anxiety for no real reason. They could also injure themselves in the process. Repeatedly scaring them could also affect their behavior around you.