No, you can avoid or run away from hordes throughout the game, but they are worth a lot of XP, so they are worth taking out whenever you can once you're equipped to do so to level up as quickly as possible and eventually unlock all of the skills.