Players aren't the only ones to can access this spell. Other spellcasters could be able to utilize the spell against you.  If a player is exiled by the spell, they are then taken to a harmless demiplane and are unable to function while there. They return after 10 rounds. If they're playing an additional planar creature, they're returned to their plane of origin. It's up to the remainder of the group to disrupt the spellcaster's focus to bring them back! The banishment spell is rare since many other fourth-level spells are more suitable for the spell slot and cause real damage to a group other than just removing one person from the group for a single minute of the battle. If the group does come across it, they'll follow the same rules that monsters do.