Some holders or dispensers are open on one side, so you can just slip the toilet paper roll on and off. These should be mounted on the wall with the open side facing away from the toilet.
This is because when you’re on the toilet, you typically pull the toilet paper towards yourself to tear a piece off. If the open-ended side were facing the toilet, the toilet paper roll would come off the holder when pulling it.