The reason cheese spoils faster in the fridge is because it is moved around a lot. Repeatedly bringing a wedge of cheese up to room temperature and then putting it back in the fridge can make the cheese spoil faster.