Only remember in main dungeon

Closes #232.