Loupe doesn't always dismiss

To reproduce on the view controller branch:

- Press and hold on the color picker view
- With another finger tap a button to push another view controller to the stack
- Let go of where the loupe was being held down
- Tap the back button and notice the loupe is still open