Fix Adjacency When Toggling

Integrate the fix from Patch Z. Should be merged correctly, will need
testing. Fixes issue #40