Revert "Use MemoryScope#restrict to filter adjustments"

This reverts commit 9a13d0cf077dba57c7ce1b8624f2a68257964b23.