Revert "Defconfig: Allow Swap"

This reverts commit 8a7e1f080da197e5b3c80c91d14f5764831ad6a9.