Revert "defconfig: set minimum frequency to 96MHz as default for now"

This reverts commit 3eebfd9a280e5485e4d7fc7f4a49b240d26fa1be.