Revert "ARM: 7080/1: l2x0: make sure I&D are not locked down on init"

This reverts commit 0e69b54fa8b48e3cdc1a78f77beab3af763a33a1.