Revert "ARM: 7080/1: l2x0: make sure I&D are not locked down on init"

This reverts commit 3ff1dc9cf568e8bd1e3fc123bff94b4e640e5429.