Revert "Revert "arm: tegra: xmm: fix for kernel panic on xmm on/off""

This reverts commit 2aa7824f6c65f06354314a2c637bb42452472155.