Revert "omap4-common: fix wake on unplug"

This reverts commit 02f0058ca60bfc425b3325d600b6f9c7e76cff3d.