Revert "ASoC: msm: Fix issues in proxy port driver due to early afe port enable.

This reverts commit 52775d8.

Change-Id: Iedb4108aab1c854c846c8a4d687509676eb0dfd1
Merge branch 'cm-10.1' of git://github.com/CyanogenMod/android_kernel_htc_msm8960 into cm-10.1