Revert "ASoC: msm: Fix issues in proxy port driver due to early afe port enable.

This reverts commit 52775d8.

Change-Id: Iedb4108aab1c854c846c8a4d687509676eb0dfd1
Merge remote-tracking branch 'upstream/cm-10.1' into cm-10.1