Revert "ASoC: msm: Fix issues in proxy port driver due to early afe port enable.

This reverts commit 52775d8.

Change-Id: Iedb4108aab1c854c846c8a4d687509676eb0dfd1