Revert "ASoC: qcom: qdsp6v2: fix crash due to asynchronous read/write"

This reverts commit 8b9f2e5ec93d0ca6ecdf69439fee478ec0ff9500.
Build v3.96 EXP2