Revert "Decrease CPU utilization on recv thread"

This reverts commit f3bbec4958ad0bc2da76bfb81856623f4732dcf3.
Going to save tackling this issue for v1.4.0.