Fix the bug that retry doesn't work on socket errors