Move fd_set block in server