Fix potential segfaults when PROXY protocol is not enabled