Safely access the connection's socket