Move local port from server config to per-session data.