Prevent invalid set_endpoint calls