Fix #3103, cleanup ports fields when protocol changes