Remove port from vlan correctly

Again with the serendipity!

Perhaps we should be explicit about this in the code anyhow?