In case of fragments, don't consider ports. Bug #847.