Only do a ports check if this is a NAT-T SA.  Otherwise other
lookups providing ports may get unexpected results.

MFC After:	2 weeks