detect: only inspect smsg for valid tcp packets

Packets that are rejected by the stream engine are not considered
part of an established tcp session. By allowing them to inspect
an smsg, some smsgs would not be properly inspected.