To differentiate between male and female cicada killers, you need to observe them closely. While the females have an ovipositor or stinger, the males don’t have one.  Their behavior is another indicator since the males tend to be aggressive toward anyone coming near the nest. The females are docile and don’t attack unless handled poorly.