From what I've seen with my crickets, the female's ovipositor (black tube) is always visible, although it can vary in size depending on the type of cricket you use and probably age as well. If a cricket doesn't have one then it's most likely male.  Male crickets just have two smallish spikes at the end of their abdomen.