You'll need to check the end of the cricket's abdomen. Females can be distinguished from males by their ovipositor — a long tube or needle-like organ. Females use it to lay their eggs in the substrate. You can only see it with (sub)adult crickets.