Males can have breasts, and females can not have them. Females can have facial hair, and males can have no facial hair at all. Many Native American males don't have much or any facial hair, for instance, and Asian men often have very little. This is true with other biological differences, too, like the pitch of a person's voice. Other gender cues are completely social.
Clothing is the most prominent example.
Think of someone wearing a tuxedo, then think of someone wearing an evening gown. Most people do, because those clothes have associations with wearers of a particular sex. Another frequent type of social gender cue is the kinds of associations we make with what someone does professionally or as a hobby. If you think that a construction worker must be a male, think again. Sure they can, and they do it every day.