Sexual orientation and sexuality are about more than just physical attraction. As the American Psychological Association states, "Sexual orientation refers to an enduring pattern of emotional, romantic, and/or sexual attractions to men, women, or both sexes. Sexual orientation also refers to a person’s sense of identity based on those attractions, related behaviors, and membership in a community of others who share those attractions." That means gender is included. People are not only sexually attracted to the physical, but the mental/sociological. Since you are so caught up on physical, let's me be clear (if I wasn't already) that "gender" can encompass the physical as well  how men and women dress, act, etc. That is part of what sexually attracts people.