As long as you pursue your desire to crossdress in private, there are few drawbacks. However, if you wish to appear in public, be aware that most people do not accept it and that it could affect you adversely. When I am dressed as a female I fantasize about dating a man. No not necessarily. Men can enjoy role-playing when dressed, feel an attraction to another man or engage in sex with another man without being gay.
Most people are somewhat bi-sexual, as are our closest animal cousins, the apes.