Many people feared them. And began to hate them. Such emotions still exist to this day. The Japanese attack on Pearl Harbor in 1941 spawned instant prejudice against not only the citizens of Japan, but Orientals in general and Japanese-Americans in particular. Hatred born of terror caused the U.S. government to force thousands of U.S. citizens of Japanese descent into concentration camps for the duration of the war.
Many people hate homosexuals.
They fear them. Because they feel threatened by them. Because they know nothing about homosexuality aside from dirty jokes and religious indoctrination. Ignorance breeds fear which breeds hatred. Which often breeds violence.