Through there may be varied reasons behind this reluctance, ignorance of what feminism actually is contributes greatly. I suppose you would receive a different answer depending on who you asked. So since you asked me, let me tell you straightaway - it is not about lesbians who hate men and want to see them suffer. There are lesbians who are a part of feminism, but that perception is too narrow to be realistic. Truth be know feminism is a political movement focusing on the advocation of women's equality.
Feminism is concerned, first and foremost, with social justice for all women.
However, it was only middle-class, Caucasian oppressions which were addressed. A great number of women who happened not to be middle class or Caucasian, who did not identify themselves with the oppressions which were being discussed, were alienated from feminism. From Margin to Center which has within it an excellent analysis of this incident. It is through analyses like hooks' that we have come to understand that while social justice is needed for all women, not all women share the same oppressions. Consequently feminism is coming to mean, not only in theory but in practice as well, justice for women of all races, ethnicities, sexualities, and classes.