The point is that although the groups were similar, they only saw each other as different. Another point is that race is in context. Therefore, race is in context of time and region because these differences change over time and location. This hate is returned by the Klingons. Neither group wants to be like the other, or assimilated.
Race exist by generalizations and stereotypes.
This is just a part of racism. Also, each group, the Africans and English, had harsh feelings for the other group. The Africans did not want to be assimilated. Free Africans embraced their individuality silently, and rather support their own identity than to be assimilated by the English. Race was always a problem in America.