It seems that Rosewood was one of those rare places in the post-bellum Deep South where blacks owned land. People there were poor but they lived in dignity and with a rich sense of community. There was a predominantly white community called Sumner nearby. Some of the blacks from Rosewood worked for some of the whites in Sumner, the women as housemaids and the men at the lumber mill. He had a black mistress, a young woman who clerked for him at the store.
Nobody has very much money here and nobody is very well-educated or very genteel.
There are not many newcomers, and outsiders are treated with suspicion, if not hostility. But in truth, the Crackers were themselves an underclass which made them all the more desperate and unreasonable and pitiable. There are no lovable white characters in this movie, and the blacks are either martyrs or survivors, but all victims. That immediately turns the entire white male population of Sumner into an angry lynch mob, searching for an unknown black. They irrationally attempt to beat information out a few Rosewood blacks they know, which starts the spiral of confrontation leading to conflagration.