Sometimes people can lack normal empathy. Lack of empathy usually results from a problem in the brain. This can result from a genetic defect, physical damage due to head trauma, or a response to the environment.