As stated above, hormones, fitness level, diet, and stress all play a part in lower belly fat. Additionally, genetics and heredity are a factor—some people simply store more fat in their stomachs than others, regardless of diet and lifestyle.