A focus control apparatus comprising: detection means configured to detect a specific object image to be focused from a sensed image; setting means configured to set an AF evaluation value detection area in the sensed image; and focus detection means configured to acquire an AF evaluation value for adjusting a focus state of a photographing optical system based on a signal output from the AF evaluation value detection area, wherein the setting means is configured to set a first AF evaluation value detection area in response to the information of the specific object image detected by the detection means and to update said first AF evaluation value detection area by tracking the detected specific object image, and to set and update a second AF evaluation value detection area so that said second AF evaluation value detection area contains and is larger than the first AF evaluation value detection area, wherein the focus detection means is configured to carry out computation using signal outputs in the first AF evaluation value detection area and the second AF evaluation value detection area to acquire the AF evaluation value for adjusting the focus state of the photographing optical system, and wherein the focus detection means is configured to generate the AF evaluation value by adding an AF evaluation value generated based on the signal output in the first AF evaluation value detection area and an AF evaluation value generated based on the signal output in the second AF evaluation value detection area at a predetermined ratio in a case where the specific object image is detected by the detection means, and to generate the AF evaluation value based on the signal output in the second AF evaluation value detection area without using the signal output in the first AF evaluation value detection area in a case where the specific object image is not detected by the detection means. detection means configured to detect a specific object image to be focused from a sensed image; setting means configured to set an AF evaluation value detection area in the sensed image; and focus detection means configured to acquire an AF evaluation value for adjusting a focus state of a photographing optical system based on a signal output from the AF evaluation value detection area, wherein the setting means is configured to set a first AF evaluation value detection area in response to the information of the specific object image detected by the detection means and to update said first AF evaluation value detection area by tracking the detected specific object image, and to set and update a second AF evaluation value detection area so that said second AF evaluation value detection area contains and is larger than the first AF evaluation value detection area, wherein the focus detection means is configured to carry out computation using signal outputs in the first AF evaluation value detection area and the second AF evaluation value detection area to acquire the AF evaluation value for adjusting the focus state of the photographing optical system, and wherein the focus detection means is configured to generate the AF evaluation value by adding an AF evaluation value generated based on the signal output in the first AF evaluation value detection area and an AF evaluation value generated based on the signal output in the second AF evaluation value detection area at a predetermined ratio in a case where the specific object image is detected by the detection means, and to generate the AF evaluation value based on the signal output in the second AF evaluation value detection area without using the signal output in the first AF evaluation value detection area in a case where the specific object image is not detected by the detection means.