An image processing apparatus comprising: focus detection means arranged to obtain a focus detection result from each of a plurality of focus detection areas, each focus detection result to be obtained corresponding to a specific time; grouping means arranged to generate at least two groups, each group including at least one focus detection area based on a comparison between a plurality of the obtained focus detection results; determination means arranged to determine whether an object within both a first group and a second group is moving, based on a focus detection result corresponding to the first group, generated based on a focus detection result corresponding to a first time, and a focus detection result corresponding to the second group, generated based on a focus detection result corresponding to a second time, the second time being later than the first time; identification means arranged to determine whether a subject corresponding to the first group is identical to a subject corresponding to the second group by comparing the focus detection result corresponding to the first group with the focus detection result corresponding to the second group; and storage means arranged to store the focus detection result corresponding to the second group by associating the focus detection result corresponding to the second group with the first group and the focus detection result corresponding to the first group in a case where the identification means identifies the subject corresponding to the first group and the subject corresponding to the second group to be the same subject, wherein the determination means is arranged to obtain a plurality of comparison results each indicating whether a change direction in a focus detection result corresponds to one of a first direction corresponding to a movement of the same subject in an approaching direction towards the image processing apparatus and a second direction corresponding to a movement of the same subject in a receding direction away from the image processing apparatus by comparing focus detection results corresponding to a plurality of groups determined to be the same subject by the identification means, and wherein in a case where a ratio of results indicating that the change direction in the focus detection result corresponds to the second direction to results indicating that the change direction in the focus detection result corresponds to the first direction is more than or equal to a predetermined ratio, the determination means is arranged to determine that the subject is moving in the second direction. focus detection means arranged to obtain a focus detection result from each of a plurality of focus detection areas, each focus detection result to be obtained corresponding to a specific time; grouping means arranged to generate at least two groups, each group including at least one focus detection area based on a comparison between a plurality of the obtained focus detection results; determination means arranged to determine whether an object within both a first group and a second group is moving, based on a focus detection result corresponding to the first group, generated based on a focus detection result corresponding to a first time, and a focus detection result corresponding to the second group, generated based on a focus detection result corresponding to a second time, the second time being later than the first time; identification means arranged to determine whether a subject corresponding to the first group is identical to a subject corresponding to the second group by comparing the focus detection result corresponding to the first group with the focus detection result corresponding to the second group; and storage means arranged to store the focus detection result corresponding to the second group by associating the focus detection result corresponding to the second group with the first group and the focus detection result corresponding to the first group in a case where the identification means identifies the subject corresponding to the first group and the subject corresponding to the second group to be the same subject, wherein the determination means is arranged to obtain a plurality of comparison results each indicating whether a change direction in a focus detection result corresponds to one of a first direction corresponding to a movement of the same subject in an approaching direction towards the image processing apparatus and a second direction corresponding to a movement of the same subject in a receding direction away from the image processing apparatus by comparing focus detection results corresponding to a plurality of groups determined to be the same subject by the identification means, and wherein in a case where a ratio of results indicating that the change direction in the focus detection result corresponds to the second direction to results indicating that the change direction in the focus detection result corresponds to the first direction is more than or equal to a predetermined ratio, the determination means is arranged to determine that the subject is moving in the second direction.