An image processing device comprising: a dividing unit configured to divide a frequency image, in which the vertical axis of the image corresponds to the y coordinates of a distance image and the horizontal axis corresponds to distance values of the distance image, and the distance values present on the horizontal lines at predetermined y coordinates in the distance image are voted into the horizontal coordinates in the image, into one or more segments along a direction of the distance values; a first selecting unit configured to select, in one or more predetermined segments from among the segments obtained by division by the dividing unit, sample points from the position of each horizontal coordinate of each segment; a first estimating unit configured to estimate geometry of a road surface from a group of sample points selected by the first selecting unit; a first determining unit configured to, in order to determine whether or not a segment that includes a first estimated road surface being the road surface estimated by the first estimating unit is a start segment for performing road surface estimation, determine whether or not the first estimated road surface satisfies a success-failure determination condition, wherein the determining unit is configured to sequentially determine whether or not each of the segments obtained by division by the dividing unit satisfies a success-failure determination condition, to determine a start segment; and a second estimating unit configured to sequentially process the segments starting from the start segment determined by the determining unit, to estimate geometry of a road surface, wherein the determining unit is configured to, when it is determined that no segment satisfies the success-failure determination condition, sequentially determine whether or not a relaxed condition obtained by relaxing the success-failure determination condition is satisfied for each of the segments, to determine the start segment, wherein the success-failure determination condition comprises a number of conditions, and relaxing the success-failure determination condition comprises reducing the number of conditions to be used at the time of performing the determination. a dividing unit configured to divide a frequency image, in which the vertical axis of the image corresponds to the y coordinates of a distance image and the horizontal axis corresponds to distance values of the distance image, and the distance values present on the horizontal lines at predetermined y coordinates in the distance image are voted into the horizontal coordinates in the image, into one or more segments along a direction of the distance values; a first selecting unit configured to select, in one or more predetermined segments from among the segments obtained by division by the dividing unit, sample points from the position of each horizontal coordinate of each segment; a first estimating unit configured to estimate geometry of a road surface from a group of sample points selected by the first selecting unit; a first determining unit configured to, in order to determine whether or not a segment that includes a first estimated road surface being the road surface estimated by the first estimating unit is a start segment for performing road surface estimation, determine whether or not the first estimated road surface satisfies a success-failure determination condition, wherein the determining unit is configured to sequentially determine whether or not each of the segments obtained by division by the dividing unit satisfies a success-failure determination condition, to determine a start segment; and a second estimating unit configured to sequentially process the segments starting from the start segment determined by the determining unit, to estimate geometry of a road surface, wherein the determining unit is configured to, when it is determined that no segment satisfies the success-failure determination condition, sequentially determine whether or not a relaxed condition obtained by relaxing the success-failure determination condition is satisfied for each of the segments, to determine the start segment, wherein the success-failure determination condition comprises a number of conditions, and relaxing the success-failure determination condition comprises reducing the number of conditions to be used at the time of performing the determination.