An image processing apparatus configured to assign a binary label representing belonging to a target region or belonging to another region to each pixel in an image, the image processing apparatus comprising: a shape setting unit configured to set a predicted shape of the target region either from registered predicted shapes or by user input; an energy function setting unit configured to: select a pixel group comprising N pixels in the image, where N is a natural number of 4 or more, which have a positional relationship representing the set predicted shape; and set an energy function comprising an N-th order term in which a variable is a label of each of the N pixels of the selected pixel group, so that a value of the N-th order term is at a minimum value when a combination of the labels assigned to the N pixels of the selected pixel group is a pattern matching the set predicted shape of the target region, and a labeling unit configured to perform the labeling by minimizing the set energy function, and the image processing apparatus being characterized in that the value of the N-th order term increases in stages from the minimum value along with an increase in a number of pixels to which a label different from the pattern is assigned, and the energy function setting unit sets the energy function so that an increment of the value of the N-th order term receives a higher weight, the larger the distance of the pixels with a label different from the pattern with respect to the outline of the predicted shape of the target region is. a shape setting unit configured to set a predicted shape of the target region either from registered predicted shapes or by user input; an energy function setting unit configured to: select a pixel group comprising N pixels in the image, where N is a natural number of 4 or more, which have a positional relationship representing the set predicted shape; and set an energy function comprising an N-th order term in which a variable is a label of each of the N pixels of the selected pixel group, so that a value of the N-th order term is at a minimum value when a combination of the labels assigned to the N pixels of the selected pixel group is a pattern matching the set predicted shape of the target region, and a labeling unit configured to perform the labeling by minimizing the set energy function, and the image processing apparatus being characterized in that the value of the N-th order term increases in stages from the minimum value along with an increase in a number of pixels to which a label different from the pattern is assigned, and the energy function setting unit sets the energy function so that an increment of the value of the N-th order term receives a higher weight, the larger the distance of the pixels with a label different from the pattern with respect to the outline of the predicted shape of the target region is. select a pixel group comprising N pixels in the image, where N is a natural number of 4 or more, which have a positional relationship representing the set predicted shape; and set an energy function comprising an N-th order term in which a variable is a label of each of the N pixels of the selected pixel group, so that a value of the N-th order term is at a minimum value when a combination of the labels assigned to the N pixels of the selected pixel group is a pattern matching the set predicted shape of the target region, and a labeling unit configured to perform the labeling by minimizing the set energy function, and the image processing apparatus being characterized in that the value of the N-th order term increases in stages from the minimum value along with an increase in a number of pixels to which a label different from the pattern is assigned, and the energy function setting unit sets the energy function so that an increment of the value of the N-th order term receives a higher weight, the larger the distance of the pixels with a label different from the pattern with respect to the outline of the predicted shape of the target region is.