An endoscope system, comprising: an image signal acquisition means for acquiring an image signal by imaging a subject; and an irregularity image generation means for generating an irregularity image by extracting information of irregularities on the subject from the image signal; wherein the irregularity image generation means includes a microstructure image generation section that generates a microstructure image as the irregularity image by extracting a microstructure of a body tissue surface layer from the image signals of a plurality of colors obtained by imaging the subject irradiated by white light and blue narrow-band light; the microstructure image generation section includes: a first brightness signal generation section that generates a first brightness signal showing an average brightness of the subject based on the image signals of the plurality of colors; and a microstructure extraction section that extracts the microstructure image by extracting a pixel region which exceeds a high brightness threshold value of the first brightness signal as a microstructure, and wherein the irregularity image generation means includes a hypertrophy enhancement image generation section that generates a hypertrophy image as the irregularity image by extracting only a hypertrophy, which has a thickness from a body tissue surface layer to a medium-deep layer, including: a gain adjustment section adapted to adjust the gain of the image signals of the plurality of colours, obtained by imaging the subject irradiated by white light only, a second brightness signal generation section that generates a second brightness signal showing an average brightness of the subject after gain adjustment; and a hypertrophy extraction section that extracts a hypertrophy image by extracting a pixel region, in which the pixel values are within a medium brightness range of the second brightness signal as the hypertrophy. an image signal acquisition means for acquiring an image signal by imaging a subject; and an irregularity image generation means for generating an irregularity image by extracting information of irregularities on the subject from the image signal; wherein the irregularity image generation means includes a microstructure image generation section that generates a microstructure image as the irregularity image by extracting a microstructure of a body tissue surface layer from the image signals of a plurality of colors obtained by imaging the subject irradiated by white light and blue narrow-band light; the microstructure image generation section includes: a first brightness signal generation section that generates a first brightness signal showing an average brightness of the subject based on the image signals of the plurality of colors; and a microstructure extraction section that extracts the microstructure image by extracting a pixel region which exceeds a high brightness threshold value of the first brightness signal as a microstructure, and wherein the irregularity image generation means includes a hypertrophy enhancement image generation section that generates a hypertrophy image as the irregularity image by extracting only a hypertrophy, which has a thickness from a body tissue surface layer to a medium-deep layer, including: a gain adjustment section adapted to adjust the gain of the image signals of the plurality of colours, obtained by imaging the subject irradiated by white light only, a second brightness signal generation section that generates a second brightness signal showing an average brightness of the subject after gain adjustment; and a hypertrophy extraction section that extracts a hypertrophy image by extracting a pixel region, in which the pixel values are within a medium brightness range of the second brightness signal as the hypertrophy. a first brightness signal generation section that generates a first brightness signal showing an average brightness of the subject based on the image signals of the plurality of colors; and a microstructure extraction section that extracts the microstructure image by extracting a pixel region which exceeds a high brightness threshold value of the first brightness signal as a microstructure, and wherein the irregularity image generation means includes a hypertrophy enhancement image generation section that generates a hypertrophy image as the irregularity image by extracting only a hypertrophy, which has a thickness from a body tissue surface layer to a medium-deep layer, including: a gain adjustment section adapted to adjust the gain of the image signals of the plurality of colours, obtained by imaging the subject irradiated by white light only, a second brightness signal generation section that generates a second brightness signal showing an average brightness of the subject after gain adjustment; and a hypertrophy extraction section that extracts a hypertrophy image by extracting a pixel region, in which the pixel values are within a medium brightness range of the second brightness signal as the hypertrophy. a gain adjustment section adapted to adjust the gain of the image signals of the plurality of colours, obtained by imaging the subject irradiated by white light only, a second brightness signal generation section that generates a second brightness signal showing an average brightness of the subject after gain adjustment; and a hypertrophy extraction section that extracts a hypertrophy image by extracting a pixel region, in which the pixel values are within a medium brightness range of the second brightness signal as the hypertrophy.