Surface inspection apparatus, non-transitory computer readable medium storing program, and surface inspection method

A surface inspection apparatus includes an imaging device configured to image a surface of an object to be inspected, and a processor configured to: calculate a numerical value representing a quality of the surface by processing an image captured by the imaging device, and notify a user of information indicating a relationship between a first orientation of a pattern on the surface detected from the image and a second orientation that gives a direction of imaging in which a sensitivity of detection by the imaging device is high.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2021-095768 filed Jun. 8, 2021.

BACKGROUND

(i) Technical Field

The present invention relates to a surface inspection apparatus, a non-transitory computer readable medium storing a program, and a surface inspection method.

(ii) Related Art

Today, in various products, parts made by molding synthetic resin (hereinafter referred to as “molded products”) are used. On the other hand, visually observable defects may appear on the surface of the molded product. This type of defect includes a “sink mark” that is an unintentionally formed dent, a “weld” that is formed at a portion where the molten resin joins, and the like.

SUMMARY

An apparatus that inspects the quality of the surface of an object (hereinafter also referred to as a “surface inspection apparatus”) primarily images a light component that is specularly reflected on the surface of the object to be inspected, and inspects the presence or absence of defects by analyzing the captured image.

By the way, defects such as sink marks basically have a linear pattern. In the case of a linear pattern, the distribution of reflected light components is anisotropic. For example, the intensity of the light component that is specularly reflected in the direction orthogonal to the pattern is strong, and the intensity of the light component that is specularly reflected in the direction parallel to the pattern is weak. Therefore, it is required that the surface inspection apparatus correctly positions the orientation of the pattern formed on the surface to be inspected.

An example of related art includes JP2018-66712A.

Aspects of non-limiting embodiments of the present disclosure relate to a surface inspection apparatus and a non-transitory computer readable medium storing a program that make it possible to improve the accuracy of inspection as compared with the case of inspecting a pattern having anisotropy without considering an orientation of imaging of a surface inspection apparatus having anisotropy in detection sensitivity.

According to an aspect of the present disclosure, there is provided a surface inspection apparatus including an imaging device configured to image a surface of an object to be inspected, and a processor configured to: calculate a numerical value representing a quality of the surface by processing an image captured by the imaging device, and notify a user of information indicating a relationship between a first orientation of a pattern on the surface detected from the image and a second orientation that gives a direction of imaging in which a sensitivity of detection by the imaging device is high.

DETAILED DESCRIPTION

First Exemplary Embodiment

Usage Example of Surface Inspection Apparatus

FIG.1is a diagram illustrating a usage example of a surface inspection apparatus1assumed in a first exemplary embodiment.

An imaging unit of the surface inspection apparatus1used in the first exemplary embodiment is a so-called area camera, and a range to be imaged (hereinafter referred to as an “imaging range”) is defined by a surface. Illuminations (not shown) are configured to include components that are specular reflection conditions over the entire imaging range.

In the case ofFIG.1, the imaging range images only a part of an object to be inspected (hereinafter also referred to as an “inspection target”)10of interest. A molded product is assumed as the inspection target10in the present exemplary embodiment.

In the case of the inspection by the area camera, the inspection by the surface inspection apparatus1and the inspection target10is performed in a stationary state. In other words, the inspection of the surface of the inspection target10is performed in a state where the surface inspection apparatus1and the inspection target10do not move relatively.

In the case ofFIG.1, the inspection target10has a plate shape, but the inspection target10may have any shape. For example, the inspection target10may have a shape having a curved surface such as a sphere or a cylinder, in addition to a polyhedron, for example.

The actual inspection target10may have holes, notches, protrusions, steps, and the like.

The types of surface finishes of the inspection target10include no processing, mirror finish processing, semi-mirror finish processing, and texturing processing.

The surface inspection apparatus1inspects defects on the surface and textures of the inspection target10.

Defects include, for example, sink marks and welds. The sink mark refers to a dent on the surface generated in the thick portion or the rib portion, and the weld refers to a streak generated in the portion where the tips of the molten resin join in the mold. The defects also include scratches and dents caused by hitting an object. Sink marks and welds are examples of one-dimensional patterns.

The texture is a visual or tactile impression, and is influenced by the color, luster, and unevenness of the surface of the object. The unevenness of the surface also includes streaks generated in cutting the mold. This type of streak is different from a defect.

FIGS.2A and2Bare diagrams illustrating an example of defects appearing on the surface of the inspection target10.FIG.2Ashows an example of sink marks, andFIG.2Bshows an example of a weld. InFIGS.2A and2B, the defective portion is surrounded by a broken line. There are four sink marks inFIG.2A.

The surface inspection apparatus1according to the present exemplary embodiment is used not only for inspection of defects and texture, but also for inspection of surface stains.

The surface inspection apparatus1quantifies a result of evaluating defects on the surface and the texture of the inspection target10to output the result.

The defects herein are unevenness and streaks appearing in the portion that should be flat, that is, sink marks and welds. The texture is evaluated by a numerical value (hereinafter also referred to as a “score”). The score is an example of a numerical value representing the quality of the surface of the inspection target10.

For example, multivariate analysis is used to calculate the score. In multivariate analysis, for example, features appearing in the luminance distribution are analyzed. An example of a feature includes a streaky pattern extending along a direction of the sink mark, for example.

In addition, there is also a method of using artificial intelligence to calculate the score. For example, the score of a partial region within the inspection range is calculated by giving the image captured by the camera to a learning model obtained by deep machine learning of the relationship between the image of the defect and the score.

The inspection target10shown inFIG.1is installed parallel to the planes defined by an X axis and a Y axis. In this case, the normal of the surface of the inspection target10is parallel to a Z axis.

On the other hand, the surface inspection apparatus1is arranged vertically above the inspection target10. In other words, an optical axis of an optical system used by the surface inspection apparatus1for imaging the inspection target10is set substantially parallel to the normal of the surface of the inspection target10. Hereinafter, the conditions required for this optical axis are also referred to as “imaging conditions”.

In this case, the surface inspection apparatus1is installed at a position that satisfies the imaging conditions. The surface inspection apparatus1may be installed by fixing the surface inspection apparatus to a specific member, or may be detachably attached to the specific member.

However, the surface inspection apparatus1may be a portable apparatus. In a case where the surface inspection apparatus is portable, an operator inspects any surface by, for example, holding the surface inspection apparatus1in his/her hand and directing the light receiving surface toward the inspection target10.

InFIG.1, for the purpose of describing the positional relationship between the surface inspection apparatus1and the inspection target10, the appearance of the surface inspection apparatus1is simplified and represented as a substantially rectangular parallelepiped.

Configuration of Surface Inspection Apparatus

FIG.3is a diagram illustrating an example of a hardware configuration of the surface inspection apparatus1used in the first exemplary embodiment.

The surface inspection apparatus1shown inFIG.3includes a processor101that controls the operation of the entire apparatus, a read only memory (ROM)102in which a basic input output system (BIOS) and the like are stored, a random access memory (RAM)103used as a work area of the processor101, an auxiliary storage device104in which programs and image data are stored, a display105that displays a captured image of the surface of the inspection target10or information on operations, an operation reception device106that receives operations of an operator, a camera107that images the surface of the inspection target10, a light source108that illuminates the surface of the inspection target10, and a communication interface (IF)109used for communication with the outside. The processor101and each part are connected to each other through a signal line110such as a bus.

The processor101, the ROM102, and the RAM103function as so-called computers.

The processor101realizes various functions through the execution of a program. For example, the processor101performs the calculation or the like of the score for evaluating the texture of the imaged surface of the inspection target10through the execution of the program.

Image data obtained by imaging the surface of the inspection target10is stored in the auxiliary storage device104. For the auxiliary storage device104, for example, a semiconductor memory or a hard disk device is used. Firmware and application programs are also stored in the auxiliary storage device104. In the following, firmware and application programs are collectively referred to as a “program”.

The program that realizes the functions described in the present exemplary embodiment and other exemplary embodiments which will be described later can be provided not only by a communication unit but also by storing the program in a recording medium such as a CD-ROM.

The display105is, for example, a liquid crystal display or an organic EL display, and displays an image of the entire inspection target10or a specific portion of the inspection target10. The display105is also used for positioning the imaging range with respect to the inspection target10.

In the case of the present exemplary embodiment, the display105is integrally provided in the main body of the surface inspection apparatus, but may be an external device connected through the communication IF109or a part of another device connected through the communication IF109. For example, the display105may be a display of another computer connected through the communication IF109.

The operation reception device106is configured with a touch sensor arranged on the display105, physical switches and buttons arranged on a housing, and the like.

In the case of the present exemplary embodiment, a power button and an imaging button are provided as an example of physical buttons. In a case where the power button is operated, for example, the light source108is turned on and the imaging by the camera107is started. Further, in a case where the imaging button is operated, a specific image captured by the camera107at the time of operation is acquired as an image for inspection.

A device that integrates the display105and the operation reception device106is called a touch panel. The touch panel is used to receive operations of a user on keys displayed in software (hereinafter also referred to as “soft keys”).

In the case of the present exemplary embodiment, a color camera is used as the camera107. For the image sensor of the camera107, for example, a charge coupled device (CCD) imaging sensor element or a complementary metal oxide semiconductor (CMOS) imaging element is used.

Since a color camera is used as the camera107, it is possible in principle to observe not only the luminance of the surface of the inspection target10but also the color tone. The camera107is an example of an imaging device.

In the case of the present exemplary embodiment, a white light source is used as the light source108. The white light source generates light in which light in a visible light band is evenly mixed.

In the case of the present exemplary embodiment, a parallel light source is used as the light source108. Further, a telecentric lens is arranged on the optical axis of the camera107.

The light source108in the present exemplary embodiment is arranged at an angle at which the light component specular-reflected on the surface of the inspection target10is mainly incident on the camera107.

The communication IF109is configured with a module conforming to a wired or wireless communication standard. For the communication IF109, for example, an Ethernet (registered trademark) module, a universal serial bus (USB), a wireless LAN, or the like is used.

Structure of Optical System

FIGS.4A and4Bare diagrams illustrating a structural example of an optical system of the surface inspection apparatus1according to the first exemplary embodiment.FIG.4Ashows schematically an internal structure of a housing100of the surface inspection apparatus1, andFIG.4Bshows a structure of an opening portion111pressed against the surface of the inspection target10at the time of inspection.

The opening portion111is provided with an opening111A into which illumination light illuminating the surface of the inspection target10and reflected light reflected by the surface of the inspection target10are input/output, and a flange111B surrounding an outer edge of the opening111A.

In the case ofFIGS.4A and4B, both the opening111A and the flange111B have a circular shape. However, the opening111A and the flange111B may have other shapes. For example, the opening111A and the flange111B may have a rectangular shape.

The opening111A and the flange111B do not have to have similar shapes, the opening111A may have a circular shape, and the flange111B may have a rectangular shape.

The flange111B is used for positioning the surface inspection apparatus1in an imaging direction with respect to the surface of the inspection target10. In other words, the flange111B is used for positioning the camera107and the light source108with respect to the surface to be inspected. The flange111B also serves to prevent or reduce the incident of external light or ambient light on the opening111A.

The housing100shown inFIG.4Ahas a structure in which two members having a substantially tubular shape are connected, and the light source108is attached to one member side, and the camera107and the processor101are attached to the other member side.

Further, the display105and the operation reception device106are attached to the side surface of the housing100on the side where the camera107is attached.

An imaging lens (not shown) is arranged on the optical axis of the camera107shown inFIG.4A.

In the case ofFIG.4A, in the flat plate-shaped inspection target10, the normal of the surface of the inspection target10is indicated by N0. Further, in FIG.4A, the optical axis of the illumination light output from the light source108is indicated by L1, and the optical axis of the reflected light specular-reflected on the surface of the inspection target10is indicated by L2. The optical axis L2herein coincides with the optical axis of the camera107.

The surface of the actual inspection target10has structural or design unevenness, curved surfaces, steps, joints, fine unevenness formed in the molding process, and the like.

Therefore, as the normal N0of the inspection target10, an average value of the normal N0of a region AR of interest in the inspection target10or the normal N0of a specific position P of interest may be used.

Further, as the normal line N0of the inspection target10, the normal line N0of the average virtual surface or the representative portion of the inspection target10may be used.

In the case ofFIG.4A, the optical axis L1of the illumination light output from the light source108and the optical axis L2of the camera107are both attached at an angle θ with respect to the normal line N0. For the angle θ, for example, approximately 30° or approximately 45° is used.

The sensitivity of detection by the surface inspection apparatus1used in the present exemplary embodiment is anisotropic.

Specifically, in a case where the surface inspection apparatus1is arranged as shown inFIG.4A, the sensitivity to detect a linear pattern extending in a Y-axis direction (hereinafter referred to as a “linear pattern”) is high, and the sensitivity to detect a linear pattern extending in an X-axis direction is low. In other words, the sensitivity to detect a linear pattern extending in a direction orthogonal to the virtual surface defined by the optical axis L1of the light source108and the optical axis L2of the camera107is high, and the sensitivity to detect a linear pattern extending in a direction parallel to the virtual surface is low.

Relationship Between Direction in which Detection Sensitivity is High and Direction of Linear Pattern

Here, the relationship between the direction in which the detection sensitivity is high and the direction of the linear pattern to be inspected will be described.

FIG.5is a diagram illustrating a direction of imaging by the surface inspection apparatus1and a direction of a sink mark formed on the inspection target10. InFIG.5, the sink mark is exaggerated for convenience of description.

In the case ofFIG.5, the direction of the sink mark is parallel to the X axis. The direction of the sink mark is an example of a first orientation.

The direction in which the sensitivity of detection by the surface inspection apparatus1is high is a direction orthogonal to a plane defined by the optical axis L1of an illumination system and the optical axis L2of an imaging system. In other words, the direction in which the sensitivity of detection by the surface inspection apparatus1is high is the direction orthogonal to the direction of imaging. The direction in which the detection sensitivity is high is an example of a second orientation.

In the case ofFIG.5, the direction in which the inspection target10is imaged in parallel with the Y axis is defined as a “direction A”.

The direction in which the inspection target10is imaged obliquely with respect to the Y axis is defined as a “direction B”. In the case ofFIG.5, the angle of the imaging direction with respect to the Y axis is approximately 30°.

The direction in which the inspection target10is imaged in parallel with the X axis is defined as a “direction C”. The direction in which the sink mark extends when viewed in the “direction C” coincides with the direction in which the sensitivity of detection by the surface inspection apparatus1is low.

FIGS.6A to6Care diagrams illustrating a relationship between the direction of imaging and a luminance profile S generated from a captured image.FIG.6Ashows the luminance profile S in a case where the image is captured in the “direction A”,FIG.6Bshows the luminance profile S in a case where the image is captured in the “direction B”, andFIG.6Cshows the luminance profile S in a case where the image is captured in the “direction C”.

In the case ofFIGS.6A to6C, from the image captured in the direction A in which the direction in which the detection sensitivity is high and the direction of the sink mark coincide, the luminance profile S having a high wave height reflecting a luminance difference generated in the sink mark portion is detected.

From the image captured in the direction B, which is oblique to the direction in which the detection sensitivity is high and the direction of the sink mark, the luminance profile S having a low wave height reflecting a luminance difference generated in the sink mark portion is detected.

In a case where the inspection target10is imaged in the direction C, the direction in which the sink mark extends is the direction in which the detection sensitivity is low. Therefore, the captured image also contains almost no difference in luminance due to the sink mark. Therefore, the luminance profile S has a substantially flat waveform.

For the above reasons, it can be seen that in order to correctly evaluate the quality of the surface of the inspection target10, for example, it is advisable to capture an image in the direction A with respect to the direction of the sink mark.

In the following, the case of calculating the score from the luminance profile S will be described, but also in the case of calculating the score by multivariate analysis of the image or in the case of calculating the score using artificial intelligence, for example, it is advisable to capture an image from a position close to the “direction A” in which the unevenness of the surface is easily imaged as a difference in luminance.

Inspection Operation

FIG.7is a flowchart illustrating an example of an inspection operation by the surface inspection apparatus1used in the first exemplary embodiment. The symbol S shown in the figure means a step.

The process shown inFIG.7is implemented through the execution of the program by the processor101(seeFIG.3).

In the surface inspection apparatus1according to the present exemplary embodiment, the light source108(seeFIGS.4A and4B) is turned on by operating the power button, and the imaging by the camera107(seeFIGS.4A and4B) is started. The captured image is displayed on the display105(seeFIGS.4A and4B).

FIG.8is a diagram illustrating an example of an operation screen120displayed on the display105. On the operation screen120shown inFIG.8, a display field of an image (hereinafter referred to as a “captured image field”)121captured by the camera107, a score field122, a legend123, and an information field124indicating a positional relationship between a linear pattern and a direction of imaging (hereinafter referred to as “imaging direction”) are arranged.

In the captured image field121, a distribution of luminance values, that is, a grayscale image is displayed. In the case ofFIG.8, a reference line121A that gives the outer edge of the inspection range used for the calculation of the score is displayed.

In the example ofFIG.8, the range surrounded by the four reference lines121A is the inspection range. For images within the inspection range, a score representing the quality of the surface is calculated.

The legend123is shown on the right side of the captured image field121. In the case ofFIG.8, the shading of the captured image field121corresponds to the gradation values “206” to “255”.

In the case of the operation screen120shown inFIG.8, since the score has not been calculated yet, the score field122is blank. Similarly, the display color of the information field124is gray. Gray is an example.

FIG.7is referred to again for description.

In the present exemplary embodiment, in a case where an operator checking the image displayed on the display105operates the imaging button, the image used for evaluating the quality of the surface is determined.

Therefore, the processor101, which has started the inspection operation by operating the power button, determines whether or not the operation of the imaging button has been received (step S1). The operation of the operation button is an example of the operation of giving an instruction to start an inspection.

While a negative result is obtained in step S1, the processor101repeats the determination in step S1.

In a case where a positive result is obtained in step S1, the processor101acquires an image to be used for inspection (step S2). Specifically, the image displayed on the display105at the time when the imaging button is operated is acquired.

In the case of the present exemplary embodiment, in a case where the imaging button is operated, the update of the image displayed in the captured image field121(seeFIG.8) is stopped even though the imaging by the camera107is continued.

Next, the processor101calculates the score using the luminance profile S within the inspection range (step S3). That is, the score is calculated for the image within the range surrounded by the four reference lines121A displayed in the captured image field121.

FIG.9is a diagram illustrating a principle of score calculation. The image shown in the captured image field121inFIG.9assumes a case where the sink mark is imaged from the “direction A”.

In this case, the luminance profile S is given as a change in a luminance value (hereinafter referred to as a “representative luminance value”) representing each coordinate in the Y-axis direction.

The representative luminance value herein is given as an integral value of the luminance values of the pixels having an identical Y coordinate. The convex waveform of the luminance profile S shows a bright region as compared with the surroundings, and the concave waveform of the luminance profile S shows a dark region as compared with the surroundings.

The score is calculated as, for example, a difference between the maximum value and the minimum value of the luminance profile S (that is, the wave height).

The score depends on the width, height, depth, number, etc. of the unevenness formed on the surface. For example, even though the height of the convex portion and the depth of the concave portion are identical, the score of the partial region where the convex portion or the concave portion having a longer width is formed becomes high.

Further, even though the widths of the convex portion and the concave portion are identical, the score of the partial region where the higher convex portion and the deeper concave portion are formed becomes high. In the case of the present exemplary embodiment, a high score means poor quality.

In the present exemplary embodiment, the partial region that contributes to the calculation of the score is defined as a space between the start point of the convex waveform and the end point of the concave waveform of the luminance profile S.

FIG.7is referred to again for description.

In a case where the score is calculated, the processor101displays the corresponding score on the operation screen120(step S4).

Next, the processor101detects the main orientation of the edge component from the features of the partial region having a high score (step S5).

In the case of the present exemplary embodiment, the processor101extracts a specific periodic component appearing in a specific direction in the partial region as an edge component. For the extraction of periodic components, for example, two-dimensional DCT (=Discrete Cosine Transform), DST (=Discrete Sine Transform), FFT (=Fast Fourier Transform), and the like are used.

Further, the processor101sets, for example, the average of the directions of the plurality of extracted edge components and the direction of the longest edge component as the main orientation of the edge components extracted from the inspection range.

Next, the processor101calculates the angle formed by the detected orientation and the orientation in which the sensitivity of detection by the camera107is high (step S6). For example, in the case ofFIG.9, the angle formed by the main direction of the edge component such as the sink mark and the X-axis direction is calculated.

After that, the processor101notifies the operator of information evaluating the current imaging direction according to the calculated size of the formed angle, and ends the process (step S7).

For example, the processor101evaluates the imaging direction in three stages. The three stages are an example, and may be two stages or four or more stages.

In the present exemplary embodiment, in a case where the formed angle is 0° or more and less than 22.5°, the processor101determines that the imaging direction is “good” and sets the display color of the information field124indicating the positional relationship to “green”.

In a case where the formed angle is 22.5° or more and less than 45°, the processor101determines that the imaging direction is “slightly good” and sets the display color of the information field124indicating the positional relationship to “yellow”.

In a case where the formed angle is 45° or more and 90° or less or the formed angle cannot be calculated, the processor101determines that the imaging direction is “re-imaging required” and sets the display color of the information field124indicating the positional relationship to “red”.

Two threshold values are used for these determinations. In the case of this example, 22.5° is used as a threshold value for distinguishing between “good” and “slightly good”. Further, 45° is used as a threshold value for distinguishing between “slightly good” and “re-imaging required”. The angles that give these threshold values are examples.

Example of Information Notification

FIGS.10A to10Care diagrams illustrating an example of notification of an evaluation result of an imaging direction using the information field124indicating a positional relationship.FIG.10Ais an example of notification in a case where the sink mark to be inspected is imaged from the “direction A”,FIG.10Bis an example of notification in a case where the sink mark to be inspected is imaged from the “direction B”, andFIG.10Cis an example of notification in a case where the sink mark to be inspected is imaged from the “direction C”.

In the case ofFIG.10Ain which the imaging direction is determined to be “good” with respect to the sink mark direction, the information field124indicating the positional relationship is displayed in green.

In the case ofFIG.10Bin which the imaging direction is determined to be “slightly good” with respect to the sink mark direction, the information field124indicating the positional relationship is displayed in yellow.

In the case ofFIG.10Cin which the imaging direction is determined to be “re-imaging required” with respect to the sink mark direction, the information field124indicating the positional relationship is displayed in red. In fact, inFIG.10C, the luminance distribution in the captured image field121is substantially uniform.

This means that the sink mark recognized by the operator is not recognized by the surface inspection apparatus1. In the case ofFIG.10C, since the formed angle cannot be calculated, the information field124indicating the positional relationship is displayed in red.

The operator who sees these displays can notice that the current imaging direction is unsuitable for imaging and evaluating defects such as sink marks of interest. Further, by changing the imaging direction and re-imaging the sink mark or the like of interest, it is possible to calculate a score with higher reliability.

Second Exemplary Embodiment

In the present exemplary embodiment, some notification methods of notifying the operator of information obtained by evaluating the size of the formed angle calculated in step S6(seeFIG.7) without using the information field124indicating the positional relationship will be described. Specifically, a notification method of the numerical value of the formed angle as it is and a notification method of the result of evaluating the formed angle by using figures, sounds, text, and the like will be described.

The appearance configuration and processing operation of the surface inspection apparatus1according to the present exemplary embodiment are basically identical to the appearance configuration and processing operation of the surface inspection apparatus1described in the first exemplary embodiment.

Example of Information Notification

Notification Example 1

FIGS.11A to11Care diagrams illustrating an example of notification of the evaluation result using an auxiliary line125indicating a direction of a detected edge.FIG.11Ais an example of notification in a case where the sink mark to be inspected is imaged from the “direction A”,FIG.11Bis an example of notification in a case where the sink mark to be inspected is imaged from the “direction B”, andFIG.11Cis an example of notification in a case where the sink mark to be inspected is imaged from the “direction C”.

In the case ofFIG.11Ain which the imaging direction is determined to be “good” with respect to the sink mark direction, the green auxiliary line125indicating the direction of the extracted main edge is combined and displayed in the captured image field121.

The auxiliary line125may be displayed on a screen prepared separately from the captured image field121. The same applies to other display colors.

Further, although the colored auxiliary line125representing the evaluation result is displayed inFIG.11A, as the auxiliary line125, a specific color determined independently of the evaluation result, for example, a white or red auxiliary line125may be used.

Further, in the case ofFIG.11Bin which the imaging direction is determined to be “slightly good” with respect to the sink mark direction, the yellow auxiliary line125indicating the direction of the extracted main edge is combined and displayed in the captured image field121.

In the case ofFIG.11Cin which the imaging direction is determined to be “re-imaging required” with respect to the sink mark direction, a caution statement126such as “Please change the orientation of imaging and re-image” is displayed in the margin of the captured image field121. By displaying the evaluation result in text, the operator can notice that the imaging direction is unsuitable for imaging defects such as sink marks of interest.

Notification Example 2

FIGS.12A to12Care diagrams illustrating an example of notification of the evaluation result using an indicator127.FIG.12Ais an example of notification in a case where the sink mark to be inspected is imaged from the “direction A”,FIG.12Bis an example of notification in a case where the sink mark to be inspected is imaged from the “direction B”, andFIG.12Cis an example of notification in a case where the sink mark to be inspected is imaged from the “direction C”.

The indicator127shown inFIGS.12A to12Cis given as a bar graph and a display range of the bar corresponds to 0° to 90°. The size of the formed angle is displayed at the position of the needle tip.

In the case ofFIGS.12A to12C, in the indicator127ranges corresponding to the three evaluations of “good”, “slightly good”, and “re-imaging required” are displayed in green, yellow, and red colors.

Also in the case ofFIG.12A, the “good” range is given at 0° or more and less than 22.5°, the “slightly good” range is given at 22.5° or more and less than 45°, and the “re-imaging required” range is given at 45° or more and 90° or less.

In the case ofFIG.12A, the formed angle is about 11°. Therefore, the needle tip points to the vicinity of the center of the green range of the indicator127.

In the case ofFIG.12B, the formed angle is about 30°. Therefore, the needle tip points to the left of the yellow range of the indicator127.

In the case ofFIG.12C, the formed angle is about 80°. Therefore, the needle tip points to the vicinity of the right end of the red range of the indicator127. In the case of the determination of “re-imaging required”, it is considered that the accuracy of detecting the main orientation of the edge is also low. Therefore, in FIG.12C, the text “estimated” is displayed on the right side of the needle tip, indicating that the position pointed by the needle tip is not always the exact angle.

In the case of the determination of “re-imaging required”, the caution statement126(seeFIGS.11A to11C) may be displayed separately from the indicator127or together with the indicator127, as in the case of Notification example 1.

Further, in the case of Notification example 2, the calculated formed angle is indicated by the position of the needle tip of the indicator, but the calculated value of the formed angle may be displayed only by text. In that case, the display of the indicator127is unnecessary.

Further, in the case of the indicator127illustrated inFIGS.12A to12C, the difference in evaluation of the formed angle is expressed by the difference in color, but a bar graph that does not include the information on the difference in evaluation may be used. For example, a simple scale may be used.

Further, in the case ofFIGS.12A to12C, the bar graph is used for displaying the indicator127, but a semicircular graph may be used.

Notification Example 3

FIGS.13A to13Care diagrams illustrating an example of notification of a recommended direction of imaging using figures, text, and the like.FIG.13Ais an example of notification in a case where the sink mark to be inspected is imaged from the “direction A”,FIG.13Bis an example of notification in a case where the sink mark to be inspected is imaged from the “direction B”, andFIG.13Cis an example of notification in a case where the sink mark to be inspected is imaged from the “direction C”.

In the case ofFIG.13A, the evaluation of the imaging direction is “good”. Therefore, in a recommended imaging direction information field128, a figure representing a circle and the sentence “good” are displayed. Further, in the case ofFIG.13A, the figure is displayed in green.

In the case ofFIG.13B, the evaluation of the imaging direction is “slightly good”. Therefore, in the recommended imaging direction information field128, an arrow-shaped figure indicating that the sink mark or the like to be evaluated should be imaged from the left side of the current position, and the sentence “slightly good” are displayed. Specifically, the arrow pointing to the upper right indicates the recommended imaging direction. Further, in the case ofFIG.13B, the figure is displayed in yellow.

In the case ofFIG.13C, the evaluation of the imaging direction is “re-imaging required”. Therefore, in the recommended imaging direction information field128, there is an arrow-shaped figure indicating that imaging should be performed to the right from a position rotated 90° clockwise from the current position, and the sentence “re-imaging required” are displayed. Further, in the case ofFIG.13C, the figure is displayed in red.

In the case of the determination of “re-imaging required”, the caution statement126(seeFIGS.11A to11C) may be displayed separately from the information field128or together with the information field128, as in the case of Notification example 2. Coloring of figures is not always necessary.

Notification Example 4

FIGS.14A to14Care diagrams illustrating an example of notification of the evaluation result by a sound effect.FIG.14Ais an example of notification in a case where the sink mark to be inspected is imaged from the “direction A”,FIG.14Bis an example of notification in a case where the sink mark to be inspected is imaged from the “direction B”, andFIG.14Cis an example of notification in a case where the sink mark to be inspected is imaged from the “direction C”.

In the case ofFIG.14A, the evaluation of the imaging direction is “good”. Therefore, the surface inspection apparatus1gives a notification by a sound effect such as “ping, pong”, for example.

In the case ofFIG.14B, the evaluation of the imaging direction is “slightly good”. Therefore, the surface inspection apparatus1gives a notification by a sound effect such as “beep beep”, for example.

In the case ofFIG.14C, the evaluation of the imaging direction is “re-imaging required”. Therefore, the surface inspection apparatus1gives a notification by a sound effect such as “boo boo”, for example.

Notification Example 5

FIGS.15A to15Care diagrams illustrating an example of notification of the evaluation result by voice.FIG.15Ais an example of notification in a case where the sink mark to be inspected is imaged from the “direction A”,FIG.15Bis an example of notification in a case where the sink mark to be inspected is imaged from the “direction B”, andFIG.15Cis an example of notification in a case where the sink mark to be inspected is imaged from the “direction C”.

In the case ofFIG.15A, the evaluation of the imaging direction is “good”. Therefore, the surface inspection apparatus1gives a notification by voice such as “The orientation of imaging is good”, for example.

In the case ofFIG.15B, the evaluation of the imaging direction is “slightly good”. Therefore, the surface inspection apparatus1gives a notification by voice such as “The orientation of imaging is tilted by 30° with respect to the best direction”.

In the case ofFIG.15C, the evaluation of the imaging direction is “re-imaging required”. Therefore, the surface inspection apparatus1gives a notification by a sound effect such as “The orientation of imaging is tilted by 45° or more with respect to the best direction”.

Notification Example 6

FIGS.16A to16Care diagrams illustrating another example of notification of the evaluation result by voice.FIG.16Ais an example of notification in a case where the sink mark to be inspected is imaged from the “direction A”,FIG.16Bis an example of notification in a case where the sink mark to be inspected is imaged from the “direction B”, andFIG.16Cis an example of notification in a case where the sink mark to be inspected is imaged from the “direction C”.

In the case ofFIG.16A, the evaluation of the imaging direction is “good”. In this case, the change of the imaging direction is unnecessary. Therefore, in the case ofFIG.16A, the voice of giving an instruction of the correction of the imaging direction is not output.

In the case ofFIG.16B, the evaluation of the imaging direction is “slightly good”. Therefore, the surface inspection apparatus1gives a notification by voice such as “Let's change the imaging position clockwise”, for example.

In the case ofFIG.16C, the evaluation of the imaging direction is “re-imaging required”. Therefore, the surface inspection apparatus1gives a notification by voice such as “The imaging direction does not seem to be correct. Please change the orientation and re-image.”, for example.

Third Exemplary Embodiment

In the case of the present exemplary embodiment, a case where a display field of an image in which the features of the partial region used to calculate the score are emphasized (hereinafter referred to as an “emphasized image field”) is displayed on the operation screen120will be described.

The appearance configuration and processing operation of the surface inspection apparatus1according to the present exemplary embodiment are identical to the appearance configuration and processing operation of the surface inspection apparatus1described in the first exemplary embodiment.

FIG.17is a flowchart illustrating an example of an inspection operation by the surface inspection apparatus1used in a third exemplary embodiment. InFIG.17, portions corresponding to the portions inFIG.7are denoted by the corresponding reference numerals.

The process shown inFIG.17is implemented through the execution of the program by the processor101(seeFIG.3).

In the surface inspection apparatus1according to the present exemplary embodiment, the light source108(seeFIGS.4A and4B) is turned on by operating the power button, and the imaging by the camera107(seeFIGS.4A and4B) is started. The captured image is displayed on the display105(seeFIGS.4A and4B).

FIG.18is a diagram illustrating an example of an operation screen120displayed on the display105. InFIG.18, portions corresponding to the portions inFIG.8are denoted by the corresponding reference numerals.

On the operation screen120shown inFIG.18, in addition to the captured image field121of the image captured by the camera107, the score field122, the legend123, and the information field124indicating the positional relationship, an emphasized image field129for displaying an image in which the features of a partial region that has contributed to the calculation of the score are emphasized is arranged.

In the case of the operation screen120shown inFIG.18, since the score has not been calculated yet, no image is displayed in the emphasized image field129.

FIG.17is referred to again for description.

In the present exemplary embodiment, in a case where an operator checking the image displayed on the display105operates the imaging button, the image used for evaluating the quality of the surface is determined.

Therefore, the processor101, which has started the inspection operation by operating the power button, determines whether or not the operation of the imaging button has been received (step S1).

While a negative result is obtained in step S1, the processor101repeats the determination in step S1.

In a case where a positive result is obtained in step S1, the processor101acquires an image to be used for inspection (step S2). Specifically, the image displayed on the display105at the time when the imaging button is operated is acquired.

In the case of the present exemplary embodiment, in a case where the imaging button is operated, the update of the image displayed in the captured image field121(seeFIG.18) is stopped even though the imaging by the camera107is continued.

Next, the processor101calculates the score using the luminance profile within the inspection range (step S3). That is, the score is calculated for the image within the range surrounded by the four reference lines121A displayed in the captured image field121.

In a case where the score is calculated, the processor101displays the corresponding score on the operation screen120(step S4).

In the case where the score is calculated, the processor101generates an image in which the features of the partial region having a high score are emphasized (hereinafter referred to as an “emphasized image”) and displays the generated image separately (step S11).

In the present exemplary embodiment, the processor101extracts a specific periodic component appearing in a specific direction from the extracted partial region, and generates an emphasized image by superimposing the feature image on the original image by the inverse transformation of the extracted periodic component.

In inverse transformation to the feature image, an intensity component (that is, a luminance value) of each pixel is normalized by the maximum value, and a gradation range of the feature image is expanded. In addition, by mapping a color component to the intensity component of the feature image, it is possible to distinguish the feature image from the original image portion expressed in gray scale.

By displaying the emphasized image, it is possible to check the surface state even in a case where it is difficult to visually recognize the minute structure in the grayscale image obtained by imaging the surface of the partial region where the score is calculated.

In the case of the present exemplary embodiment, the generated emphasized image is displayed side by side in the operation screen identical to the grayscale image captured by the camera107.

Next, the processor101detects the main orientation of the edge component from the features of the partial region having a high score (step S5).

Subsequently, the processor101calculates the angle formed by the detected orientation and the orientation in which the sensitivity of detection by the camera107is high (step S6).

After that, the processor101notifies the operator of information evaluating the current imaging direction according to the calculated size of the formed angle, and ends the process (step S7).

Display Example

FIGS.19A to19Care diagrams illustrating a display example of the operation screen120including the emphasized image field129.FIG.19Ais a display example in a case where the sink mark to be inspected is imaged from the “direction A”,FIG.19Bis a display example in a case where the sink mark to be inspected is imaged from the “direction B”, andFIG.19Cis a display example in a case where the sink mark to be inspected is imaged from the “direction C”.

InFIGS.19A to19C, portions corresponding to the portions inFIGS.10A to10Care denoted by the corresponding reference numerals.

In the case ofFIGS.19A to19C, an image in which the image in the inspection range is emphasized is displayed in the emphasized image field129. Therefore, it is possible to observe a fine state of the inspection target10which is the target of imaging, not only in the case where the imaging direction is good but also in the case where the imaging direction is not good. The emphasized image field129herein is an example of a second image.

Fourth Exemplary Embodiment

In the case of the present exemplary embodiment, the surface inspection apparatus1(seeFIG.1) that does not require the operation of the imaging button in calculating the score will be described.

The appearance configuration and the like of the surface inspection apparatus1according to the present exemplary embodiment are identical to the appearance configuration and the like of the surface inspection apparatus1described in the first exemplary embodiment.

FIG.20is a flowchart illustrating an example of an inspection operation by the surface inspection apparatus1used in a fourth exemplary embodiment. InFIG.20, portions corresponding to the portions inFIG.7are denoted by the corresponding reference numerals.

In the case ofFIG.20, in the processor101(seeFIG.3), the light source108(seeFIGS.4A and4B) is turned on by operating the power button, the imaging by the camera107(seeFIGS.4A and4B) is started, and the score calculation and the like are performed at the same time.

Therefore, in a case where the processor101acquires the image being captured by the camera107(step S21), the processor101calculates the score using the luminance profile within the inspection range (step S3).

Hereinafter, the processor101displays the corresponding score on the operation screen120(step S4).

Since the following processing operation is identical to the processing operation of the first exemplary embodiment, the description thereof will be omitted.

Fifth Exemplary Embodiment

In the present exemplary embodiment, an example in which a physical operator for changing the inspection range is arranged in the housing100(seeFIGS.4A and4B) will be described.

FIG.21is a diagram illustrating a structural example of an optical system of a surface inspection apparatus1A according to a fifth exemplary embodiment. InFIG.21, portions corresponding to the portions inFIG.1are denoted by the corresponding reference numerals.

A so-called line camera is used for an imaging unit of the surface inspection apparatus1A used in the present exemplary embodiment. Therefore, the imaging range is linear.

In the case of the present exemplary embodiment, at the time of inspection, an inspection target10is moved in the direction of the arrow while being installed on a uniaxial stage20. By moving the uniaxial stage20in one direction, the entire inspection target10is imaged.

The positional relationship between a camera107(seeFIGS.4A and4B) and a light source108(seeFIGS.4A and4B) is identical to the positional relationship between the camera107and the light source108of the first exemplary embodiment, except that the line camera is used as the camera107(seeFIGS.4A and4B). Meanwhile, in a case where a line camera is used as the camera107, it is necessary to arrange an illumination108having a specular reflection component corresponding to each angle of view.

FIG.22is a diagram illustrating optical conditions required in a case where a line camera107A is used. In the case ofFIG.22, illumination light is output in a plurality of directions from each position of an illumination108A in which light emitting portions are arranged in a line, and one of the pieces of illumination light is specularly reflected on the surface of the inspection target10and incident on the line camera107.

In the case of the area camera described above, the incident of the specular reflection component corresponding to each angle of view is ensured by devising the arrangement of the surface light source and the point light source or the like.

Other Exemplary Embodiments

(1) Although the exemplary embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the scope described in the above-described exemplary embodiments. It is clear from the description of the claims that the above-described exemplary embodiments with various modifications or improvements are also included in the technical scope of the present invention.

(2) In the above-described exemplary embodiments, a color camera is used as the camera107(seeFIGS.4A and4B), but a monochrome camera may also be used. Further, the surface of the inspection target10(seeFIG.1) may be inspected using only the green (G) component of the color camera.

(3) In the above-described exemplary embodiments, a white light source is used as the light source108(seeFIGS.4A and4B), but the illumination light may be any color.

Further, the illumination light is not limited to visible light, but may be infrared light, ultraviolet light, or the like.

(4) In the above-described exemplary embodiments, the surface inspection apparatus1(seeFIG.1) using one light source108(seeFIGS.4A and4B) has been described, but the surface of the inspection target10is illuminated by using a plurality of light sources.

For example, two light sources may be used. In that case, one light source may be arranged at an angle at which a specular-reflected light component is mainly incident on the camera107(seeFIGS.4A and4B), and the other light source may be arranged at an angle at which a diffusely reflected light component is mainly incident on the camera107. In this case, the two light sources may be arranged on both sides of the optical axis of the camera107, or may be arranged on one side with respect to the optical axis of the camera107.

(5) In the above-described exemplary embodiments, a parallel light source is used as the light source108(seeFIGS.4A and4B), but a point light source or a surface light source which is a non-parallel light source may be used. Further, a non-telecentric lens may be used on the optical axis of the camera107(seeFIGS.4A and4B). In a case where a telecentric lens or parallel light is not used, the apparatus can be downsized and the cost can be reduced as compared with the surface inspection apparatus1(seeFIG.1) described in the exemplary embodiments. In a case where a telecentric lens or parallel light is not used, the illumination is arranged so that there is a specular reflection component corresponding to each angle of view of the camera's optical system.

(6) In the above-described exemplary embodiments, the processor101(seeFIG.3) of the surface inspection apparatus1(seeFIG.1) that images the inspection target10(seeFIG.1) realizes a function of evaluating the angle formed by the direction of the one-dimensional pattern present in the inspection target10and the orientation in which the sensitivity of detection by the camera107(seeFIG.3) is high, and notifying the operator of the evaluation result. However, an equivalent function may be realized by a processor of an external computer or server that acquires image data from the surface inspection apparatus1.