Quality evaluation

Implementations of the present disclosure relate to methods, systems, and computer program products for quality evaluation. In one implementation, a computer-implemented method is disclosed. In the method, a pattern period may be extracted from an image of a target object, the pattern period indicating a period of a pattern that is repeated in the image. A reference image may be generated by repeating the pattern based on the extracted pattern period. Quality of the target object may be evaluated by comparing the generated reference image and the image of the target object. In other implementations, a computer-implemented system and a computer program product for quality evaluation are disclosed.

BACKGROUND

The present disclosure relates generally to product quality control. Specifically, the present disclosure relates to methods, systems, and products for evaluating quality of a target object.

Nowadays, Liquid Crystal Device (LCD) is getting more popular in almost all types of digital devices with displays. For example, mobile phones, pads, monitors, notebooks, and the like are equipped with LCD displays having various sizes and resolutions. The LCD display is manufactured from a plurality of layers such as a polarizing filter layer, a Thin Field Transistor (TFT) layer and the like. However, during the manufacturing procedure, defects may occur in each of these layers. At this point, how to evaluate whether a layer is qualified in a fast and convenient way becomes a focus.

SUMMARY

In one aspect, a computer-implemented method is disclosed. According to the method, a pattern period may be extracted from an image of a target object, where the pattern period indicates a period of a pattern that is repeated in the image. A reference image may be generated by repeating the pattern based on the extracted pattern period. Quality of the target object may be evaluated by comparing the generated reference image and the image of the target object.

In another aspect, a computer-implemented system is disclosed. The computing system comprises a computer processor coupled to a computer-readable memory unit, where the memory unit comprises instructions that when executed by the computer processor implements a method. According to the method, a pattern period may be extracted from an image of a target object, where the pattern period indicates a period of a pattern that is repeated in the image. A reference image may be generated by repeating the pattern based on the extracted pattern period. Quality of the target object may be evaluated by comparing the generated reference image and the image of the target object.

In another aspect, a computer program product is disclosed. The computer program product comprises a computer readable storage medium having program instructions embodied therewith. The program instructions are executable by an electronic device to cause the electronic device to perform actions of: extracting a pattern period from an image of a target object, the pattern period indicating a period of a pattern that is repeated in the image; generating a reference image by repeating the pattern based on the extracted pattern period; and evaluating quality of the target object by comparing the generated reference image and the image of the target object.

It is to be understood that the summary is not intended to identify key or essential features of implementations of the present disclosure, nor is it intended to be used to limit the scope of the present disclosure. Other features of the present disclosure will become easily comprehensible through the description below.

DETAILED DESCRIPTION

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

Referring now toFIG. 1, a schematic of an example of a cloud computing node is shown. Cloud computing node is only one example of a suitable cloud computing node and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the invention described herein. Regardless, cloud computing node is capable of being implemented and/or performing any of the functionality set forth hereinabove.

Workloads layer90provides examples of functionality for which the cloud computing environment may be utilized. Examples of workloads and functions which may be provided from this layer include: mapping and navigation91; software development and lifecycle management92; virtual classroom education delivery93; data analytics processing94; transaction processing95; and quality evaluation processing96. Hereinafter, reference will be made toFIGS. 4 to 11to describe details of the quality evaluation processing96.

Usually, elements in each layer for building the LCD are closely related to pixels in the LCD, and then each layer may include a great number of tiny elements. Taking a TFT object as an example of a layer for building the LCD, sizes of the elements in the TFT object result in the inability to directly detect the quality of the TFT object. Instead, a high-resolution microscope camera may be used to take photos of the TFT object, and then examinations may be made to the photos so as to evaluate the quality of the TFT.

There have been provided several approaches in the field of quality evaluation. According to one approach, those photos may be manually checked by human eyes. On one hand, this approach results in significant manpower and time overhead, and thus the number of the manufactured LCD is heavily dependent on the qualified TFT objects approved by the human check. On the other hand, the accuracy of the human eyes is not high enough to ensure that the TFT objects that have passed the human check are really qualified ones.

According to another approach, a classifier such as a deep learning network may be trained to identify the qualified/unqualified objects. However, this kind of classifier depends on the historical knowledge and should involve considerable training datasets. Accordingly, the classifier cannot handle new defect types and retraining is required based on new training datasets, such that the retrained classifier may deal with the new defect types. Further, this approach can only identify qualified and unqualified objects without the ability of identifying a location of the defect in the unqualified object.

For the sake of description, implementations of the present disclosure will be described by taking a TFT object as an example of a target object that is to be evaluated. In the context of the present disclosure, the target object may be a flat object such as any layer for building the LCD display. For example, the target object may be selected from a group including any of a polarizing filter film, a color filter film, a liquid film, a thin field transistor film, a diffuser film, a backlight film, a back substrate, and so on.

In order to at least partially solve the above and other potential problems, a new method for quality evaluation is disclosed according to implementations of the present disclosure. Hereinafter, reference will be made toFIG. 4for a general description of the present disclosure.FIG. 4depicts an example diagram400for evaluating quality of a target object according to one implementation of the present disclosure. Reference will be made to an image410which is an image that is generated from a TFT object by, for example, a high-resolution microscope camera. Like other raw materials for building the LCD display, the TFT object has periodic appearances such as repeated patterns including vertical and horizontal lines, as shown inFIG. 4. Due to natures of the pixels in the LCD display, repeated patterns appear in almost all the layers for building the LCD.

In the implementation of the present disclosure, the periodic information may be utilized for defect detection. It is to be understood that the image410is just an example image of a target object, where the portion in the dash block shows one of the repeated pattern412with a pattern period414. Although only two patterns are included in the image410, in another example, the image410may include more patterns. Once the pattern period414and the pattern412are extracted from the image410, a reference image420may be generated by repeating the pattern412based on the pattern period414.

Usually, the microscope camera for shooting the image410has a basic function of placing defects in the TFT object to a central area of the image410. At this point, if the pattern412is selected from an area that is away from the center of the image410, then it is very likely that the selected pattern412may be a clean pattern free of any defect. As the pattern412is clean and has no defect, the reference image420that is generated by repeating the clean pattern412may be a clean background for further comparison.

Next, the image410may be compared with the reference image420to see if there is any difference between the two images410and420. Based on the comparing result, the quality of the target object may be evaluated. Specifically, if there is no difference between the two images, it may indicate that the target object is qualified; otherwise, it may indicate that there might possibly be one or more defects (as illustrated in a circle430) within an area corresponding to the detected difference.

The above implementation may provide a much flexible and convenient way for quality evaluation. With this implementation, no human interaction will be involved in the procedure of the evaluating, and each of the above steps may be performed in an automatic manner. Meanwhile, the comparison between the image410and the reference image420may be based on each and every pixel, and thus the accuracy of the comparison may assure that the quality evaluation is much more reliable than a human check. Further, the present implementation does not need historical knowledge for the training. Instead, even if only one image410is provided, a pattern412may be determined from the surrounding portions of the image410. By repeating the pattern412, the clean reference image420may be generated and severed as the base for the comparison.

The above paragraphs have provided the general description of the present disclosure, further reference will be made toFIG. 5for the detailed steps of the quality evaluation.FIG. 5depicts an example flowchart of a method500for evaluating quality of a target object according to one implementation of the present disclosure. As shown inFIG. 5, a pattern period414may be extracted510from the image410of a target object, here the pattern period414may indicate a period of a pattern that is repeated in the image410. Once the pattern period414is determined, a pattern412may be selected from the surrounding portions that are away from the center of the image410so as to excluding defects into the pattern412. For example, the pattern412may start at a location near the left boundary of the image410to ensure that the pattern412is clean.

In this implementation, the pattern period414is illustrated along the horizontal direction, and the pattern412is a sub-image with a height of a full height of the image410and a width of the pattern period414. In another implementation, the pattern period414may be determined along a vertical direction, such that the pattern412is a sub-image having a width of a full width of the image410and having a height of the pattern period414.

Based on the extracted pattern period414, a reference image420may be generated520by repeating the pattern412. Continuing the above example, if the pattern period414is along the horizontal direction, then the pattern412may be also repeated along the horizontal direction. Alternatively, if the pattern period414is along the vertical direction, then the pattern412may be also repeated along the vertical direction.

Further, the quality of the target object may be evaluated530by comparing the generated reference image420and the image410of the target object. As the reference image420indicates an ideal image without a defect, the difference between the image410and the reference image420may indicate the defect(s) in the target object.

In one implementation of the present disclosure, an initial period may be obtained by a symmetric analysis to the image, and then the initial period may be refined based on a correlation analysis associated with the initial period so as to obtain the pattern period414. Reference will be made toFIGS. 6 and 7for the symmetric analysis and the correlation analysis, respectively.

FIG. 6depicts an example diagram600for extracting an initial period by a symmetric analysis to the image according to one implementation of the present disclosure. According toFIG. 6, an area610may be selected from the image410, based on which a Symmetric Average Magnitude Sum Function (SAMSF) may be performed. Here, the height of the area610may be set to a value that is less than or equals to the full height of the image410, and the width of the area610may be set to the full width of the image410. Supposing a resolution of the image410is 1024*768, the resolution of the area610may be set to, for example, 1024*200.

With the SAMSF, the image410may be processed and then a graph620may be generated. In the graph620, the x axis indicates the pixel with a maximum of 1024, and the y axis indicates a symmetric indicator determined from the SAMSF, and the amplitude of the symmetric indicator may be set according to the definition of the SAMSF. A curve622represents the symmetric indicator corresponds to the pixels in the image410. Here, the curve622is of a symmetrical shape and thus the left portion (or the right portion) of the curve622may be considered for determining the initial period.

Referring to the curve622, there is a peak624with the values of (444, 608.5) in the left portion of the curve622, where the x value of the peak624may represent a rough value of the period. At this point, the peak624may indicate that a precise value of the pattern period414may fall within a range near the value “444.” In other words, the pattern period414may be about 444 pixels. Accordingly, the initial period may be set to 444 for further processing. Although the above paragraphs have described how to determine the initial period from the peak624resulting from the SAMSF, the SAMSF is only an example method for symmetric analysis and other methods may be adopted in another implementation.

Further, reference will be made toFIG. 7for describing details of the correlation analysis.FIG. 7depicts an example diagram700for refining the initial period based on a correlation analysis associated with the initial period according to one implementation of the present disclosure. In this implementation, a range near the initial period may be set for the correlation analysis. For example, an offset such as “44” may be predefined for the initial period. In another example, another value in pixel or a certain percentage (such as 10%) may be predefined as the offset. Supposing the offset is set to 44, then the range may be defined as [400, 488].

In one implementation of the present disclosure, a current period may be selected from a range associated with the initial period, and then a first sub-image and a second sub-image may be determined by dividing the image410with the current period. Further, the current period may be identified as the pattern period414in response to a correlation occurring between the first sub-image and second sub-image.

The correlation analysis may be performed in several rounds of processing. Referring toFIG. 7, two sub-images710and720may be extracted from the area610according to a current period selected from the range [400, 488]. Supposing the width of the image410is 1024, the pixels along the width are labelled with numbers from 1 to 1024, and the current period is set to 400 in the first round, then the width of the sub-image710may be associated with the pixels from 1 to 400. Further, the sub-image720follows the sub-image710and may be extracted from the area610. At this point, the sub-image720may be associated with the pixels from 401 to 800. Based on the extracted two sub-images710and720, a correlation analysis may be performed. It is to be understood that although the sub-image710starts from the first pixel (pixel No. 1) from the left side in the image410, in another example, the sub-image720may start from another pixel such as the second pixel, the third pixel, or another pixel along the horizontal direction of the image410.

The correlation analysis may be performed according to pixel-based comparison. Continuing the above example, when the size of the area610is 1024*200, the sizes of the sub-images710and720may be defined as 400*200 in the first round. At this point, the pixels in the sub-images710and720may be compared to determine the correlation. Various methods may be adopted for determining the correlation. In one implementation, the difference between RGB values of corresponding pixels, respectively located in the sub-images710and720, may be used for determining the correlation. In this implementation, the correlation for current period may be represented as a number. The greater the correlation is, the worse the correlation is.

It is to be understood thatFIG. 7is just an example for determining the correlation for the first round where the current period is set to 400. Based on this example, the steps for the second round may be similar as those of the first round. In the second round, the current period may be set to 401 and then the sub-image710may be associated with the pixels from 1 to 401, and the sub-image720may be associated with the pixels from 402 to 802. Afterwards, the two sub-images710and720(with the sizes of 401*200) may be compared to determine the correlation for the current period “401.” Here, the current period may be gradually increased from 400 to 488, and then the correlation for all the potential current periods may be determined to find a best period that results in the best correlation between the two sub-images710and720. In this implementation, the best period that leads to the lowest correlation value may be selected as the pattern period414. For example, if the lowest correlation is obtained when the current period is set to 450, then the pattern period414may have a value of 450.

Although above paragraphs have provided descriptions as to determining the pattern period414along the horizontal direction, in another implementation, the above procedures may be performed along the vertical direction of the image410.

Having described how to determine the pattern period414in the above paragraphs, reference will be made toFIG. 8for details of determining a defect area in the image410. In one implementation of the present disclosure, the reference image420and the image410may be aligned, and then a defect area including a difference may be detected by comparing the aligned reference image420and the image410.

FIG. 8depicts an example diagram for identifying a difference between the image410and a reference image420, according to one implementation of the present disclosure. According toFIG. 8, the image410may be aligned (as illustrated by an arrow810) to the reference image420. After the alignment, the pattern in the image410may be overlapped to the pattern in the reference image420. Accordingly, the image410of the to-be-evaluated object may be compared (as illustrated by an arrow812) with the clean reference image420with no defect so as to see if there is any difference in the two images410and420.

Here, a difference between each pair of corresponding pixels in the image410and the reference image420may be determined, so as to form a difference bitmap820. In the difference bitmap820, a pixel in black may indicate that there is no difference between the pair of corresponding pixels located in the image410and the reference image420. However, a pixel in white may indicate that there are differences. For example, the two white dots in the difference bitmap820may possibly be caused by defects in the target object. It is to be understood that noises (such as the white line822in the difference bitmap820) may be involved in the above procedure, at this point, digital image processing may be performed (as illustrated by an arrow814) to the difference bitmap820to remove the noise and obtain a final difference bitmap830. Then, the quality of the target object may be evaluated according to the final difference bitmap830.

In one implementation of the present disclosure, the quality may be evaluated as “qualified” in response to no defect area having been detected. Continuing the example ofFIG. 8, if the pixels in the final difference bitmap830are all in black, then it may be determined that there is no defect area. Thereby, the quality of the target object may be evaluated as “qualified.” In one implementation, the quality may be evaluated as “unqualified” in response to a defect area having been detected. Referring toFIG. 8, two white dots exist in the final reference bitmap830, and thus the areas corresponding to these white dots may be considered as the defect areas. It is to be understood thatFIG. 8is only an example illustration for storing the difference between the image410and the reference image420by the bitmaps820and830. In another implementation, other data structures may be adopted for storing the difference.

The above paragraphs have described how to classify the target object as “qualified” objects or “unqualified” objects. Sometimes, fine classifications may be provided to the “unqualified” objects. In one example, based on whether the target object may be repaired, the “unqualified” objects may be further divided into “repairable” objects and “non-repairable” objects.

In one implementation of the present disclosure, repairability of the target object may be determined based on a location of the defect area in the reference image in response to the defect area having been detected. For details, reference will be made toFIG. 9, which depicts an example diagram900for determining repairability of the target object based on a location of the defect area in the reference image420, according to one implementation of the present disclosure. According toFIG. 9, the location of the defect area914may be determined based on a bounding box of the differences912. Although the differences912inFIG. 9includes two bad points and the defect area914is identified as one area with two defects, in another example, two defect areas may be respectively determined from the two bad points.

Usually, the larger the area occupied by the defect is, the less the repairability is. Specifically, if the defect occupies more than one unit (for example, the defect crosses a boundary line of a pattern), usually the defect cannot be repaired. Accordingly, whether the defect area crosses a boundary line of a pattern may be a standard for judging whether the target object may be repaired. In one implementation of the present disclosure, an intersection relationship may be determined, where the interaction relationship indicates whether there is an intersection between the location of the defect area and at least one boundary line of a pattern in the reference image420. Further, repairability of the target object may be determined based on the intersection relationship. If there are one or more intersections, then the target object may be considered as “non-repairable,” otherwise, the target object may be a “repairable” one.

Reference will be made toFIG. 9for describing how to determine the quality based on the intersection relationship.FIG. 9depicts an example diagram900for determining repairability of the target object based on a location of the defect area in the reference image according to one implementation of the present disclosure. In one implementation of the present disclosure, with respect to a vertical direction and a horizontal direction of the image, a projection corresponding to the defect area914may be determined. If there is a sharp peak in the projection, then it may be determined that the defect area914crosses a boundary line along the direction. Otherwise, if there is no intersection along the two directions, then it may be determined that the defect area914does not cross any boundary line and thus the defect in the defect area914may be repaired.

Reference will be made toFIG. 9for details of determining the projection. As shown inFIG. 9, along the horizontal direction, a block920′ associated with the defect area914may be obtained from the reference image420. Here, the block920′ may be a vertical block corresponding to the block920that includes the defect area914and covers the full height of the reference image420. Then, a projection of the block920′ may be determined at each pixel along the horizontal direction. In other words, the block920′ may be projected to the horizontal direction and a value at a pixel in the projection may indicate accumulated values of pixels in a vertical line crossing the pixel.

The following paragraphs will describe how to determine the projection. Supposing a size of the defect area914is 30*60, then a size of the block920′ that is obtained along the horizontal direction may be 30*768. Here, the projection may be an array including 30 values, each of these values may be calculated by accumulated values of pixels in a vertical line crossing the pixel in the block920′. For the block920′, values in the projection at the pixels from No. 1 to No. 30 will be determined based on the following rules.

With respect to the first pixel (pixel No. 1) in the block920′, the values (such as the RGB values) of each pixel in a vertical line crossing the first pixel may be accumulated. With respect to the second pixel (pixel No. 2) in the block920′, the values of each pixel in a vertical line crossing the second pixel may be accumulated. Based on similar rules, the values for the other pixels may be determined. Based on these values in the projection, a curve922may be generated, where the x axis indicates the number of the pixel, and the y axis indicates the values at each pixel.

Here, the block920′ is projected to the horizontal direction, and the curve922shows a line shape near the x axis. Based on whether there is a sharp peak in the curve922, the intersection relationship may be determined. Specifically, if there is a sharp peak, it may be determined that an intersection occurs and the defect area914crosses a boundary line of a pattern. In turn, the target object may be identified as “non-repairable.” Otherwise, if no sharp peak is detected, it may indicate that the defect area914does not cross a vertical boundary line. With regard to the curve922, no sharp peak occurs.

Further, a block930′ may be determined along the vertical direction. Here, the block930′ may be a horizontal block corresponding to the block930that includes the defect area914and covers the full width of the reference image420. Then, similar as that in generating the curve922, the block930′ may be projected to the vertical direction to generate the curve932. As the curve932contains no sharp peak, it may be determined that no intersection occurs with a horizontal boundary line.

In one implementation of the present disclosure, the quality may be evaluated as “repairable” in response to no sharp peak occurring in projections along any of a vertical and a horizontal direction of the image. According to the above descriptions andFIG. 9, no intersection occurs along any of the vertical and horizontal directions, thereby the defect area914crosses no boundary line and thus the defect related to the defect area914may be repairable.

In one implementation of the present disclosure, the quality may be evaluated as “non-repairable” in response to a peak occurring in a projection along one of a vertical and a horizontal direction of the image410. Reference will be made toFIG. 10to describe another example where the defect area crosses a boundary line of a pattern. InFIG. 10, an irregular shape in the defect area1014represents a defect1012that crosses a vertical boundary line of a pattern.FIG. 10shows similar processing as that illustrated inFIG. 9, where a vertical block1020′ corresponding to a block1020may be determined, and a horizontal block1030′ corresponding to a block1030may be determined. Further, a curve1022and a curve1032may be generated based on respective projections of the vertical block1020′ and the horizontal block1030′, respectively. FromFIG. 10, as a sharp peak occurs in the curve1022and no sharp peak occurs in the curve1032, it may be determined that the defect area1014crosses a vertical boundary line and thus the target object may be identified as “non-repairable.”

In one implementation of the present disclosure, quality of a second target object may be evaluated by comparing the reference image and an image of the second target object, where models of the target object and the second target object are the same. Usually, the target objects with a same model may have the same pattern. For example, with respect to a batch of TFT objects that are manufactured by the same factory with the same model, only one reference image420may be generated for evaluating the quality of all the TFT objects in the batch. In this implementation, the procedures for extracting the pattern414and the generating the reference image420may be omitted.

It is to be understood that the above paragraphs have described example implementations where the defect area914crosses no boundary line and the defect area1014crosses one boundary line, in other implementations, the defect area may cross more boundary lines. Once the defect area crosses at least one boundary line, the target object may be evaluated as “non-repairable.”

Although the “unqualified” objects are classified into “repairable” and “non-repairable” ones in the above paragraphs, in another implementation, more accurate classifications may be provided according to a size and/or a location of the defect area. For example, the “unqualified” objects may be classified into types such as “easy to repair,” “hard to repair,” “non-repairable,” and the like.

FIG. 11depicts an example diagram1100of a system for evaluating quality of a target object according to one implementation of the present disclosure. According toFIG. 11, an extracting unit1110may be provided for extracting a pattern period from an image of a target object, the pattern period indicating a period of a pattern that is repeated in the image. A generating unit1120may be provided for generating a reference image by repeating the pattern based on the extracted pattern period. An evaluating unit1130may be provided for evaluating quality of the target object by comparing the generated reference image and the image of the target object.