Patent Description:
For image recognition processes, deep learning is used, for example. Deep learning is known as a methodology of machine learning using a multilayer neural network. For example, a convolutional neural network is used for a multilayer neural network. A convolutional neural network is formed by a multilayer neural network in which convolution and pooling in local regions are repeated. <NPL> describes a structure of a fully convolutional neural network in which all connected layers forming a convolutional neural network are configured as convolutional layers is also proposed.

In order to use a fully convolutional neural network in an image recognition process, it is necessary to train the fully convolutional neutral network by using training data. In the case of a fully convolutional neural network, an image segmented by colors is used as training data. Generally, an image segmented by colors is produced by a manual operation. The larger the number of items of training data, the larger the workload required for the manual operation. Meanwhile, the larger the number of items of training data used for training, the higher the precision of the image recognition process.

The invention subject to the appended claims addresses the above-described issue and an illustrative purpose thereof is to provide a technology of inhibiting a reduction in the precision of an image recognition process while at the same time inhibiting an increase in the workload required for learning. Advantageous embodiments are subject to the dependent claims and are discussed in the following.

According to the invention, it is possible to inhibit a reduction in the precision of an image recognition process while at the same time inhibiting an increase in the workload required for learning.

A summary will be given before describing the embodiment of the present disclosure in specific details. The example embodiments described herein below relate to a processing device for performing an inspection of a solar cell by performing an image recognition process on an image that captures the solar cell. An example of inspection of a solar cell is a determination on whether or not an internal crack is found in the solar cell. A solar cell in which an internal crack is produced is easily broken in response to an impact and so is defined as a defective product. In the case that the solar cell is made of a monocrystal silicon, an internal crack is produced at an angle of <NUM>°. Even though it is known that an internal crack occurs at a predefined angle, it is difficult to detect an internal crack if the internal crack overlaps an etching unevenness. For this reason, an image recognition process is used for inspection of a solar cell.

In the related art, an image capturing a solar cell is subjected to Gabor filtering to highlight the line at <NUM>° before performing deep learning. If an electrode at an angle of <NUM>° is formed on the surface of the solar cell, however, such an electrode is also highlighted by Gabor filtering, which makes it difficult to use Gabor filtering. In this situation, it is effective to use a convolutional neural network for an image that captures a solar cell. A convolutional neural network includes a convolutional layer that extracts a feature through a convolutional process using a plurality of filters, a pooling layer that obtains invariance in local data through a pooling process that aggregates responses in a certain region, and a fully connected layer that performs recognition by using a probability calculated by a Softmax function, etc. Of these, the fully bonded layer of a convolutional neural network makes it difficult to process an image or an arbitrary size. To resolve this, a fully convolutional neural network in which the fully connected layer of the convolutional neural network is replaced by a convolutional layer is used.

In a fully convolutional neural network, a downsampling process in which processes in a convolutional layer and a pooling layer are repeated is performed, and then an upsampling process in which processes in a convolutional layer and an up-pooling layer are repeated is performed. It should be noted here that the downsampling process reduces the spatial dimension of the image and the upsampling process enlarges the spatial dimension of the image. The image output from the fully convolutional neural network (hereinafter, referred to as "output image") through these processes has a dimension equivalent to that of the image that is input (hereinafter, referred to as "input image"). Further, the output image is subjected to segmentation by filling objects in the image with colors. A neural network including a fully convolutional neural network need be trained for learning before performing an image recognition process. In the case of a fully convolutional neural network, training data is used to learn a filter coefficient in the convolutional layer. As mentioned above, however, the workload required to produce training data is large. It is necessary to increase the number of items of training data to improve the precision of the image recognition process. If the number of items of training data is increased, the workload required to produce training data is further increased.

In order to inhibit an increase in the workload to produce training data, the embodiments of the invention use a convolutional neural network in which an upsampling process in a fully convolutional neural network is excluded. It should be noted here that training data that yields an output image having a <NUM>×<NUM> spatial dimension and an input image corresponding to that training data are used for learning. The input image corresponding to the training data may be referred to as "image for learning". The training data only indicates a result of segmentation (e.g., OK), an internal crack, a color unevenness, etc., and positional information is deleted. This facilitates producing training data. Therefore, even if the number of items of training data is increased, the workload required to produce training data is inhibited from increasing. Meanwhile, it is only necessary, in an image recognition process, to identify a position where an internal crack is produced in a solar cell that should be identified as being defective. The position can be identified if a processing result is obtained in the form of a like image of the solar cell. Therefore, the precision of the image recognition process is inhibited from being reduced when employing the invention.

<FIG> show an example subject of image recognition processing in accordance with the embodiments of the invention. <FIG> is a perspective view showing a configuration of a solar cell <NUM> subject to inspection, i.e., subject to an image recognition process. The solar cell <NUM> is made of, for example, a monocrystal silicon and has a plate shape including a light receiving surface <NUM> and a back surface <NUM>. The light receiving surface <NUM> is a surface that sunlight is mainly incident to, and the back surface <NUM> is a surface facing away the light receiving surface <NUM>. <FIG> shows an infrared image <NUM> of the solar cell <NUM> of <FIG> captured from the side of the light receiving surface <NUM> or the back surface <NUM>. The image <NUM> shows adsorption pads <NUM>, an etching unevenness <NUM>, an electrode reinforcing wire <NUM>, a saw mark <NUM>, and a wafer thickness distribution <NUM> appearing as patterns on the solar cell <NUM>. These features are publicly known so that a description thereof is omitted. The figure shows a pattern of a non-defective product instead of a defective product.

<FIG> shows an infrared image <NUM> of the solar cell <NUM> of <FIG> captured from the side of the light receiving surface <NUM> or the back surface <NUM>. <FIG> represents the image <NUM> of the solar cell <NUM> different from the image of <FIG>. As shown in the figure, an internal crack <NUM> marked by "x" that is an intersection between a line extending in a direction at an angle of <NUM>° and a line extending in a direction at an angle of <NUM>° is shown on the solar cell <NUM>. The figure shows a pattern of a defective product. The figure only shows the internal crack <NUM> for clarity of explanation. In reality, however, the internal crack <NUM> is potentially found in the image <NUM> as shown in <FIG>. For example, the internal crack <NUM> lays hidden in the adsorption pad <NUM> or in the etching unevenness <NUM>. It is therefore not easy to accurately determine whether the internal crack <NUM> is included in the image <NUM>.

<FIG> shows an outline of a process in a processing device <NUM> according to a comparative example. The processing device <NUM> has a configuration of a convolutional neural network that includes the fully connected layer described above. The processing device <NUM> includes a first convolutional layer 42a, a second convolutional layer 42b, a third convolutional layer 42c, a fourth convolutional layer 42d, a fifth convolutional layer 42e, a sixth convolutional layer 42f, which are generically referred to as convolutional layers <NUM>, a first pooling layer 44a, a second pooling layer 44b, a third pooling layer 44c, a fourth pooling layer 44d, a fifth pooling layer 44e, which are generically referred to as pooling layers <NUM>, and a fully connected layer <NUM>. The convolutional layers <NUM>, the pooling layers <NUM>, and the fully connected layer <NUM> are shown as blocks to give an image of the respective processes. An input image <NUM> is an image subject to an image recognition process in the processing device <NUM> and represents at least a portion of the image <NUM> described above. The input image <NUM> is input to the first convolutional layer 42a.

Each convolutional layer <NUM> is shown as a hexahedron comprised of two first faces <NUM> each having a square shape and each having a depth direction and a height direction, and four second faces <NUM> sandwiched by the two first faces <NUM>. For clarity of illustration, only the first face <NUM> and the second face <NUM> of the first convolutional layer 42a are denoted by reference symbols. The magnitude of the first face <NUM> indicates the size of the spatial dimension of the image processed by the convolutional layer <NUM>, i.e., the size of the image. The convolutional layer <NUM> subjects the image to spatial filtering, successively shifting a spatial filter of a size smaller than the size of the image.

Spatial filtering is a publicly known technology, and a description thereof is omitted. Spatial filtering is equivalent to a convolutional process. The convolution process extracts a feature amount of the image. Padding, etc. may be performed in the convolutional layer <NUM>. The convolutional layer <NUM> may also use a plurality of spatial filters in parallel and perform a plurality of spatial filtering steps in parallel in the image. Using a plurality of spatial filters in parallel increases the image. The number of spatial filters used in parallel in the convolutional layer <NUM> is referred to as the number of channels, which is indicated by the length of the second face <NUM> in the horizontal direction.

Each pooling layer <NUM> is configured in a manner similar to that of the convolutional layer <NUM>. The pooling layer <NUM> reduces the size of the image by aggregating a plurality of pixels included in an arbitrary region in the image into a single pixel. For aggregation of a plurality of pixels into a single pixel, average pooling or maximum pooling is performed. In average pooling, an average value of a plurality of pixel values in the region is used for the single pixel. In maximum pooling, the maximum value of a plurality of pixel values in the region is used for the single pixel. A pooling process is performed to reinforce the robustness for translation of a representative value or an average value in the region of interest.

In the illustrated example, the processes are performed in the order of the first convolutional layer 42a, the first pooling layer 44a, the second convolutional layer 42b, the second pooling layer 44b, the third convolutional layer 42c, the third pooling layer 44c, the fourth convolutional layer 42d, the fourth pooling layer 44d, the fifth convolutional layer 42e, the fifth pooling layer 44e, and the sixth convolutional layer 42f. In other words, a convolutional process and a pooling process are repeatedly performed. By repeating a convolutional process and a pooling process, the size of the image is progressively reduced. The sixth convolutional layer 42f outputs an image having a <NUM>×<NUM> spatial dimension and having one or more channels to the fully connected layer <NUM>. In this case, the number of channels is "<NUM>", by way of one example.

The fully connected layer <NUM> receives an image from which a feature amount is extracted from the sixth convolutional layer 42f. The fully connected layer <NUM> identifies the image by performing organization into a plurality of classes based on the feature amount. A publicly known technology may be used in the process in the fully connected layer <NUM>, and a description thereof is omitted. The result of organization in the fully connected layer <NUM>, i.e., the result of identification, is an output <NUM>. The output <NUM> shows a probability for each of <NUM> classes including "OK", "internal crack", "etching unevenness", "pin hole", "black dot", "chip", "adsorption pad", and "bus bar". In this case, the probability "<NUM>" for "internal crack" is high so that the input image <NUM> is identified to include the internal crack <NUM>.

A learning (training) process is performed for the processing device <NUM> before performing an image recognition process as described above. In a learning process, training data that yields the known output <NUM> and an image for learning corresponding to the training data are input to the processing device <NUM> to learn a coefficient in the fully connected layer <NUM> and a coefficient of the spatial filter of each convolutional layer <NUM>. The image for learning has the same size as the input image <NUM> and is an original image that outputs the training data when the image recognition process is performed accurately. For a learning process like this, a publicly known technology may be used. In a learning process, the larger the amount of combinations of training data and images for learning used, the higher the precision of the coefficients and the more improved the precision of the image recognition process.

When the learning process is terminated, the aforementioned image recognition process is performed. The input image <NUM> produced by cropping the original image <NUM> into a predetermined size is input to the processing device <NUM>. The processing device <NUM> performs an image recognition process in a two-dimensional manner to determine whether the internal crack <NUM> or the like is found in the input image <NUM>. If the internal crack <NUM> is found in the input image <NUM>, the processing device <NUM> produces the output <NUM> indicating the presence of the internal crack <NUM>. This is equivalent to organization into <NUM> classes in the image recognition process.

The convolutional neural network used in the processing device <NUM> includes the fully connected layer <NUM>. Since the size of the output <NUM> from the fully connected layer <NUM> is fixed, the size of the input image <NUM> that can be processed by the convolutional neural network is also fixed. In other words, the processing device <NUM> can only perform a recognition process on the input image <NUM> of a limited size. For this reason, a limitation to the size of the input image <NUM> is provided in the processing device <NUM>.

<FIG> shows an outline of a process in a processing device <NUM> according to a comparative example. The processing device <NUM> has a configuration of a fully convolutional neural network. The processing device <NUM> includes a first convolutional layer 62a, a second convolutional layer 62b, a third convolutional layer 62c, a fourth convolutional layer 62d, a fifth convolutional layer 62e, a sixth convolutional layer 62f, a seventh convolutional layer <NUM>, an eighth convolutional layer <NUM>, a ninth convolutional layer 62i, a tenth convolutional layer 62j, an eleventh convolutional layer <NUM>, a twelfth convolutional layer <NUM>, a thirteenth convolutional layer <NUM>, which are generically referred to as convolutional layers <NUM>, a first pooling layer 64a, a second pooling layer 64b, a third pooling layer 64c, a fourth pooling layer 64d, a fifth pooling layer 64e, which are generically referred to as pooling layers <NUM>, a first up-pooling layer 66a, a second up-pooling layer 66b, a third up-pooling layer 66c, a fourth up-pooling layer 66d, and a fifth up-pooling layer 66e, which are generically referred to as up-pooling layers <NUM>. The convolutional layers <NUM>, the pooling layers <NUM>, and the up-pooling layers <NUM> are shown as blocks to give an image of the respective processes.

An input image <NUM> is an image subject to an image recognition process in the processing device <NUM>. A fully convolutional neural network does not include the fully connected layer mentioned above so that a limit to the size of the input image <NUM> is not provided. For this reason, the input image <NUM> may be the image <NUM> mentioned above. The input image <NUM> is input to the first convolutional layer 62a.

The convolutional layer <NUM> performs a process similar to that of the convolutional layer <NUM> described above, and the pooling layer <NUM> performs a process similar to that of the pooling layer <NUM> described above. The up-pooling layer <NUM> performs a process opposite to the process in the pooling layer <NUM>. In other words, the size of the image is reduced in the pooling layer <NUM>, but the size of the image is enlarged in the up-pooling layer <NUM>. A publicly known technology may be used in the process in the up-pooling layer <NUM>, and a description thereof is omitted.

In a fully convolutional neural network, the downsampling processing unit <NUM> and the upsampling processing unit <NUM> are arranged in the stated order. In the downsampling processing unit <NUM>, the first convolutional layer 62a, the first pooling layer 64a, the second convolutional layer 62b, the second pooling layer 64b, the third convolutional layer 62c, the third pooling layer 64c, the fourth convolutional layer 62d, the fourth pooling layer 64d, the fifth convolutional layer 62e, the fifth pooling layer 64e, and the sixth convolutional layer 62f are arranged in the stated order. In other words, a convolutional process and a pooling process are repeatedly performed. Further, by repeating a convolutional process and a pooling process, the size of the image is progressively reduced.

In the upsampling processing unit <NUM>, the seventh convolutional layer <NUM>, the eighth convolutional layer <NUM>, the first up-pooling layer 66a, the ninth convolutional layer 62i, the second up-pooling layer 66b, the tenth convolutional layer 62j, the third up-pooling layer 66c, the eleventh convolutional layer <NUM>, the fourth up-pooling layer 66d, the twelfth convolutional layer <NUM>, the fifth up-pooling layer 66e, and the thirteenth convolutional layer <NUM> are arranged in the stated order. In other words, a convolutional process and an up-pooling process are repeatedly performed. Further, by repeating a convolutional process and a pooling process, the size of the image is progressively enlarged. The thirteenth convolutional layer <NUM> outputs an image (hereinafter, referred to as "output image <NUM>") of a size close to that of the input image <NUM>.

By subjecting the input image <NUM> to an image recognition process in the downsampling processing unit <NUM> and the upsampling processing unit <NUM>, the output image <NUM> is obtained. Each object included in the output image <NUM> is filled with a color determined by the class. In other words, the output image <NUM> resulting from the image recognition process is subjected to segmentation by filling objects with colors. In the case the internal crack <NUM> is included in the input image <NUM>, for example, the output image <NUM> will include an internal crack region <NUM>. The internal crack region <NUM> is a region recognized as the internal crack <NUM> and is filled with a color different from those of the other regions in the output image <NUM>. In the case an etching unevenness and an adsorption pad are included in the input image <NUM>, the output image <NUM> will include a region recognized as an etching unevenness (hereinafter, referred to as "etching unevenness region") and a region recognized as an adsorption pad (hereinafter, referred to as "adsorption pad region"). In that case, the internal crack region <NUM>, the etching unevenness region, the adsorption region, and other regions are filled with different colors. The output image <NUM> is also called a feature map.

A learning (training) process is performed for the processing device <NUM> before performing an image recognition process described above. In a learning process, training data that yields the known output image <NUM> and an image for learning corresponding to the training data are input to the processing device <NUM> to learn a coefficient of the spatial filter of each convolutional layer <NUM>. The training data is an image subjected to segmentation by filling objects with colors. Generally, the training data like this is produced by a manual operation. For this reason, the larger the number of items of training data, the larger the workload required to produce the training data. If the number of items of training data is decreased to reduce the workload, the precision of the image recognition process is reduced. In order to inhibit a reduction in the precision of the image recognition process while at the same time inhibiting an increase in the workload in this situation, it is necessary to reduce the workload required to produce one item of training data. Since a limit to the size of the input image <NUM> is not provided, the image for learning and the input image <NUM> may have different sizes.

<FIG> show a configuration of a processing device <NUM>. In particular, <FIG> shows a configuration for a learning process, and <FIG> shows a configuration for an image recognition process. The processing device <NUM> in <FIG> and the processing device in <FIG> may be the same device or different devices. The processing device <NUM> includes a first input unit <NUM>, a second input unit <NUM>, and a processing unit <NUM> as features for a learning process and includes an input unit <NUM>, a correction unit <NUM>, a processing unit <NUM>, and an output unit <NUM> as features for an image recognition process. Further, the processing device <NUM> is connected to an imaging device <NUM>. In other words, the processing unit <NUM> is trained in a learning process, and the processing unit <NUM> is used in an image recognition process. Before describing the configuration of the processing device <NUM>, the configuration of the processing unit <NUM> will be described as in the case of <FIG> and <FIG>.

<FIG> shows an outline of a process in the processing unit <NUM>. The processing unit <NUM> includes a first convolutional layer 142a, a second convolutional layer 142b, a third convolutional layer 142c, a fourth convolutional layer 142d, a fifth convolutional layer 142e, a sixth convolutional layer 142f, which are generically referred to as convolutional layers <NUM>, a first pooling layer 144a, a second pooling layer 144b, a third pooling layer 144c, a fourth pooling layer 144d, and a fifth pooling layer 144e, which are generically referred to as pooling layers <NUM>. The convolutional layers <NUM> and the pooling layers <NUM> are shown as blocks to give an image of the respective processes.

An input image <NUM> is an image subject to an image recognition process in the processing device <NUM>. Like a fully convolutional neural network, the neural network in the processing unit <NUM> does not include a fully connected layer so that a limit to the size of the input image <NUM> is not provided. For this reason, the input image <NUM> may be the image <NUM> mentioned above. The input image <NUM> is input to the first pooling layer 144a. The convolutional layer <NUM> performs a process similar to that of the convolutional layer <NUM> and the convolutional layer <NUM> described above, and the pooling layer <NUM> performs a process similar to that of the pooling layer <NUM> and the pooling layer <NUM> described above.

As in a fully convolutional neural network, a downsampling processing unit <NUM> is provided in the processing unit <NUM>. Unlike the case of a fully convolutional neural network, however, an upsampling processing unit is not provided in the processing unit <NUM>. In the downsampling processing unit <NUM>, the first convolutional layer 142a, the first pooling layer 144a, the second convolutional layer 142b, the second pooling layer 144b, the third convolutional layer 142c, the third pooling layer 144c, the fourth convolutional layer 142d, the fourth pooling layer 144d, the fifth convolutional layer 142e, the fifth pooling layer 144e, and the sixth convolutional layer 142f are arranged in the stated order. In other words, a convolutional process and a pooling process are repeatedly performed.

<FIG> shows an outline of a process in the convolutional layer <NUM>. An image <NUM> is subject to a process in the convolutional layer <NUM> and is comprised of <NUM>×<NUM> pixels by way of one example. Each pixel has a pixel value. The convolutional layer <NUM> produces an expanded image <NUM> by expanding the border of the image <NUM> by paddings <NUM>. Each padding <NUM> is also a pixel, and the expanded image <NUM> is comprised of <NUM>×<NUM> pixels. The convolutional layer <NUM> subjects the expanded image <NUM> to convolution by successively shifting a filter <NUM> having a size smaller than that of the expanded image <NUM>. The filter <NUM> has a size of, for example, <NUM>×<NUM>.

The pixel values of the padding <NUM> are determined as follows. In the case the filter <NUM> is applied to the expanded image <NUM> so as to include paddings <NUM>, the processing unit <NUM> uses, in the padding <NUM>, a pixel value in a portion of the image to which the filter <NUM> is applied, i.e., a pixel value of one of pixels. In the case the filter <NUM> is applied to a top left portion of the expanded image <NUM> as shown in <FIG>, five paddings <NUM> and four pixels are included. In this case, one of the pixels value of the four pixels is used as the pixel value of each padding <NUM>. For example, the pixel value of the pixel closest to the padding <NUM> is used as the pixel value of the padding <NUM>. Alternatively, in the case the filter <NUM> is applied to the expanded image <NUM> so as to include paddings <NUM>, the processing unit <NUM> uses, in the paddings <NUM>, a statistical value of a portion of the image to which the filter <NUM> is applied, i.e., a statistical value of pixel values of pixels. In the case the filter <NUM> is applied to the top left portion of the expanded image <NUM> as shown in <FIG>, a statistical value of the pixel values of the four pixels is used as the pixel value of each padding <NUM>. The statistical value may be an average value or a median value. Reference is made back to <FIG>.

By repeating a convolutional process and a pooling process, the size of the image is progressively reduced. The sixth convolutional layer 142f outputs an image (hereinafter, referred to as "output image <NUM>") having one or more channels. The pooling layer <NUM> may be provided in the final stage, and the pooling layer <NUM> may output the output image <NUM>. In this case, too, the number of channels is "<NUM>", by way of one example. In other words, by subjecting the input image <NUM> to an image recognition process in the downsampling processing unit <NUM>, the output image <NUM> is obtained. The output image <NUM> is described later in detail.

Based on the configuration of the processing unit <NUM> as described above, a description will now be given of the learning process in the processing device <NUM> with reference to <FIG>. The first input unit <NUM> receives training data that yields the known output image <NUM>, and the second input unit <NUM> receives an image for learning corresponding to the training data received in the first input unit <NUM>. The processing unit <NUM> has the configuration of <FIG>, and a coefficient of the spatial filter of each convolutional layer <NUM> is learned, based on the training data received in the first input unit <NUM> and the image for learning received in the second input unit <NUM>.

In the embodiments of the invention, the size of the training data has a <NUM>×<NUM> spatial dimension. For this reason, the training data does not have positional information on an object included in the image for learning. The training data for one channel merely indicates the presence of an object of one of the <NUM> classes in the output <NUM> mentioned above. In the case the internal crack <NUM> is included in the image for learning, for example, the training data of the class "internal crack" indicates the presence of an internal crack. The same is true of the other classes. Therefore, the training data for <NUM> channels merely indicates the presence of the objects of the respective classes. In other words, the training data for one channel need only indicate whether an object of one class is present or not. The training data need not be an image subjected to segmentation by filling objects with colors. Thus, as compared with the case of producing g an image subjected to segmentation by filling objects with colors, the workload to produce one item of training data is reduced. As a result, it is possible to increase the number of items of training data while also inhibiting an increase in the workload.

Meanwhile, the image for learning is an original image that outputs the training data when the image recognition process is performed accurately, and its size is defined to result in training data having a <NUM>×<NUM> spatial dimension. For learning of a coefficient of the spatial filter under a situation like this where training data and an image for learning are used, a publicly known technology may be used so that a description thereof is omitted.

A description will now be given of an image recognition process in the processing device <NUM> with reference to <FIG>. In the case the processing devices <NUM> in <FIG> and in <FIG> are different devices, the coefficient of the spatial filter derived by learning in the processing unit <NUM> in <FIG> is set in the processing unit <NUM> in <FIG>.

For example, the imaging device <NUM> is an infrared camera and images the solar cell <NUM> of <FIG> subject to inspection, i.e., subject to an image recognition process, with infrared light. For example, an image is captured in <NUM> bits and <NUM> pixels. The imaging device <NUM> outputs the captured image to the processing device <NUM>. The input unit <NUM> of the processing device <NUM> receive the image (hereinafter, referred to as "image <NUM>") captured by the imaging device <NUM>. <FIG> show an image <NUM> processed in the processing device <NUM>. In particular, <FIG> shows an image <NUM> input to the input unit <NUM>. The image <NUM> shows the light receiving surface <NUM> or the back surface <NUM> of the solar cell <NUM>. Generally, the solar cell <NUM> is shown tilted in the image <NUM>. <FIG> will be described later, and reference is made back to <FIG>. The input unit <NUM> outputs the image <NUM> to the correction unit <NUM>.

The correction unit <NUM> receives the image <NUM> from the input unit <NUM>. The correction unit <NUM> corrects the tilt of the solar cell <NUM> in the image <NUM>. As shown in <FIG>, the correction unit <NUM> identifies a first side L1, a second side L2, a third side L3, and a fourth side L4 of the solar cell <NUM> in the image <NUM> of <FIG>. The correction unit <NUM> also derives a first point of intersection P1 between the extensions of the first side L1 and the second side L2. The correction unit <NUM> also derives a second point of intersection P2, a third point of intersection P3, and a fourth point of intersection P4 as in the case of the first point of intersection P1. Subsequently, the correction unit <NUM> uses perspective projection transform to deform the image <NUM> so as to move the first point of intersection P1, the second point of intersection P2, the third point of intersection P3, and the fourth point of intersection P4 to the defined coordinates. <FIG> shows the result of deformation. The correction unit <NUM> outputs the corrected image <NUM> to the processing unit <NUM>.

The processing unit <NUM> receives the corrected image <NUM> from the correction unit <NUM>. The processing unit <NUM> has a configuration as shown in <FIG>, and the received image <NUM> corresponds to the input image <NUM>. Therefore, the corrected image <NUM> will be referred to as the input image <NUM> below. As mentioned above, the processing unit <NUM> subjects the input image <NUM> to a process of a convolution neural network in which the fully connected layer <NUM> is excluded. The convolutional neural network in the processing unit <NUM> includes the downsampling processing unit <NUM>, and the downsampling processing unit <NUM> includes a plurality of convolutional layers <NUM> and a plurality of pooling layers <NUM>. Therefore, the processing unit <NUM> performs a downsampling process but does not perform an upsampling process.

Further, in the convolutional neural network in the processing unit <NUM>, the spatial filter of the convolutional layer <NUM> is trained to learn the processing result having a <NUM>×<NUM> spatial dimension and having <NUM> channels, i.e., the training data, In particular, the spatial filter of the convolutional layer <NUM> is trained to learn whether the objects of the <NUM> classes corresponding one to one to the <NUM> channels are present or absent. One of the objects of the <NUM> classes is the internal crack <NUM>.

The output image <NUM> resulting from the process or the inspection is obtained through the image recognition process by the processing unit <NUM> that has been trained. The output image <NUM> shows the presence or absence of the objects of the <NUM> classes. Meanwhile, the size of the output image <NUM> is smaller than the size of the input image <NUM>. Therefore, the output image <NUM> lacks, for example, accurate positional information on the internal crack <NUM>. However, inspection by the processing device <NUM> only requires detecting whether the internal crack <NUM> is present or absent so that accurate positional information on the internal crack <NUM> is not necessary. For this reason, lack of accurate positional information on the internal crack <NUM> does not result in a reduction in the precision of the image recognition process. In the case the image for learning and the input image <NUM> have different sizes, the output image <NUM> has a spatial dimension larger than <NUM>×<NUM>, unlike the training data. The processing unit <NUM> outputs the output image <NUM> to the output unit <NUM>, and the output unit <NUM> outputs the output image <NUM> outside.

As mentioned above, the image for learning and the input image <NUM> have different sizes. In particular, the size of the input image <NUM> is larger than the size of the image for learning. The image for learning is input to the fully convolutional neural network to train the filter <NUM> of the convolutional layer and corresponds to the training data. When the input image <NUM> like this is input to the processing device <NUM>, the output image <NUM> output from the output unit <NUM> will have a spatial dimension smaller than the size of the input image <NUM> and larger than <NUM>×<NUM>.

<FIG> show the output image <NUM> output from the output unit <NUM>. <FIG> shows that the output image <NUM> is substantially entirely colored with the same color. This makes it clear that the internal crack <NUM> is not found in the solar cell <NUM>. Meanwhile, <FIG> shows that the image is filled with colors in accordance with the class. The central portion of the output image <NUM> is colored with a darker color than the other portions. This makes it clear that the internal crack <NUM> is found in the central portion of the solar cell <NUM>. In other words, the output image <NUM> has a spatial dimension larger than <NUM>×<NUM> so that the relative position of the internal crack <NUM> in the solar cell <NUM> as well as whether it is present or absent is identified by the feature map. The feature map is also called a heat map.

The device, the system, or the entity that executes the method according to the disclosure is provided with a computer. By causing the computer to run a program, the function of the device, the system, or the entity that executes the method according to the disclosure is realized. The computer is comprised of a processor that operates in accordance with the program as a main hardware feature. The disclosure is non-limiting as to the type of the processor so long as the function is realized by running the program. The processor is comprised of one or a plurality of electronic circuits including a semiconductor integrated circuit (IC) or a large-scale integration (LSI). The plurality of electronic circuits may be integrated in one chip or provided in a plurality of chips. The plurality of chips may be aggregated in one device or provided in a plurality of devices. The program is recorded in a non-transitory recording medium such as a computer-readable ROM, optical disk, and hard disk drive. The program may be stored in a recording medium in advance or supplied to a recording medium via wide area communication network including the Internet.

A description will now be given of the operation of the processing device <NUM> having the configuration described above. <FIG> show a flowchart showing a sequence of steps performed in the processing device <NUM>. <FIG> is a flowchart showing a sequence of steps in a learning process. The first input unit <NUM> receives an input of training data, and the second input unit <NUM> receives an input of image for learning (S10). The processing unit <NUM> uses the training data and the image for learning to learn a coefficient of the spatial filter of each convolutional layer <NUM> (S12). When any combination of training data and image for learning remains unprocessed (Y in S14), control is returned to step <NUM>. When no combinations of training data and image for learning remain (N in S14), the process is terminated.

<FIG> is a flowchart showing a sequence of steps in an image recognition process. The input unit <NUM> receives an input of the image <NUM> (S50). The correction unit <NUM> corrects the tilt of the image <NUM> (S52). The processing unit <NUM> performs an image recognition process by using the corrected image <NUM> as the input image <NUM> (S54). The output unit <NUM> outputs the output image <NUM> resulting from the image recognition process (S56).

According to embodiments of the invention, a fully convolutional neural network in which the spatial filter of the convolutional layer <NUM> has learned training data having a <NUM>×<NUM> spatial dimension is used. Therefore, the workload required to produce training data is reduced. Further, since the workload required to produce training data is reduced, the workload required for learning is inhibited from increasing. Further, since the workload required to produce training data is reduced, the number of items of training data can be increased. Further, by increasing the number of items of training data, the precision of learning can be improved.

Further, since a convolutional neural network in which the spatial filter of the convolutional layer <NUM> has learned training data having a <NUM>×<NUM> spatial dimension is used, it is possible to detect whether a target object is present or absent. Further, since whether a target object is present or absent is detected, the precision of the image recognition process is inhibited from being lowered. Further, since a convolutional neural network in which the spatial filter of the convolutional layer <NUM> has learned training data having a <NUM>×<NUM> spatial dimension is used, it is possible to inhibit a reduction in the precision of the image recognition process while at the same time inhibiting an increase in the workload required for learning.

Further, since the convolutional neural network performs a downsampling process by using the convolutional layers <NUM> and the pooling layers <NUM> and does not perform an upsampling process, training data having a <NUM>×<NUM> spatial dimension can be used for learning. Further, since an image capturing the solar cell <NUM> is input in a situation in which the spatial filter of the convolutional layer <NUM> has learned at least the presence or absence of the internal crack <NUM>, an inspection can be made to determine whether the internal crack <NUM> is present or absent.

Further, by inputting the input image <NUM> of a size larger than the size of the image for learning, the output image <NUM> having a spatial dimension equal to or larger than <NUM>×<NUM> can be obtained. Further, since the output image <NUM> having a spatial dimension equal to or larger than <NUM>×<NUM> can be obtained, relative positional information can be obtained. Further, by using a pixel value of a pixel included in the image as a pixel value of the padding <NUM>, deterioration in detection precision due to the padding <NUM> is inhibited. Further, by using a statistical value of pixel values of pixels included in the image as a pixel of the padding <NUM>, deterioration in detection precision due to the padding <NUM> is inhibited.

Given above is a description of the present invention based on an exemplary embodiment. The embodiment is intended to be illustrative only.

The configuration of the processing unit <NUM> according to the described embodiment is such that the plurality of convolutional layers <NUM> and the plurality of pooling layers <NUM> are alternately arranged. Alternatively, the processing unit <NUM> may have a configuration of a GoogleNet-based network, a DenseNet-based network, etc. According to this variation, the flexibility in configuration can be improved.

The processing device <NUM> according to the embodiment performs a process for detecting whether the internal crack <NUM> is present or absent by referring to the input image <NUM> of the solar cell <NUM>. Alternatively, the input image <NUM> that captures an image of an object other than the solar cell <NUM> may be subject to the process. Further, a process for detecting whether a defect other than the internal crack <NUM> is present or absent may be performed. Still further, a process for detecting one or more elements that could be included in an object may be performed instead of a process for detecting whether a defect in the object is present or absent. According to this variation, the scope of application of the processing device <NUM> can be broadened.

<NUM> solar cell, <NUM> internal crack, <NUM> processing device, <NUM> first input unit, <NUM> second input unit, <NUM> processing unit, <NUM> image, <NUM> imaging device, <NUM> input unit, <NUM> correction unit, <NUM> output unit, <NUM> input image, <NUM> convolutional layer, <NUM> pooling layer, <NUM> output image, <NUM> downsampling processing unit.

Claim 1:
A processing device (<NUM>) comprising:
an input unit (<NUM>) configured to receive an input image (<NUM>, <NUM>) to be subjected to an image recognition process;
a processing unit (<NUM>) that includes a down-sampling processing unit (<NUM>) configured to subject the input image (<NUM>, <NUM>) to the image recognition process using a fully convolutional neural network that includes convolutional layers (<NUM>) and pooling layers (<NUM>) and which does not include a fully connected layer, wherein the image recognition process performed by the down-sampling processing unit (<NUM>) subjects the input image (<NUM>, <NUM>) to a down-sampling process and does not include an up-sampling process, wherein the image recognition process performed by the down-sampling processing unit (<NUM>) includes spatial filtering of the input image (<NUM>, <NUM>) using spatial filters (<NUM>) of the convolutional layers (<NUM>); and
an output unit (<NUM>) configured to output an output image (<NUM>) as a result of the image recognition process in the processing unit (<NUM>) performed on the input image (<NUM>, <NUM>), wherein the output image (<NUM>) has a spatial dimension smaller than the size of the input image (<NUM>, <NUM>) and larger than <NUM>×<NUM> and represents a feature map indicating presence or absence of an object of a respective class in the input image (<NUM>, <NUM>);
wherein the processing unit (<NUM>) is further configured to perform a learning process in which the spatial filters (<NUM>) of the convolutional layers (<NUM>) in the fully convolutional neural network are trained using an image for learning and training data having a <NUM>×<NUM> spatial dimension and indicating the presence of the object of the respective class, wherein image recognition performed on the image for learning results in the training data as the result of the image recognition process in the processing unit (<NUM>), and
wherein a size of the input image (<NUM>, <NUM>) is larger than a size of the image for learning.