Method for deriving additional and further pictures from an original picture, and device applying the method

A method for deriving further and additional pictures from an original picture, for Artificial Intelligence (AI) training purposes, is applied in a device. The device establishes an original picture set and sets the original pictures as a training picture set for AI training. The original pictures are rotated or flipped or both to obtain amplification pictures. The original pictures are annotated, and each of the amplification pictures is annotated according to a preset conversion rule. The original pictures, the amplification pictures, the annotated original pictures, and the annotated amplification pictures are stored, for inclusion in the AI training picture set.

This application claims priority to Chinese Patent Application No. 201910141909.1 filed on Feb. 26, 2019, the contents of which are incorporated by reference herein.

FIELD

The subject matter herein generally relates to data processing.

BACKGROUND

In the prior art, AI model training pictures obtained from a production line are limited by equipment, cost, time, and other factors, so that the number of pictures used for training may not be sufficient, thereby reducing the accuracy of the AI deep learning model.

DETAILED DESCRIPTION

The present disclosure, including the accompanying drawings, is illustrated by way of examples and not by way of limitation. Several definitions that apply throughout this disclosure will now be presented. It should be noted that references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and such references mean “at least one.”

FIG. 1illustrates a device10for deriving additional and further pictures from an original picture. The device10includes a storage100and a processor200. The storage100is connected to the processor200. In one embodiment, the storage100stores an amount of data of the device10. The amount of data can be, but is not limited to, a program code for picture flipping or rotation, annotation, and annotation conversion. In at least one exemplary embodiment, the storage100can include various types of non-transitory computer-readable storage mediums. For example, the storage100can be an internal storage system of the target service provider server 1, such as a flash memory, a random access memory (RAM) for temporary storage of information, a read-only memory (ROM) for permanent storage of information, a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), a One-time Programmable Read-Only Memory (OTPROM), an Electrically-Erasable Programmable Read-Only Memory (EEPROM), and/or a Compact Disc Read-Only Memory (CD-ROM). In another embodiment, the storage100can also be an external storage system, such as a hard disk, a storage card, or a data storage medium. In the embodiment, the processor200can be a central processing unit, or a common processor, a digital signal processor, a dedicated integrated circuit, ready-made programmable gate arrays, or other programmable logic devices, discrete door or transistor logic devices, discrete hardware components, and so on. In another embodiment, the processor200can be a microprocessor or any conventional processor. The processor200can also be a control center of the device10, using various interfaces and lines to connect the various parts of the device10.

In one embodiment, the device10can amplify and annotate original pictures to get amplification pictures. In one embodiment, the original picture can be used as a new picture for AI deep learning model training to improve the accuracy of AI deep learning model as long as there is a change from the original picture. Both the original pictures and the amplification pictures stored in the memory100are annotated respectively by the unified annotation format, which is used to train the AI deep learning model.

The device10flips the original picture to get a flipped picture. Therefore, the original picture and the flipped picture form a mirror symmetry. In one embodiment, the original picture can be flipped vertically or horizontally or both.

The device10rotates the original picture at a preset angle to get a first amplifying picture. In one embodiment, the preset angle can be in a range between 0 degrees and 360 degrees. When the original picture is rotated by a slight angle to produce a change, the original picture so rotated can be a new training picture for the AI deep learning model.

Similarly, a second amplifying picture can be acquired by rotating the original picture at the preset angle. The first amplification picture and the second amplification picture form a amplification picture set, which is used to increase the number of pictures for training in the AI deep learning model.

The original picture and the flipped picture are rotated K times in the same direction to get the first amplification picture and the second amplification picture. Each rotation angle can be calculated according to formula θ=(2×270°)/N, wherein K is calculated according to formula K=(N/2)−1, N is a multiple of the number of the amplification pictures (namely the first amplification picture and the second amplification picture) and N is an even number. Depending on the required multiplication of pictures, the original picture or the flipped pictures are rotated 90 degrees, 180 degrees, or 270 degrees.

In one embodiment, the original picture, the flipped picture, the first amplification picture and the second amplification picture are stored in the preset storage address of the storage100, for annotation process.

The program code stored in the storage100is also for annotating the original picture. Such process includes calculating dimensional coordinates of the original picture, a point coordinate of annotation site in the original picture, and a format coordinate of annotation site in the original picture. The annotation site in the original picture is obtained by using segmentation method, which is a way of annotating a picture that depicts the outline frame of the picture's annotation site in the form of polygonal point coordinates.

In one embodiment, the dimensional coordinates correspond to the outline of the original picture that represents the original picture, and the dimensional coordinates here are represented by S. A value of the dimensional coordinates is measured by the number of unit areas of the dimensional coordinates. In one embodiment, the values of the dimensional coordinates of one unit area of the original picture are two, in both horizontal and vertical axes of the dimensional coordinates. The values of the dimensional coordinates of two unit areas arrange horizontally in the original picture are three, in horizontal axes, the values of the dimensional coordinates of two unit areas arrange horizontally in the original picture are two in vertical axes. The value of the dimensional coordinates of n unit areas arrange horizontally or vertically in the original picture are n+1 in the horizontal axes of the dimensional coordinate or in the vertical axes of the dimensional coordinate, wherein n is the number of unit areas. The point coordinate is one of the point coordinates in the annotation site of the original picture, represented by A. The format coordinate consists of the minimum horizontal axes coordinate value of the annotation site, the minimum vertical axes coordinate value of the annotation site, the width of the annotation site, and the height of the annotation site. The format coordinate is represented by a bounding box (Bbox). The Bbox is a rectangle that can cover the annotation point of the original picture.

In one embodiment, the program code stored in the storage100is also for annotating the amplification picture. In one embodiment, the flipped picture, the first amplification picture, and the second amplification picture are all together the amplification picture.

In one embodiment, the dimensional coordinate of the amplification picture is acquired by converting the coordinate value of the dimensional coordinate of the original picture. The dimensional coordinate of the amplification picture is represented by S′. The point coordinates of the annotation site of the amplification picture are obtained by converting the coordinate value of the point coordinate and the coordinate value of the dimensional coordinate in the annotation site of the original picture, which is represented by A′. The format coordinate of the annotation site of the amplification picture is acquired by converting the coordinate value of the point coordinate and the coordinate value of the format coordinate in the annotation part of the original picture, which is represented by Bbox′.

FIG. 2illustrates system300for deriving additional and further pictures from an original picture, the system300includes one or more modules running in the device10. The system300includes a picture building module310, a flip module320, an annotation module330, and a storing module340. In one embodiment, the picture building module310, the flip module320, the annotation module330, and the storing module340are stored in the storage100of the device10and executed by the processor200of the device10. In another embodiment, the picture building module310, the flip module320, the annotation module330, and the storing module340are program segments or code embedded in the processor200.

The picture building module310establishes an original picture set including a number of the original pictures, and sets the original pictures as a training picture set for training an AI deep learning model.

In one embodiment, the picture building module310acquires the original pictures from an external device, and establishes the original picture set according to the original pictures. The external device can be a picture database.

In another embodiment, the picture building module310annotates the original picture and establishes the training picture set according to the annotated original pictures for training the AI deep learning model.

The flip module320rotates or flips the number of original pictures to get amplification pictures and puts the amplification pictures in the training picture set for training the AI deep learning model.

In one embodiment, the flip module320flips and rotates the original picture to get the amplification pictures and establishes an amplification picture set based on the amplification pictures. The amplification picture set is used to increase the number of the training pictures for training the AI deep learning model.

In one embodiment, the flip module320flips the original picture to get the flip picture, so that the original picture is mirrored symmetrically with the flip picture. In one embodiment, the flip module320rotates the original picture through a preset angle to get the first amplification picture and rotates the flip picture in the same direction through the same angle to get the second amplification picture. In at least one embodiment, the preset angle is in a range between 0° and 360°.

In one embodiment, the flip module320rotates the original picture K times at the preset angle to get the first amplification pictures, and rotates the flipped picture K times at the preset angle in the same direction to get the second amplification pictures. The first amplification pictures and the second amplification pictures form the training pictures. The preset angle can be calculated according to formula θ=(2×270°)/N, wherein K is calculated according to formula K=(N/2)−1, N is a multiple of the number of the amplification pictures (namely the first amplification picture and the second amplification picture) and N is an even number. Depending on the number of derivations which are desired, the original picture or the flipped pictures are rotated 90 degrees, 180 degrees, or 270 degrees, respectively.

FIG. 3illustrates the multiple amplification pictures from an original picture that has been flipped and rotated. In one embodiment, N is 8, namely, the number of derived training pictures is 8 times that of the original picture.

First, the original picture a1can be flipped to get the flip picture b1after a vertical flip, and the original picture a1and the flip picture b1form a mirror symmetry. The number of training pictures is increased to two times the original one picture.

Then, the K is calculated according to the formula K=(8/2)−1=3, and the original picture a1and the flip picture b1are rotated three times in clockwise. Each time, a rotation angle is (2×360)/8=90°, namely, the rotation angle corresponding to the first time is 90° and the original pictures are rotated at 90° to get the first amplification pictures a1. The rotation angle corresponding to the second time is 180° and the original pictures are rotated at 180° to get the first amplification pictures a2. The rotation angle corresponding to the third time is 270° and the original pictures are rotated at 270° to get the first amplification pictures a3. The flip pictures b1are rotated three times in clockwise direction to get the second amplification pictures b2, b3, and b4. After the original picture a1and the flip picture b1are so rotated, the number of the training pictures is increased to 8 times the original one picture.

The annotation module330annotates the original picture and annotates the amplification picture according to a preset conversion rule. In one embodiment, annotating the original pictures includes calculating the dimensional coordinate of the original picture and the point coordinates and format coordinates of the annotation site of the original picture. The dimensional coordinate is the outline of the original picture that represents the original picture, and the dimensional coordinates here are represented by S=(Sx, Sy). In one embodiment, the coordinate value of the horizontal axes coordinate of the dimensional coordinate is determined according to unit areas of size in the horizontal axes coordinate of the dimensional coordinate. The coordinate value of the vertical axes coordinate of the dimensional coordinate is determined according to unit areas of size in the vertical axes coordinate of the dimensional coordinate.

InFIG. 3, the original picture occupies 6 unit areas of size in the horizontal axes coordinate, then the coordinate value of the horizontal axes coordinate Sxis 7. The original picture occupies 4 units area of size in the vertical axes coordinate, then the coordinate value of the vertical axes coordinate Syis 5. The point coordinate is the coordinate of one of points in the annotation site, which is represented by A=(X, Y). When the upper left corner ofFIG. 3is set as origin, the point coordinate A=(4,3). The format coordinate consists of the minimum horizontal axes coordinate value of the annotation site in the original picture, the minimum vertical axes coordinate value of the annotation site in the original picture, the width of the annotation site in the original picture, and the height of the annotation site in the original picture. The format coordinate is represented by Bbox=(Xmin, Ymin, W, H). Therein, Xminis the minimum coordinate value of the horizontal axes coordinates of the annotation site, Yminis the minimum coordinate value of vertical axes coordinates of the annotation site, W is the coordinate value of the width of the annotation site, and H is the coordinate value of the height of the annotation site. When the upper left corner ofFIG. 3is set as the origin, the Xminis 4, the Yminis 3, the W is 2, and the H is 1, namely, the Bbox=(4, 3, 2, 1).

The dimensional coordinate of the amplification picture, point coordinate of the amplification picture, and the format coordinate of the amplification picture are represented respectively by S′, A′ and Bbox′.

The preset conversion rule is as follows. The device converts the dimensional coordinate of the annotation site of the annotated original pictures to get the dimensional coordinate of the annotation site of the annotated amplification pictures and converts the dimensional coordinate of the annotation site of the annotated original pictures and the point coordinate of the annotation site of the annotated original pictures to get the point coordinate of the annotation site of the annotated amplification pictures. The rule also converts the dimensional coordinate of the annotation site of the annotated original pictures and the format coordinate of the annotation site of the annotated original pictures to get the format coordinate of the annotation site of the annotated amplification pictures.

In detail, the coordinate value of the dimensional coordinate of the amplification picture is the same as the coordinate value of the dimensional coordinate of the original picture, namely, S′=S=(Sx, Sy). The coordinate value of the horizontal axes coordinate of the point coordinate in the amplification picture is same as the coordinate value of the horizontal axes coordinate of the point coordinate in the original picture. The coordinate value of the vertical axes coordinate of the point coordinate in the amplification picture can be calculated by making the coordinate value of the vertical axes coordinate of the dimensional coordinate in original picture subtract one and the coordinate value of the vertical axes coordinate of the point coordinate in original picture. Namely, the point coordinate of the amplification picture is S′=(X, Sy−1−Y). The coordinate value of the horizontal axes coordinate of the point coordinate in the amplification picture is same as the coordinate value of the horizontal axes coordinate of the point coordinate in the original picture. The coordinate value of the horizontal axes coordinate of the format coordinate in the amplification picture is same as the coordinate value of the horizontal axes coordinate of the format coordinate in the original picture. The coordinate value of the vertical axes coordinate of the format coordinate in the amplification picture can be calculated by making the coordinate value of the vertical axes coordinate of the format coordinate in original picture subtract one, the coordinate value of the vertical axes coordinate of the format coordinate in the original picture, and the width of the format coordinate in the amplification picture. The coordinate value of the width of the annotation site in the amplification picture and the coordinate value of the height of the annotation site in the amplification picture remain unchanged during flip process and rotation process. Namely, the format coordinate of the amplification picture is S′=(Xmin, Sy−1−Y−H, W, H).

The derived pictures acquired by rotating the original picture at 90 degrees, 180 degrees, and 270 degrees are in the following table 1.

The storing module340stores the original picture, the amplification picture, the annotated original picture, and the annotated amplification picture, and names the original picture and the amplification picture to train an AI depth learning model.

The device10gets the annotated amplification picture from the original picture, and trains the AI depth learning model by using the annotated amplification picture to get an AI deep learning model with higher accuracy.

FIG. 4illustrates a flowchart of an embodiment of a method for deriving additional and further pictures from an original picture. The method is provided by way of example, as there are a variety of ways to carry out the method. The method described below can be carried out using the configurations illustrated inFIGS. 1-3, for example, and various elements of these figures are referenced in explaining the example method. Each block shown inFIG. 4represents one or more processes, methods, or subroutines carried out in the example method. Furthermore, the illustrated order of blocks is by example only and the order of the blocks can be changed. Additional blocks may be added or fewer blocks may be utilized, without departing from this disclosure. The example method can begin at block401.

At block401, the device establishes an original picture set including a number of original pictures, and sets the original pictures as a training picture set for training an AI deep learning model.

In one embodiment, the device acquires the original pictures from an external device, and establishes the original picture set according to the original pictures. The external device can be picture database.

In another embodiment, the device annotates the original picture, establishes the training picture set according to the annotated original pictures for training an AI deep learning model.

At block402, the device rotates or flips the number of original pictures to get amplification pictures.

In one embodiment, the device flips and rotates the original picture to get the amplification pictures, and establish an amplification picture set based on the amplification pictures. The amplification picture set is used to increase the number of the training picture s for training the AI deep learning model.

In one embodiment, the method further includes: at block4021, the device flips the original picture to get the flip picture and makes the original picture mirrored symmetrically with the flip picture; at block4022, the device rotates the original picture through a preset angle to get a first amplification picture and rotates the flip picture in the same direction to get a second amplification picture. In at least one embodiment, the preset angle is in a range between 0° and 360°.

In one embodiment, the device rotates the original pictures K times at the preset angle to get the first amplification pictures, and rotates the flipped pictures K times at the preset angle in the same direction to get the second amplification pictures. The first amplification pictures and the second amplification pictures forms the training pictures. The preset angle can be calculated according to formula θ=(2×270°)/N, wherein K is calculated according to formula K=(N/2)−1, N is a multiple of the number of the amplification pictures (namely the first amplification picture and the second amplification picture) and N is an even number. Depending on the multiple of the number of the amplification picture, the original picture or the flipped pictures are rotated 90 degrees, 180 degrees, or 270 degrees, respectively.

In one embodiment, the N is 8, namely, the number of training picture is 8 times that of the original picture.

First, the original picture a1can be flipped to get the flip picture b1after a vertical flip, and the original picture a1and the flip picture b1form a mirror symmetry. The number of training pictures is increases to two times the original one picture.

Then, the K is calculated according to the formula K=(8/2)−1=3, and the original picture a1and the flip picture b1are rotated three times along in clockwise. Each time, a rotation angle is (2×360)/8=90°, namely, the rotation angle corresponding to the first time is 90° and the original pictures are rotated at 90° to get the first amplification pictures a1. The rotation angle corresponding to the second time is 180° and the original pictures are rotated at 180° to get the first amplification pictures a2. The rotation angle corresponding to the third time is 270° and the original pictures are rotated at 270° to get the first amplification pictures a3. The flip pictures b1are rotated three times in clockwise to get the second amplification pictures b2, b3, and b4. After the original picture a1and the flip picture b1are so rotated, the number of the training pictures is increased to 8 times the original one picture.

At block403, the device annotates the original pictures.

In one embodiment, annotating the original pictures includes calculating the dimensional coordinates of the original pictures and the point coordinates and format coordinates of the annotation site of the original pictures. The dimensional coordinates are the outline of the original picture that represents the original picture, and the dimensional coordinates here are represented by S=(Sx, Sy). In one embodiment, the coordinate value of the horizontal axes coordinate of the dimensional coordinate is determined according to unit areas of size in the horizontal axes coordinate of the dimensional coordinate, and the coordinate value of the vertical axes coordinate of the dimensional coordinate is determined according to unit areas of size in the vertical axes coordinate of the dimensional coordinate.

InFIG. 3, the original picture occupies 6 unit areas of size in the horizontal axes coordinate, then the coordinate value of the horizontal axes coordinate Sxis 7. The original picture occupies 4 units area of size in the vertical axes coordinate, then the coordinate value of the vertical axes coordinate Syis 5. The point coordinate is the coordinate of one of points in the annotation site, which is represented by A=(X, Y). When the upper left corner ofFIG. 3is set as origin, and the point coordinate A=(4,3). The format coordinate consists of the minimum horizontal axes coordinate value the annotation site in the original picture, the minimum vertical axes coordinate value the annotation site in the original picture, the width of the annotation site in the original picture, and the height of the annotation site in the original picture. The format coordinate is represented by Bbox=(Xmin, Ymin, W, H). Therein Xminis the minimum coordinate value of the horizontal axes coordinates of the annotation site, Yminis the minimum coordinate value of vertical axes coordinates of the annotation site, W is the coordinate value of the width of the annotation site, and H is the coordinate value of the height of the annotation site. When the cupper left corner ofFIG. 3is set as origin, and Xminis 4, Yminis 3, W is 2, and H is 1, namely, Bbox=(4, 3, 2, 1).

At block404, the device annotates the amplification picture according to a preset conversion rule.

The dimensional coordinate of the amplification picture, point coordinate of the amplification picture and the format coordinate of the amplification picture are represented respectively by S′, A′ and Bbox′.

The preset conversion rule is illustrated as follow. The device convert the dimensional coordinate of the annotation site of the annotated original pictures to get the dimensional coordinate of the annotation site of the annotated amplification pictures, convert the dimensional coordinate of the annotation site of the annotated original pictures and the point coordinate of the annotation site of the annotated original pictures to get the point coordinate of the annotation site of the annotated amplification pictures, and convert the dimensional coordinate of the annotation site of the annotated original pictures and the format coordinate of the annotation site of the annotated original pictures to get the format coordinate of the annotation site of the annotated amplification pictures.

In detail, the coordinate value of the dimensional coordinate of the amplification picture is the same as the coordinate value of the dimensional coordinate of the original picture, namely, S′=S=(Sx, Sy). The coordinate value of the horizontal axes coordinate of the point coordinate in the amplification picture is same as the coordinate value of the horizontal axes coordinate of the point coordinate in the original picture. The coordinate value of the vertical axes coordinate of the point coordinate in the amplification picture can be calculated by making the coordinate value of the vertical axes coordinate of the dimensional coordinate in original picture subtract one and the coordinate value of the vertical axes coordinate of the point coordinate in original picture. Namely, the point coordinate of the amplification picture is S′=(X, Sy−1−Y). The coordinate value of the horizontal axes coordinate of the point coordinate in the amplification picture is as same as the coordinate value of the horizontal axes coordinate of the point coordinate in the original picture. The coordinate value of the horizontal axes coordinate of the format coordinate in the amplification picture is as same as the coordinate value of the horizontal axes coordinate of the format coordinate in the original picture. The coordinate value of the vertical axes coordinate of the format coordinate in the amplification picture can be calculated by making the coordinate value of the vertical axes coordinate of the format coordinate in original picture subtract one, the coordinate value of the vertical axes coordinate of the format coordinate in the original picture, and the width of the format coordinate in the amplification picture. The coordinate value of the width of the annotation site in the amplification picture and the coordinate value of the height of the annotation site in the amplification picture remain unchanged during flip process and rotation process. Namely, the format coordinate of the amplification picture is S′=(Xmin, Sy−1−Y−H, W, H).

The pictures which are acquired by rotating the original picture at 90 degrees, 180 degrees, and 270 degrees are illustrated in the following table 1.

At block405, the device stores the original picture, the amplification picture, the annotated original picture, and the annotated amplification picture in the training picture set.

The device for picture amplification and annotation gets the annotated amplification picture from the original picture, and trains the AI depth learning model by using the annotated amplification picture and the original picture to get a higher accuracy AI deep learning model.