Patent ID: 12236703

DETAILED DESCRIPTION OF THE EMBODIMENTS

To make the foregoing objectives, features, and advantages of the present disclosure clearer and more comprehensible, the specific implementations of the present disclosure are described in detail below with reference to the drawings. The following describes many details in order to provide a thorough understanding of the present disclosure. However, the present disclosure can be implemented in many other ways other than those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the present disclosure, and thus the present disclosure is not limited to the specific embodiments disclosed below. The technical features in the embodiments of the present disclosure may be combined correspondingly under the premise of no conflict.

The present disclosure provides a method for fish identification based on body surface texture features and geometric features. The method firstly employs machine vision and deep learning techniques to quantize body surface texture features and geometric features of a fished farmed in circulating water, respectively, and then realizes coupled analysis of these features by means of a small sample learning network, and thus realizes accurate identity recognition of a fish individual.

Meanwhile, the present disclosure further provides a system for fish identification based on body surface texture features and geometric features. The system includes a underwater camera, a server, a fill light, a water temperature sensor, a turbidity sensor, etc. The underwater camera and the fill light are mounted inside a farming pond and connected to the server.

According to a particular embodiment of the present disclosure, the above-mentioned method/system includes or performs the following steps.

(1) The server triggers the underwater camera to read a real-time picture, and automatically marks feature points of a fish individual in a current farming pond using a deep learning algorithm YoLov8. According to an example of the present disclosure, as shown inFIG.1, 20 feature points (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, and 20) on the surface of the fish body are obtained. The feature point 1 is a fish mouth, i.e., a fish front endpoint; the feature point 18 is a fish tail midpoint, i.e., a fish caudal fin base midpoint, which is a midpoint of a connecting line of an upper fish caudal fin base point 13 and a lower fish caudal fin base point 10; in this way, an error caused by fish tail swing can be effectively prevented; a body length is calculated based on a distance from a snout to the caudal fin base midpoint; moreover, 19 is a highest point of the fish body; an intersection point of a connecting line of the fish body front endpoint and the fish tail midpoint and a perpendicular line to the highest point of the fish body is a fish body center point; a body width is calculated based on a height of a perpendicular line emanated from the highest point of the fish body, i.e., based on the feature point 19 and the feature point 5; and the feature point 15 is an intersection point of the body length and the body width, which is also a midpoint of the entire fish. These feature points are basic feature points, and other feature points are optional feature points which can be adjusted as needed or according to a fish type. In the example as shown inFIG.1, the optional feature points may be selected as shown in the figure. For example, 2 is a fish eye; 3, 4, and 20 are 3 feature points of the fish head; 5, 6, 7, and 8 are 4 feature points of the pectoral fin; 9, 11, 12, and 14 are 4 feature points of the fish tail; and 16 and 17 are feature points of the dorsal fin. According to the above-mentioned distribution of the feature points, 20 feature points are manually marked on an acquired photo (two photos for each fish individual, and a total of 100 fish individuals). The marked photos are used for training on YoLov8 to obtain a corresponding network for subsequent automatic marking. The specific structure and the training manner of YoLov8 belong to the prior art, and can be implemented with reference to the related prior art, which will not be repeated here.

Individual body length (L) information of a fish school is obtained based on the obtained feature points.

Due to underwater shooting, in consideration of an influence of a fish tail swing frequency f on an individual body length of a fish body, the fish body may bend when the fish tail swings such that the obtained fish body length is less than an actual fish body length. Therefore, the actual fish body length is set to L, and a bent fish body length to L0, a bending degree to k, a fish body width to x1, and a fish tail width to x2, and the following formula is established:

k=x⁢2x⁢1⁢sin⁢2⁢Π⁢❘"\[LeftBracketingBar]"f❘"\[RightBracketingBar]"(1)where f may be measured by an existing method (which, for example, may be measured by YOLOV8); and |f| refers to ascertaining a value of the frequency with no consideration of units.

(1) When the fish tail swing frequency f is less than or equal to 0.5, the bending degree of the fish body may be ignored to obtain:
L=L0,
L0=√{square root over ((xp−xq)2+(yp−yq)2)}  (2)

(2) When the fish tail swing frequency f is greater than 0.5, the bending degree of the fish body is as follows:

K=,L-L⁢0L,(2)
and the actual fish body length is derived as follows:

L=L⁢01-k.(4)
A final calculation formula for the actual fish body length is obtained according to formulas (1), (2), (3), and (4) as follows:

Li=x⁢1x⁢1-x⁢2⁢sin⁢2⁢Π⁢❘"\[LeftBracketingBar]"f❘"\[RightBracketingBar]"⁢(xp-xq)2+(yp-yq)2;where 1≤i≤N, N being a number of farmed fish individuals in the current farming pond; Lirepresents a body length of an ith object; p and q are serial numbers of feature points; xp, yp, xq, and yq represent coordinates of the fish body front endpoint p and the feature point fish tail midpoint q, respectively. In this example, p, q=1, 2, . . . , 20. Here, p is 18, and q is 1. The bent fish body length L0 is a Euclidean distance between the feature points 1 and 18.

(2) The geometric features of the fish are quantized.

It is assumed that the coordinates of a top left corner of a fish body image are (0, 0), and Euclidean distances between the fish body center point (the feature point 15) in the current farming pond and other feature points are separately calculated using Yolov8.

Due to underwater shooting, in consideration of an influence of an oxygen concentration on a fish body form, the oxygen concentration is E when the fish form is normal (which is the same as the state illustrated inFIG.1), and the oxygen concentration is E1 in other forms (e.g., the head is facing up) (the oxygen concentration in this state may be measured in advance). When the oxygen concentration changes, the Euclidean distance between the feature points 15 and 19 is measured as h1 and the Euclidean distance between the feature points 5 and 19 is measured as h at this point, and the following formula can be established:

E⁢1/E=h⁢1h/12
(where ½ refers to that the feature point 15 is the midpoint of the body width at the original oxygen concentration), and the following formula can be established:

E=E⁢1⁢h2⁢h⁢1.
The actual Euclidean distances between the feature points 1, 2, 3, 4, 5, 6, 7, 8, 9, 14, 16, 17, 19, and 20, and the feature point 15 are calculated by the following formula:

dmo=E⁢(xm-xo)2+(ym-yo)2=E⁢1⁢h2⁢h⁢1⁢(xm-xo)2+(ym-yo)2
(these feature points are not influenced by the bending degree of the fish tail, but not influenced by the oxygen concentration). Here, m is selected from 1, 2, 3, 4, 5, 6, 7, 8, 9, 14, 16, 17, 19, and 20, and o is 15. The actual distances between the feature points 10, 11, 12, 13, and 18, and the feature point 15 are calculated by the following formula:

dmo=x⁢1x⁢1-x⁢2⁢sin⁢2⁢Π⁢❘"\[LeftBracketingBar]"f❘"\[RightBracketingBar]"⁢E⁢(xm-xo)2+(ym-yo)2.
Therefore, the following formula is derived:

dmo=x⁢1x⁢1-x⁢2⁢sin⁢2⁢Π⁢❘"\[LeftBracketingBar]"f❘"\[RightBracketingBar]"⁢E⁢1⁢h2⁢h⁢1⁢(xm-xo)2+(ym-yo)2
(influenced by both the oxygen concentration and the bending degree of the fish tail). Here, m is selected from 10, 11, 12, 13, and 18, and o is 15. The actual Euclidean distances are normalized to obtain a geometric feature:

Zmo=dmoL,
where L represents the fish body length.

A connecting line of the feature points 1 and 15 is a vector a and a connecting line of other feature point and the feature point 15 is a vector bm(m is other feature point than 1 and 15), where vector

→a=P1-P15=(x1-x15,y1-y15),
and vector

→bm=Pm-P15=(xm-x15,ym-y15).
A dot product and actual modulus lengths of the two vectors are calculated by the following formula:

→a·→bm=(x1-x15)⁢(xm-x15)+(y1-y15)⁢(ym-y15),
where the modulus length is a Euclidean distance: dpq=√{square root over ((xp−xq)2+(yp−yq)2)}. Assuming that dais a modulus length between the feature points 1 and 15, the actual modulus length is

d⁢1=da⁢E=da⁢E⁢1⁢h2⁢h⁢1.
With dbmas the modulus length between the feature points m and 15, the actual modulus length is

d⁢2=dbm⁢E=dbm⁢E⁢1⁢h2⁢h⁢1,
where m is selected from the feature points 2, 3, 4, 5, 6, 7, 8, 9, 14, 16, 17, 19, and 20.

The actual modulus length is

d⁢2=dbm⁢E⁢x⁢1x⁢1-x⁢2⁢sin⁢2⁢Π⁢❘"\[LeftBracketingBar]"f❘"\[RightBracketingBar]",
where m is selected from feature points 10, 11, 12, 13, and 18. Accordingly, a radian angle θ′ between the vector a and the vector bmmay be calculated, and a cosine value of the angle is as follows:

cos⁢θ′=(→a·→bm)/d⁢1×d⁢2
(d1 and d2 are the actual modulus length between the feature points 1 and 15 and the actual modulus length between other feature point m and the feature point 15, respectively). Finally, the calculated radian is converted to angle θ=arccos(cos θ′)×(180/Π), thereby obtaining an angle feature.

A concat operation is performed on the obtained distance feature and angle feature to obtain a fused multi-layer feature (i.e., a geometric feature), and the geometric feature is coupled with a texture feature.

(3) The body surface texture features of a fish are quantized using an improved ResNet network. Due to underwater shooting, the influences of water turbidity and illumination intensity on fish body surface texture are taken into consideration. A convolutional block attention module (CBAM) module is added to a ResNet model to enhance the concern extent and the extraction capability of the model to fish body texture features. CBAM is changed from reducing dimensions first to increasing dimensions first and then reducing dimensions, and “cascade connection” of a CAM and a SAM is changed to “parallel connection”, where formulas of channel attention and spatial attention are as follows:
Mc(F)=U(MLP(AvgPool(F)+MLP(MaxPool(F))=u(W1(W2(Fcavg))+W1(W0(Fcmax)
Ms(F)=U(f7×7[AvgPool(F); MaxPool(F)])=u(f7×7([FSavg;FSmax]))

A RELU activation function in the ResNet is replaced with a leaky ReLU activation function. The Leaky ReLU is an excellent variant of the ReLU. When x<0, a negative gradient value is obtained. The problem of ReLU nerve death is solved.

The CBAM module is added to a tail end of a ResNet module. More information features can be extracted through the foregoing convolution operation. An extracted feature map is put into the CBAM, allowing the network to adaptively acquire the texture information of fish individuals. An output weighted feature information matrix and an input feature information matrix are added up to reduce related feature information lost in the convolution process. Finally, the added feature map is subjected to leaky Relu non-linear activation once to obtain a final output result which is sent to a small sample learning network.

(4) Small sample learning is performed using a small sample learning model based on measurement. Concat operation is performed on the obtained surface texture feature and geometric feature (the concat operation is to directly connect two original features). Assuming that channels for two branches of inputs of the body surface texture features and the geometric features are X1, X2, . . . , XCand Y1, Y2, . . . , YC, respectively, a single output of concat is Zconcat=Σi=1cXi*Ki+Σi=1cYi*Ki+c, where Kiand Ki+crepresent convolution kernels corresponding to an ith channel and an (i+c)th channel known in the concat operation of the small sample learning network, respectively. Due to underwater shooting, in consideration of the influence of turbidity F factor, when the turbidity increases, the obtained texture definition (Q) decreases, and the following formula can be established:

Q=−(½F+⅓). Then, the final output is as follows:
Zconcat=QΣi=1cXi*Ki+Σi=1cYi*Ki+c=Zconcat=−(½F+⅓)Σi=1cXi*Ki+Σi=1cYi*Ki+c.

The obtained fused multi-layer feature output is used as a model input, and then the model input is mapped to an embedded space through a network model for fish identity recognition, and a final result is obtained, as shown inFIG.3.

Application Case:

Fish identification testing is conducted on industrially farmedAcrossocheilus fasciatus(1000 individuals) and spotted maigre (800 individuals) using the experimental method disclosed herein, and comparison is made with other network models to verify the effectiveness of the method. Identification results are as shown in the table.

Accuracy rateGrouperSpotted maigreConvNeXt72.1369.06AlexNet68.2462.12VGG1679.5578.65Method of the present90.788.27disclosure

It can be seen that compared with some existing detection networks, the method of the present disclosure can identify individuals of the same fish more accurately, and has good effects for different fishes.

Moreover, a person skilled in the art should understand that the embodiments of the present disclosure may be provided as a method, a system, or a computer program product. Therefore, the present disclosure may use a form of hardware only embodiments, software only embodiments, or embodiments with a combination of software and hardware. Furthermore, the present disclosure may adopt a form of a computer program product that is implemented on one or more computer-usable storage media (including but not limited to a disk memory, compact disc read-only memory (CD-ROM) and an optical memory) that include computer program codes.

The present disclosure is described with reference to the flowcharts and/or block diagrams of the method, the device (system), and the computer program product according to the embodiments of the present disclosure. It should be understood that computer program instructions may be used to implement each process and/or each block in the flowcharts and/or the block diagrams and a combination of a process and/or a block in the flowcharts and/or the block diagrams. These computer program instructions may be provided for a general-purpose computer, a dedicated computer, an embedded processor, or a processor of any other programmable data processing device to generate a machine, so that the instructions executed by a computer or a processor of any other programmable data processing device generate an apparatus for implementing a specific function in one or more processes in the flowcharts and/or in one or more blocks in the block diagrams.

These computer program instructions may be stored in a computer readable memory that can instruct a computer or another programmable data processing device to work in a specific manner, such that the instructions stored in the computer readable memory generate an artifact that includes an instruction apparatus. The instruction apparatus implements a specific function in one or more processes in the flowcharts and/or in one or more blocks in the block diagrams.

These computer program instructions may alternatively be loaded onto a computer or another programmable data processing device, such that a series of operations and steps are performed on the computer or the another programmable device, thereby generating computer-implemented processing. Therefore, the instructions executed on the computer or the another programmable device provide steps for implementing a specific function in one or more processes in the flowcharts and/or in one or more blocks in the block diagrams.

The above described are merely preferred embodiments of the present disclosure, and are not intended to limit the present disclosure. Various changes and variations can be made by those of ordinary skill in the related technical field without departing from the spirit and scope of the present disclosure. All technical solutions obtained by means of equivalent replacements or equivalent variations should fall within the protection scope of the present disclosure.