# Measurement of Fish Morphological Features through Image Processing and Deep Learning Techniques
## 2 Materials and Methods  
### Fish Image Dataset  
Four fish species that are popular in the Mediterranean diet were studied: _Dicentrarchus labrax_ (sea bass), _Diplodus puntazzo_, _Merluccius merluccius_ (cod fish) and _Sparus aurata_ (sea bream). Except for the cod fish, the rest of the species are also cultured in fish farms. ImageNet [25] and other popular databases contain only a small number of fish images. For example, ImageNet contains _whales, devilfish, anemonefish_, _lionfish_, _blowfish_ and _garfish_, but none of these species are studied in this paper. There are several other fish image datasets that can be found in [26], but (1) there are not many photographs available for the specific species and (2) they are mainly photographs taken from shore. In this work, we focus on profile fish photographs taken from the side of the fish. Alternative ways to reassure that a fish is captured when its side is parallel with the camera image plane are the following: (1) trap the fish in a fishway where it will not have many degrees of freedom for a while, (2) examine a number of consecutive video frames and select the one where the fish has its maximum dimensions and (3) apply affine coordinate transformations to rotate appropriately a slanted fish contour before measuring its dimensions [7]. In the affine transformation between two planes, a point \(p^{(1)}\) in the first plane with coordinates \(\left[p_{x}^{(1)},p_{y}^{(1)}\right]\) was mapped to \(p^{(2)}=\left[p_{x}^{(2)},p_{y}^{(2)}\right]\) in the second plane. A rotation matrix \(\left[ \begin{array}{cc}r_{a}&r_{b}\\ -r_{b}&r_{a}\end{array} \right]\) multiplied the coordinates of \(p^{(1)}\), and an offset was potentially added to get \(p^{(2)}\). The \(r_{a}\), \(r_{b}\) and offset values that maximized the fish contour dimensions could be selected, since a slanted or rotated fish seemed to have smaller dimensions than a fish in the expected position. Since we generally did not know the distance of the fish, only relative distances and areas could be estimated. However, if the fish was captured when it passed a narrow fishway [5], or if two photographs were taken from different cameras, the absolute dimensions and fish mass could also be estimated, as will be explained in Section 2.4. The contribution of light refraction was negligible, but correction methods could also be applied to improve the accuracy. From the four fish families that were examined, the cod fish and sea bass had long bodies with small heights, while the sea bream and _Diplodus puntazzo_ had shorter lengths and bigger heights. The dataset used consisted of 25 photographs from each species. Many of these photographs were captured underwater. In the deep learning approach, the dataset was augmented using mirror images, jitter and random crop. In this way, a final training dataset that consisted of 4000 images was produced. Three quarters of them displayed fish, and the rest of them displayed a small part of the fish or background. Similar to [20], the background could be as simple as seawater only, or it could contain more complex structures and objects. The size of the original images was 80 \(\times\) 160, and this size was used in the image processing approach. The photographs were resized to 224 \(\times\) 224 pixels in the deep learning approach to comply with the pretrained model that would be used, as will be explained in the following paragraphs. The photographs that populated our dataset were retrieved from [26], and others were publically available on the Internet. An initial dataset of 25 photographs per species is not adequately large for deep learning techniques. However, it is interesting to measure the accuracy that can be achieved in this case, since future versions of our system may also have to be trained with a small number of photographs to support a new species. In the image processing approach, three representative shape patterns were used for each species in order to apply pattern matching.