# Intelligent Fish Freshness Assessment

Authors: Hamid Gholam Hosseini; Dehan Luo; Guanggui Xu; Hongxiu Liu; Deena Benjamin
Journal: Journal of Sensors
 (2008)

---

## Abstract

Fish species identification and automated fish freshness assessment play important roles in fishery industry applications. This paper describes a method based on support vector machines (SVMs) to improve the performance of fish identification systems. The result is used for the assessment of fish freshness using artificial neural network (ANN). Identification of the fish species involves processing of the images of fish. The most efficient features were extracted and combined with the down-sampled version of the images to create a 1D input vector. Max-Win algorithm applied to the SVM-based classifiers has enhanced the reliability of sorting to 96.46%. The realisation of Cyranose 320 Electronic nose (E-nose), in order to evaluate the fish freshness in real-time, is experimented. Intelligent processing of the sensor patterns involves the use of a dedicated ANN for each species under study. The best estimation of freshness was provided by the most sensitive sensors. Data was collected from four selected species of fishes over a period of ten days. It was concluded that the performance can be increased using individual trained ANN for each specie. The proposed system has been successful in identifying the number of days after catching the fish with an accuracy of up to  91%.


---

## Body

## 1. Introduction

Automatic  fish sorting by  species is an important process for many fishery applications
such as freshness assessment, marine ecology issues, and automate logging of
the catch in commercial and research fishing vessels. In this work, fish
identification is done with the aim of improving the accuracy of freshness
assessment system. Traditionally, the patterns of fish skin as well as the
whole shape have been used by fishery researchers to identify the fish.
However, the lighting conditions, freshness of the fish, and changes in skin
colour influence the fish type identification. This makes it difficult to
manually classify the fish species and correctly interpret the findings.The
use of fish image as inputs for the training system was not effective as
neither memory usage is optimum nor fish type is easily recognizable. Fish skin
images could be incorporated in the proposed fish identification system as a
supplement and not as the necessary object identification input. Fish type
identification based on fish eye images and useful parameters extracted from
these images is the methodology adopted in the designed system.Support vector
machine (SVM) has proven to be an
effective tool for pattern recognition  [1]. The typical method for applying SVM to multiclass problems (N classes) is to
construct N number of binary-SVM classifiers, each of which identifies one class among N different classes.
These types of SVMs are referred to as 1-v-r (short for one-versus-rest).
Another typical method is to combine all possible binary (two-class)
classifiers. In this case, for N-class problems, N(N-1)/2 classifiers should be
trained. These types of SVMs are referred to as 1-v-1 (short for one-versus-one)
[2]. We adopted 1-v-1 technique for this project to identify the fish species
due to its superior accuracy.Upon
identification the species of the fish, its freshness, is to be assessed. Fish
quality is a complex concept involving a whole range of factors, freshness
being one of the most important. It indicates the degree of various physical,
chemical, biochemical and microbiological changes in the fish. Of the various
sensory and instrumental methods adopted to ascertain the freshness of fishes,
E-nose-based methods continue to be the primary choice [3]. An E-nose system
comprises of a sensor array, an interfacing electronic circuitry, a sniffing
mechanism and a pattern-recognition unit that acts as an odour classifier.The odour
that emanates from the fish when detected by the E-nose generates a
characteristic pattern or sensor print. Collecting the response from each
sensor, as they convert the chemical reaction into an electrical signal,
results in smell prints. The type of odour and the degree of selectivity of
odour depend on the choice and number of sensors in the sensor array. The data
processing and pattern recognition techniques can be applied to the sensor
signals to differentiate substances or train a system to provide identification
based on the collection of known responses.E-nose has
been used for fish freshness monitoring and odour evaluation [4]. Smell detection
techniques and other approaches to sensor electronics such as feature
extraction and data processing techniques have been employed in freshness
assessment of fishery products  [5]. Various methodologies used to ascertain the
freshness of the fish are briefly outlined below.A European
project that used two types of E-noses based on different sampling procedures
and sensor technologies investigated the possibility of developing a multisensor
device using a combination of E-noses, spectroscopic, texture-meters, image
analysers, colour meters, and measuring the electrical properties to measure
and estimate fish freshness [6]. Metallic potentiometric electrodes were
employed as sensors to ascertain the freshness of fish. These electrodes were
found to be sensible to various unspecific elements and hence multivariate
analysis techniques were used to draw conclusions from the experiment [7].
E-nose systems have been used to determine the aroma and flavour of tea and
spices [8] and identify typical wine aromas [9] and cigarette brands [10].
In this paper,  we are presenting the use of E-nose to other more complex
mediums in the determination of fish freshness.


## 2. Methodology

The block
diagram shown in Figure1 explicitly depicts two processing components of the
system. The SVM-based method
processes the images of fish eye for identification purpose and the ANN-based method processes the smell prints for fish
freshness assessment. Moreover, the information of the SVM classifier is used to enhance the performance of the ANN-based
technique as explained below in detail.Figure 1
Fish species identification and freshness assessment system.


### 2.1. Species Identification

SVM is a supervised learning technique that can be applied to classification
or regression [11]. SVM algorithms are based on the statistical
learning theory and the Vapnik-Chervonenkis (VC)
dimension [12].These algorithms try to find a
unique hyperplane with maximum margin in the SVM models to separate different
classes. SVM can also be used in classifying complicated models by employing
nonlinear kernels.  The role of kernel functions is to perform computations in
the original input space rather than the high-dimensional (even infinite)
feature space [11]. Because only the inner product is involved in SVM, learning
and predicting is much faster than a multilayer neural network [5]. Figure 2
shows the structure of an SVM with a test pattern of   xt  and the support
vectors   x1,…,xn which are mapped into a feature
space with the nonlinear function ϕ(x),  and the
computed dot products [11].Figure 2
Structure of an SVM  [12].SVM has a
faster training time among other statistical training methods and utilizes less
memory space. However, in image classification applications the use of raw
images as inputs for training SVM can result in the utilization of a large
memory space and consequently a long training time. Hence, in this study, we
used features of the fish eye and fish skin images as part of training SVM
instead of the raw images.Optimum
separation hyperplane (OSH) is the hyperplane with the maximum margin for a
given finite set of learning patterns [11]. Linear SVM performs binary pattern
classification by finding OSH function f(x)=wTx−b. The
parameters  w and b are determined by learning from
examples. wT is the transpose matrix of w.
Suppose that linearly separable data {xi}   and their class labels {yi} (yi∈{−1,+1}) are given. Among possible linear discriminant
functions that classify the given data without errors, the classifier function
is aimed to find an OSH as the separable surface to separate two classes of
patterns with maximal margin (the maximal distance between the surface and the
nearest data point of each class) [5]. These nearest points to the hyperplane
are called support vectors. Resultant hyperplane is called OSH. The optimal
weight vector, w,
is defined by (1)w=∑i=1lαiyixi. Therefore, the equation of the hyperplane separating two
different classes is given by (2)f(x)=∑i=1lαiyixi⋅x−b. The coefficients {αi} are determined by solving a quadratic
programming problem. αi is nonzero only for support vectors. This
means, the parameter w of a linear SVM is given by linear combination of
support vectors.The above learning method can be extended to the case of nonlinear
classifiers.LetΦ:X→F be a nonlinear map from the input space X to some feature space F . We can construct a linear classifier in the
feature space so that the margin is maximized as we did in the input space. The
discriminant function is given by (3)f(x)=∑i=1l〈αiyiφ(xi),φ(x)〉F−b, where 〈⋅,⋅〉F. indicates inner product in the feature space F.To learn nonlinear SVM, we do not need to explicitly specify
the nonlinear mappingϕ.
What we need is a “kernel function”  k  (x,   y) which gives inner
product in the feature space: (4)k(x,y)=〈φ(x),φ(y)〉F. By introducing a kernel function, a feature space is
implicitly specified.The decision function for an input vectorx is defined
as (5)f(x)=∑i∈SVαiyik(x,xi)−b, where xi(i∈SV) denotes a learning vector, and SV denotes the set of the support vectors. k (a,  b) is the kernel
function. The classification result is determined by the sign of the function f(x).The parameters{αi} are obtained by solving the following
quadratic programming problem:(6)Maxmine:∑i=1lαi−12∑i=1l∑j=1lαiαjk(xi,xj)yiyj.Subject to:∑i=1lαiyi=0,0≤αi≤C,C is a
parameter which penalizes the learning errors.SVM’s learning method is
strongly related to structural risk minimization [12], and thus SVM can be
expected to have good generalization power.Max-Win [13]
algorithm is probably the most typical 1-v-1 method for multiclass recognition.
The total number of classifiers is N(N−1)/2  for an N-class problem [14]. As
the recognition method using 1-v-1 for a multiclass problem, Max-Win algorithm
is used. In Max-Win algorithm, each SVM casts one vote and the recognition
result is the class with the maximum number of votes.An input
pattern is given to all pairwise classifiers. Receiving the input, each SVM
determines the class to which the input pattern belongs. It is considered as a
“vote”. We can use the number of “votes” for each class as the measure of
confidence. The final answer is the class which acquires the maximum number of
votes. The recognition time is proportional toN(N−1)/2, which is the number
of the SVMs. Usually, the performance of the Max-Win algorithm is very good.
However, its evaluation time which relates to the number of operations, O(N2),
is very slow.A new
strategy is used in the design of a two-level classifier. It employs voting
mechanism tovote but not make the final prediction of all classifiers
of the first level which is called level-0. The relation between votes and the
true classes is  induced by classification  algorithm
of the second level named level-1. Figure 3 shows the block diagram of this
classifier.Figure 3
The explanation of two-level classifiers for multiclass.In Figure3,
there are n=N(N−1)/2 SVM classifiers in level-0 which indicates that the
input pattern is evaluated by each 1-v-1 SVM (i versus j) situated in this
level. Any SVM that classified the input as an ith class, it is
considered as “not-j”. The output of each classifier of this level will be
considered as an input for level-1, which is used to label the class of the
input.The procedure
can be summarized as follows:(i)
apply the
testing data as input to all SVM classifiers in level-0 in parallel,(ii)
use the
prediction of all SVM classifiers in level-0 as input data for level-1
classifiers, and(iii)
use MAX-Win
algorithm to determine the output of level-1 as the identification result.
The
experiments were carried out using a database of 232 images of fish eyes from
four different fish types. The samples of fish eye images which were used in
this research are shown in Figure 4.Fish eye images of four species under study.
(a)(b)(c)(d)All images were taken under a uniform lighting condition. Other
photographical parameters such as the position of fish related to the camera
were also fixed during the image acquisition process in order to reduce the
environmental-related errors.Use of the
most efficient features extracted from the fish eyes and fish skin, in addition
to the downsampled version of the selected images, resulted in reduced training
time and improved performance in fish species identification. Selecting an
optimum image size ensures reduced computation cost and faster response while
training and testing the SVM. However, downsizing the image can lead to
compromising the amount of information provided and increase the errors in
identification. Hence, a100×100 pixel image size was chosen as ideal for the
proposed system. A  100×100 pixel image was later downsized to a 400×1 image
during down sampling. The 400 fish eye image samples of each fish type were
selected and concatenated with the 4 extracted features from the same image in
order to generate a data set for SVM.The
compressed part of each image was used as one part of the input to the SVM. The
second part of the input vector is 4 feature parameters extracted from the fish
eye image. They are statistical features including themean,  max,  std,  var of the image.Therefore,
instead of using raw fish image, we used 404 samples (400 from the compressed
image and 4 extracted parameters) to minimize the structural complexity of the
SVM and reduce the training time.An SVM
implementation program called LSSVM lab  [15] toolbox was used in this work.
Least squares support vector machines (LSs-SVMs) [16, 17] are reformulations to
standard SVM. We trained six SVM classifiers, each of which for classifying two
classes.The quality
of SVM for classification depends on the combination of several parameters:
capacity parameterC, the kernel type K, and its corresponding parameters γ.
Also, there are no clear guidelines for selecting the optimum set of
theoretical parameters. Therefore, the only practical way of finding an
optimally predictive SVM model is through extensive experiments. It is well
known that the results of SVM approach lie largely on the choice of a kernel,
which determines the sample distribution in the mapping space. Therefore, the
kernel functions should be decided first. In this work, we used the MLP
(multilayer perceptron) kernel as well as linear kernel. We chose the optimal
value of penalty coefficient C to be 10, and the parameter  γ equal to 0.2
empirically.


### 2.2. Freshness Assessment

A
portable Cyranose 320 E-nose comprising an array of thirty-two polymer carbon
black composite sensors was used in our experiments. The conductivity of these
sensors changes resulting in an increase in resistance value when they are
exposed to vapours or stimulus aroma. The variations in resistance are recorded
as sensor outputs across the array, in the form of a digital pattern. These
patterns are distinct and unique for different vapours [18]. Possession of a
built-in neural network that can be trained to identify specific stimuli and
the generation of stimuli specific smell prints are the factors that make E-nose
as the preferred choice over other commercially available products.Four
species were selected: Red Snapper, Gurnard, Tarakihi, and Trevally. The minced
flesh of the selected fishes stored in the refrigerator. A fixed amount of meat
of each fish was taken from the fridge and exposed to room temperature prior to
each experiment in order to bring the sample to the room temperature. These
samples were placed into four sterile glasses (one glass for each fish) in a
temperature-controlled environment. Readings were taken from the headspace of
the samples by manually introducing the E-nose system. For the purpose of
evaluation, the position of the E-nose sensor on the sample is fixed. The
baseline purge time, sampling time, and purge time of E-nose were accurately
set for each experiment to avoid error. Time allowed for the E-nose to record a
complete smell print was 30 seconds of sampling time. Sufficient time was
allowed for each data collection for a completed sample smell print and for
cleaning the chamber. Different samples of the same fish were measured everyday
(except days 3 and 4) over ten-day period. The smell prints generated from
these measurements were used to ascertain the fish freshness. The proposed
method may be used in other countries for the identification of fish species by
training the data sets to be selected accordingly.


#### 2.2.1. Experimental Data

The process
of converting the odour into sensor patterns results in the generation of a
time series of thousands of resistance values for each sensor. We converted the
odour of four selected types of fish to smell prints over days 1, 2, 5, 6, 7,
8, 9, and 10 after  catching the fish (no data was collected on days 3 and 4).
With a sampling interval of 30 seconds, approximately 2000 samples were
collected by each sensor during each process. Therefore, about 2,048,000 data
samples [4 (fish)×  8 (days) × 32 (sensors) × 2000 (samples) = 2,048,000]  were
obtained. Obviously, processing this amount of raw data for classification
purposes requires a complex classifier. Figure 5 shows the plot of 666 samples
of the smell print response by sensor 1 after exposing to the red Snapper fish.Figure 5
Part of the
smell print response by senor one exposed to red Snapper.Figure6
shows the relative responses of all 32 sensors for the selected fish
(Tarakihi).Figure 6
The relative distance between the smell print responses over two consecutive
days.The relative
value was calculated, based on the difference between the smell-prints of two
consecutive days (e.g.,  between day 7 and 8). It can be observed that some
sensors show better responses compared to others. This is due to the fact that
E-nose sensors are manufactured in such a way to respond differently to
different odours, in the present case Sensors 3, 4, and 6 responded favourably
to the fish smells. However, some sensors (e.g., sensor 23) with good responses
were eliminated due to their instability responses to different fishes.
Therefore, for training and testing the neural network classifiers, we selected
the smell-print of sensors 3, 4, and 6 only.In order to
reduce the size of the data and to further simplify the classifier, we decided
to evaluate the assessment test only for 4 days (day 2, 5, 7, and 8). Thus, the
size of the initial data was reduced to 96,000 samples: [4 (fish)× 4 (days) × 3 (sensors)
× 2000 (samples) =  96,000].Moreover, it
was found that each smell print of 2000 samples consisted of approximately 8
similar pulses with a pulse width of less than 250 samples. To achieve a better
data reduction factor, we applied a window with the width of 200 samples to
each smell print. The window selected a pulse starting from its rising edge in
the selected smell print. This method enabled the time series of each smell
print to be fixed to a 200-sample interval and the size of the initial data
reduced to 9 600 samples:(7)[4(fish)×4(days)×3(sensors)×200(samples)=9,600].


#### 2.2.2. Parameter Extraction

Extraction of
few numerical features that can robustly describe the smell print from the raw
data resulted in the reduction in the size of data set used. In this context,
the number of samples to describe a single sensor response was reduced from 200
samples to 50 samples as discussed in [19].The most
efficient features were extracted from the selected pulse as well as its
down-sampled version. Hence the input vector of each classifier comprised of
two components. A fixed 200-sample interval initiated at the rising edge of a
pulse in the smell-print was selected and down-sampled by a factor of 5 to
generate the 40-sample compressed part, which is the first component of input
to the neural network architecture. The second part of the input vector
included 10 feature parameters extracted at the same 200-sample interval. These
statistical features are: themedian, minimum (min), maximum (max), s tandard deviation (std), variance (var), median,
mean/median absolute deviation (mad), the geometry mean (geomean), standard
deviation (std_m), and variance (var_m) of the mean-removed data .We used 50
components (10 features plus 40 compressed components) instead of using 200
samples of raw smell prints. The reduction in the size of data enabled the
reduction of network complexity and improved the performance of data
classification.


## 2.1. Species Identification

SVM is a supervised learning technique that can be applied to classification
or regression [11]. SVM algorithms are based on the statistical
learning theory and the Vapnik-Chervonenkis (VC)
dimension [12].These algorithms try to find a
unique hyperplane with maximum margin in the SVM models to separate different
classes. SVM can also be used in classifying complicated models by employing
nonlinear kernels.  The role of kernel functions is to perform computations in
the original input space rather than the high-dimensional (even infinite)
feature space [11]. Because only the inner product is involved in SVM, learning
and predicting is much faster than a multilayer neural network [5]. Figure 2
shows the structure of an SVM with a test pattern of   xt  and the support
vectors   x1,…,xn which are mapped into a feature
space with the nonlinear function ϕ(x),  and the
computed dot products [11].Figure 2
Structure of an SVM  [12].SVM has a
faster training time among other statistical training methods and utilizes less
memory space. However, in image classification applications the use of raw
images as inputs for training SVM can result in the utilization of a large
memory space and consequently a long training time. Hence, in this study, we
used features of the fish eye and fish skin images as part of training SVM
instead of the raw images.Optimum
separation hyperplane (OSH) is the hyperplane with the maximum margin for a
given finite set of learning patterns [11]. Linear SVM performs binary pattern
classification by finding OSH function f(x)=wTx−b. The
parameters  w and b are determined by learning from
examples. wT is the transpose matrix of w.
Suppose that linearly separable data {xi}   and their class labels {yi} (yi∈{−1,+1}) are given. Among possible linear discriminant
functions that classify the given data without errors, the classifier function
is aimed to find an OSH as the separable surface to separate two classes of
patterns with maximal margin (the maximal distance between the surface and the
nearest data point of each class) [5]. These nearest points to the hyperplane
are called support vectors. Resultant hyperplane is called OSH. The optimal
weight vector, w,
is defined by (1)w=∑i=1lαiyixi. Therefore, the equation of the hyperplane separating two
different classes is given by (2)f(x)=∑i=1lαiyixi⋅x−b. The coefficients {αi} are determined by solving a quadratic
programming problem. αi is nonzero only for support vectors. This
means, the parameter w of a linear SVM is given by linear combination of
support vectors.The above learning method can be extended to the case of nonlinear
classifiers.LetΦ:X→F be a nonlinear map from the input space X to some feature space F . We can construct a linear classifier in the
feature space so that the margin is maximized as we did in the input space. The
discriminant function is given by (3)f(x)=∑i=1l〈αiyiφ(xi),φ(x)〉F−b, where 〈⋅,⋅〉F. indicates inner product in the feature space F.To learn nonlinear SVM, we do not need to explicitly specify
the nonlinear mappingϕ.
What we need is a “kernel function”  k  (x,   y) which gives inner
product in the feature space: (4)k(x,y)=〈φ(x),φ(y)〉F. By introducing a kernel function, a feature space is
implicitly specified.The decision function for an input vectorx is defined
as (5)f(x)=∑i∈SVαiyik(x,xi)−b, where xi(i∈SV) denotes a learning vector, and SV denotes the set of the support vectors. k (a,  b) is the kernel
function. The classification result is determined by the sign of the function f(x).The parameters{αi} are obtained by solving the following
quadratic programming problem:(6)Maxmine:∑i=1lαi−12∑i=1l∑j=1lαiαjk(xi,xj)yiyj.Subject to:∑i=1lαiyi=0,0≤αi≤C,C is a
parameter which penalizes the learning errors.SVM’s learning method is
strongly related to structural risk minimization [12], and thus SVM can be
expected to have good generalization power.Max-Win [13]
algorithm is probably the most typical 1-v-1 method for multiclass recognition.
The total number of classifiers is N(N−1)/2  for an N-class problem [14]. As
the recognition method using 1-v-1 for a multiclass problem, Max-Win algorithm
is used. In Max-Win algorithm, each SVM casts one vote and the recognition
result is the class with the maximum number of votes.An input
pattern is given to all pairwise classifiers. Receiving the input, each SVM
determines the class to which the input pattern belongs. It is considered as a
“vote”. We can use the number of “votes” for each class as the measure of
confidence. The final answer is the class which acquires the maximum number of
votes. The recognition time is proportional toN(N−1)/2, which is the number
of the SVMs. Usually, the performance of the Max-Win algorithm is very good.
However, its evaluation time which relates to the number of operations, O(N2),
is very slow.A new
strategy is used in the design of a two-level classifier. It employs voting
mechanism tovote but not make the final prediction of all classifiers
of the first level which is called level-0. The relation between votes and the
true classes is  induced by classification  algorithm
of the second level named level-1. Figure 3 shows the block diagram of this
classifier.Figure 3
The explanation of two-level classifiers for multiclass.In Figure3,
there are n=N(N−1)/2 SVM classifiers in level-0 which indicates that the
input pattern is evaluated by each 1-v-1 SVM (i versus j) situated in this
level. Any SVM that classified the input as an ith class, it is
considered as “not-j”. The output of each classifier of this level will be
considered as an input for level-1, which is used to label the class of the
input.The procedure
can be summarized as follows:(i)
apply the
testing data as input to all SVM classifiers in level-0 in parallel,(ii)
use the
prediction of all SVM classifiers in level-0 as input data for level-1
classifiers, and(iii)
use MAX-Win
algorithm to determine the output of level-1 as the identification result.
The
experiments were carried out using a database of 232 images of fish eyes from
four different fish types. The samples of fish eye images which were used in
this research are shown in Figure 4.Fish eye images of four species under study.
(a)(b)(c)(d)All images were taken under a uniform lighting condition. Other
photographical parameters such as the position of fish related to the camera
were also fixed during the image acquisition process in order to reduce the
environmental-related errors.Use of the
most efficient features extracted from the fish eyes and fish skin, in addition
to the downsampled version of the selected images, resulted in reduced training
time and improved performance in fish species identification. Selecting an
optimum image size ensures reduced computation cost and faster response while
training and testing the SVM. However, downsizing the image can lead to
compromising the amount of information provided and increase the errors in
identification. Hence, a100×100 pixel image size was chosen as ideal for the
proposed system. A  100×100 pixel image was later downsized to a 400×1 image
during down sampling. The 400 fish eye image samples of each fish type were
selected and concatenated with the 4 extracted features from the same image in
order to generate a data set for SVM.The
compressed part of each image was used as one part of the input to the SVM. The
second part of the input vector is 4 feature parameters extracted from the fish
eye image. They are statistical features including themean,  max,  std,  var of the image.Therefore,
instead of using raw fish image, we used 404 samples (400 from the compressed
image and 4 extracted parameters) to minimize the structural complexity of the
SVM and reduce the training time.An SVM
implementation program called LSSVM lab  [15] toolbox was used in this work.
Least squares support vector machines (LSs-SVMs) [16, 17] are reformulations to
standard SVM. We trained six SVM classifiers, each of which for classifying two
classes.The quality
of SVM for classification depends on the combination of several parameters:
capacity parameterC, the kernel type K, and its corresponding parameters γ.
Also, there are no clear guidelines for selecting the optimum set of
theoretical parameters. Therefore, the only practical way of finding an
optimally predictive SVM model is through extensive experiments. It is well
known that the results of SVM approach lie largely on the choice of a kernel,
which determines the sample distribution in the mapping space. Therefore, the
kernel functions should be decided first. In this work, we used the MLP
(multilayer perceptron) kernel as well as linear kernel. We chose the optimal
value of penalty coefficient C to be 10, and the parameter  γ equal to 0.2
empirically.


## 2.2. Freshness Assessment

A
portable Cyranose 320 E-nose comprising an array of thirty-two polymer carbon
black composite sensors was used in our experiments. The conductivity of these
sensors changes resulting in an increase in resistance value when they are
exposed to vapours or stimulus aroma. The variations in resistance are recorded
as sensor outputs across the array, in the form of a digital pattern. These
patterns are distinct and unique for different vapours [18]. Possession of a
built-in neural network that can be trained to identify specific stimuli and
the generation of stimuli specific smell prints are the factors that make E-nose
as the preferred choice over other commercially available products.Four
species were selected: Red Snapper, Gurnard, Tarakihi, and Trevally. The minced
flesh of the selected fishes stored in the refrigerator. A fixed amount of meat
of each fish was taken from the fridge and exposed to room temperature prior to
each experiment in order to bring the sample to the room temperature. These
samples were placed into four sterile glasses (one glass for each fish) in a
temperature-controlled environment. Readings were taken from the headspace of
the samples by manually introducing the E-nose system. For the purpose of
evaluation, the position of the E-nose sensor on the sample is fixed. The
baseline purge time, sampling time, and purge time of E-nose were accurately
set for each experiment to avoid error. Time allowed for the E-nose to record a
complete smell print was 30 seconds of sampling time. Sufficient time was
allowed for each data collection for a completed sample smell print and for
cleaning the chamber. Different samples of the same fish were measured everyday
(except days 3 and 4) over ten-day period. The smell prints generated from
these measurements were used to ascertain the fish freshness. The proposed
method may be used in other countries for the identification of fish species by
training the data sets to be selected accordingly.


### 2.2.1. Experimental Data

The process
of converting the odour into sensor patterns results in the generation of a
time series of thousands of resistance values for each sensor. We converted the
odour of four selected types of fish to smell prints over days 1, 2, 5, 6, 7,
8, 9, and 10 after  catching the fish (no data was collected on days 3 and 4).
With a sampling interval of 30 seconds, approximately 2000 samples were
collected by each sensor during each process. Therefore, about 2,048,000 data
samples [4 (fish)×  8 (days) × 32 (sensors) × 2000 (samples) = 2,048,000]  were
obtained. Obviously, processing this amount of raw data for classification
purposes requires a complex classifier. Figure 5 shows the plot of 666 samples
of the smell print response by sensor 1 after exposing to the red Snapper fish.Figure 5
Part of the
smell print response by senor one exposed to red Snapper.Figure6
shows the relative responses of all 32 sensors for the selected fish
(Tarakihi).Figure 6
The relative distance between the smell print responses over two consecutive
days.The relative
value was calculated, based on the difference between the smell-prints of two
consecutive days (e.g.,  between day 7 and 8). It can be observed that some
sensors show better responses compared to others. This is due to the fact that
E-nose sensors are manufactured in such a way to respond differently to
different odours, in the present case Sensors 3, 4, and 6 responded favourably
to the fish smells. However, some sensors (e.g., sensor 23) with good responses
were eliminated due to their instability responses to different fishes.
Therefore, for training and testing the neural network classifiers, we selected
the smell-print of sensors 3, 4, and 6 only.In order to
reduce the size of the data and to further simplify the classifier, we decided
to evaluate the assessment test only for 4 days (day 2, 5, 7, and 8). Thus, the
size of the initial data was reduced to 96,000 samples: [4 (fish)× 4 (days) × 3 (sensors)
× 2000 (samples) =  96,000].Moreover, it
was found that each smell print of 2000 samples consisted of approximately 8
similar pulses with a pulse width of less than 250 samples. To achieve a better
data reduction factor, we applied a window with the width of 200 samples to
each smell print. The window selected a pulse starting from its rising edge in
the selected smell print. This method enabled the time series of each smell
print to be fixed to a 200-sample interval and the size of the initial data
reduced to 9 600 samples:(7)[4(fish)×4(days)×3(sensors)×200(samples)=9,600].


### 2.2.2. Parameter Extraction

Extraction of
few numerical features that can robustly describe the smell print from the raw
data resulted in the reduction in the size of data set used. In this context,
the number of samples to describe a single sensor response was reduced from 200
samples to 50 samples as discussed in [19].The most
efficient features were extracted from the selected pulse as well as its
down-sampled version. Hence the input vector of each classifier comprised of
two components. A fixed 200-sample interval initiated at the rising edge of a
pulse in the smell-print was selected and down-sampled by a factor of 5 to
generate the 40-sample compressed part, which is the first component of input
to the neural network architecture. The second part of the input vector
included 10 feature parameters extracted at the same 200-sample interval. These
statistical features are: themedian, minimum (min), maximum (max), s tandard deviation (std), variance (var), median,
mean/median absolute deviation (mad), the geometry mean (geomean), standard
deviation (std_m), and variance (var_m) of the mean-removed data .We used 50
components (10 features plus 40 compressed components) instead of using 200
samples of raw smell prints. The reduction in the size of data enabled the
reduction of network complexity and improved the performance of data
classification.


## 2.2.1. Experimental Data

The process
of converting the odour into sensor patterns results in the generation of a
time series of thousands of resistance values for each sensor. We converted the
odour of four selected types of fish to smell prints over days 1, 2, 5, 6, 7,
8, 9, and 10 after  catching the fish (no data was collected on days 3 and 4).
With a sampling interval of 30 seconds, approximately 2000 samples were
collected by each sensor during each process. Therefore, about 2,048,000 data
samples [4 (fish)×  8 (days) × 32 (sensors) × 2000 (samples) = 2,048,000]  were
obtained. Obviously, processing this amount of raw data for classification
purposes requires a complex classifier. Figure 5 shows the plot of 666 samples
of the smell print response by sensor 1 after exposing to the red Snapper fish.Figure 5
Part of the
smell print response by senor one exposed to red Snapper.Figure6
shows the relative responses of all 32 sensors for the selected fish
(Tarakihi).Figure 6
The relative distance between the smell print responses over two consecutive
days.The relative
value was calculated, based on the difference between the smell-prints of two
consecutive days (e.g.,  between day 7 and 8). It can be observed that some
sensors show better responses compared to others. This is due to the fact that
E-nose sensors are manufactured in such a way to respond differently to
different odours, in the present case Sensors 3, 4, and 6 responded favourably
to the fish smells. However, some sensors (e.g., sensor 23) with good responses
were eliminated due to their instability responses to different fishes.
Therefore, for training and testing the neural network classifiers, we selected
the smell-print of sensors 3, 4, and 6 only.In order to
reduce the size of the data and to further simplify the classifier, we decided
to evaluate the assessment test only for 4 days (day 2, 5, 7, and 8). Thus, the
size of the initial data was reduced to 96,000 samples: [4 (fish)× 4 (days) × 3 (sensors)
× 2000 (samples) =  96,000].Moreover, it
was found that each smell print of 2000 samples consisted of approximately 8
similar pulses with a pulse width of less than 250 samples. To achieve a better
data reduction factor, we applied a window with the width of 200 samples to
each smell print. The window selected a pulse starting from its rising edge in
the selected smell print. This method enabled the time series of each smell
print to be fixed to a 200-sample interval and the size of the initial data
reduced to 9 600 samples:(7)[4(fish)×4(days)×3(sensors)×200(samples)=9,600].


## 2.2.2. Parameter Extraction

Extraction of
few numerical features that can robustly describe the smell print from the raw
data resulted in the reduction in the size of data set used. In this context,
the number of samples to describe a single sensor response was reduced from 200
samples to 50 samples as discussed in [19].The most
efficient features were extracted from the selected pulse as well as its
down-sampled version. Hence the input vector of each classifier comprised of
two components. A fixed 200-sample interval initiated at the rising edge of a
pulse in the smell-print was selected and down-sampled by a factor of 5 to
generate the 40-sample compressed part, which is the first component of input
to the neural network architecture. The second part of the input vector
included 10 feature parameters extracted at the same 200-sample interval. These
statistical features are: themedian, minimum (min), maximum (max), s tandard deviation (std), variance (var), median,
mean/median absolute deviation (mad), the geometry mean (geomean), standard
deviation (std_m), and variance (var_m) of the mean-removed data .We used 50
components (10 features plus 40 compressed components) instead of using 200
samples of raw smell prints. The reduction in the size of data enabled the
reduction of network complexity and improved the performance of data
classification.


## 3. Neural Network Classifiers

The inherent
massive parallel distributed architectures in artificial neural networks are
employed for information processing and signal classification of different
systems. They are characterized as computational models and consist of a
collection of processing elements called neurodes (neural nodes), and weighted
connections (synapses) [10].  An ANN can be trained by the input data to learn
to extract the statistical properties of the input data for identification
purposes. During the learning process, a meaningful relationship between input
and output variables can be established. Furthermore, ANN identifies the
relationships between different input data and output variables (classes) in
the training phase by adapting the weight factors assigned to the
interconnections between the layers. In the testing phase, the new data (such
as a new smell print) can be interpreted by the network, and the output is
calculated based on the fixed weight factors [20].Smell print
classification was performed by feed-forward ANNs consisting of three layers of
neurons. Preprocessed data was presented at the input layer of ANNs for further
processing in the hidden and output layers. ANNs are computational modeling
tools that have been extensively used in many disciplines to model complex
problems [21]. They have been applied to E-nose data for the purpose of
classification [22–29]. The trained ANN can be employed for classification of
fish freshness and the identification of the day after catching.Extracted
features from the E-nose sensors and ANNs were used to assess the freshness of
the fish by classifying the smell print data according to the day of data
collection.Of the different
neural network architectures that are employed to ascertain an optimum solution
to the intelligent analysis of actual fish freshness, the four networks, based
on a multilayer feed-forward model of three layers including the input, hidden,
and output layers, were selected.The input vector
included a total of 50 elements which indicated that the number of neurons in
the input layer should equal 50. The optimum number of neurons in the hidden
layer was chosen empirically to equal 15 and the transform function of “tansig”
was selected for this layer. The output (target) vector can be defined by a
combination of 1 second and 0 second to represent each class. To identify the
actual day that the fish was caught into four different categories, the number
of neurons in the output layer can be either 2 or 4. We chose 2 neurons for the
output layer to have 4 combinations of (00, 01, 10, and 11) among the outputs
assigned to each class. The transform function of “tansig” was also selected
for this layer.Each ANN is
initialized with random weights to the connections among the neurons. A
training data set, with a known outcome, is entered into the input neurons. The
ANN compares its own output values with the known outcome and calculates the
mean square error (MSE) value. This error value will change as the weights to
the connections are updated. The ANN attempts to minimize the error by
adjusting the weights according to the learning algorithm. This process is
repeated for a predefined number of epochs. Finally, the new data with unknown
outcome values can be tested by the ANN in the recall phase. It should be noted
that there is a small classification errors due to the nonzero value of MSE for
each network.


## 4. Results

### 4.1. Species Identification

As  mentioned
in the previous section, the experiments were carried out using a database of
232 images collected from fish eyes of four different fish types. The data base
was divided in to two groups. The first group (training data set) includes a
total of 176 images which contains 44 eye images from each fish type.The second
group (testing data set) includes the rest of 56 images and includes 14 eye
images of each fish type. The results of applying the proposed SVM classifier
to the testing data for the purpose of species identification are tabulated in
Table1.Table 1
The results of correct
classification of four types of fish by the proposed SVM classifier.
Fish typeClassification rateGurnard100%Tarakihi95±  2%Red Snapper95± 2%Trevally95± 4%


### 4.2. Freshness Assessment

The same
training parameters were chosen for all networks. In order to reduce any error
caused by weight initialization, each network was trained five times by
randomly selecting inputs from the training sets.In Table2,
the second index assigned to each Net indicates the number of attempts
for training the network. For example, Net1-3 means: Net1 at the third training
attempt.Table 2
The training performance of NET1 to NET4.
Fish typeAssignedNo.
ofMSEAveragenetworkattemptsMSESnapperNet1Net1-13.27333e-0072.50E-02Net1-29.55535e-008Net1-30.125094Net1-42.78121e-007Net1-51.51417e-008GurnardNet2Net2-17.04806e-0071.25E-02Net2-20.0416668Net2-30.0208342Net2-41.28665e-006Net2-51.39123e-007TarakihiNet3Net3-11.07294e-0095.44E-02Net3-24.39311e-008Net3-30.25Net3-41.39601e-007Net3-50.0218682TrevallyNet4Net4-15.83483e-0084.17E-03Net4-22.41354e-008Net4-32.41375e-007Net4-43.80799e-008Net4-50.0208341The training
epoch was set to 100, and the TRAINLM training function was used (because it is
very fast) for all four networks. The TRAINLM is a very fast network-training
function in MATLAB that updates weight and bias values according to
Levenberg-Marquardt optimization [17].


### 4.3. Test Results

From the
results presented in Table2, we selected the best trained networks, which are
Net1-5 (for Snapper), Net2-5 (for Gurnard), Net3-1 (for Tarakihi), and Net4-2
(for Trevally).The selected networks were used to test the fish freshness using 32 test
data sets. As discussed in previous sections, the data sets included the
extracted features of the smell print of four fish selected for the days of
data collection (days 2, 5, 7, and 8).The correct rate of identification and as the average number of days
after catching the fish is reported in its average for the number of days after
catching the fish are reported in Table3.Table 3
Correct
identification of the number of days after catching the fish by the selected
networks.
Fish
typeSelectedNo.
of daysCorrectAveragenetworkafter catchingclassificationcorrectfishrate(%)rate(%)SnapperNet1-5291.67%95.835100%791.67%8100%GurnardNet2-5283.33%93.75591.67%7100%8100%TarakihiNet3-1283.33%91.67591.67%791.67%8100%TrevallyNet4-2291.67%95.835100%791.67%8100%


## 4.1. Species Identification

As  mentioned
in the previous section, the experiments were carried out using a database of
232 images collected from fish eyes of four different fish types. The data base
was divided in to two groups. The first group (training data set) includes a
total of 176 images which contains 44 eye images from each fish type.The second
group (testing data set) includes the rest of 56 images and includes 14 eye
images of each fish type. The results of applying the proposed SVM classifier
to the testing data for the purpose of species identification are tabulated in
Table1.Table 1
The results of correct
classification of four types of fish by the proposed SVM classifier.
Fish typeClassification rateGurnard100%Tarakihi95±  2%Red Snapper95± 2%Trevally95± 4%


## 4.2. Freshness Assessment

The same
training parameters were chosen for all networks. In order to reduce any error
caused by weight initialization, each network was trained five times by
randomly selecting inputs from the training sets.In Table2,
the second index assigned to each Net indicates the number of attempts
for training the network. For example, Net1-3 means: Net1 at the third training
attempt.Table 2
The training performance of NET1 to NET4.
Fish typeAssignedNo.
ofMSEAveragenetworkattemptsMSESnapperNet1Net1-13.27333e-0072.50E-02Net1-29.55535e-008Net1-30.125094Net1-42.78121e-007Net1-51.51417e-008GurnardNet2Net2-17.04806e-0071.25E-02Net2-20.0416668Net2-30.0208342Net2-41.28665e-006Net2-51.39123e-007TarakihiNet3Net3-11.07294e-0095.44E-02Net3-24.39311e-008Net3-30.25Net3-41.39601e-007Net3-50.0218682TrevallyNet4Net4-15.83483e-0084.17E-03Net4-22.41354e-008Net4-32.41375e-007Net4-43.80799e-008Net4-50.0208341The training
epoch was set to 100, and the TRAINLM training function was used (because it is
very fast) for all four networks. The TRAINLM is a very fast network-training
function in MATLAB that updates weight and bias values according to
Levenberg-Marquardt optimization [17].


## 4.3. Test Results

From the
results presented in Table2, we selected the best trained networks, which are
Net1-5 (for Snapper), Net2-5 (for Gurnard), Net3-1 (for Tarakihi), and Net4-2
(for Trevally).The selected networks were used to test the fish freshness using 32 test
data sets. As discussed in previous sections, the data sets included the
extracted features of the smell print of four fish selected for the days of
data collection (days 2, 5, 7, and 8).The correct rate of identification and as the average number of days
after catching the fish is reported in its average for the number of days after
catching the fish are reported in Table3.Table 3
Correct
identification of the number of days after catching the fish by the selected
networks.
Fish
typeSelectedNo.
of daysCorrectAveragenetworkafter catchingclassificationcorrectfishrate(%)rate(%)SnapperNet1-5291.67%95.835100%791.67%8100%GurnardNet2-5283.33%93.75591.67%7100%8100%TarakihiNet3-1283.33%91.67591.67%791.67%8100%TrevallyNet4-2291.67%95.835100%791.67%8100%


## 5. Discussion and Conclusion

The study
undertaken has dealt with fish species identification from fish eye samples
using statistical learning. New sets of features have been extracted from the
original fish eye image followed by the development of a two-level classifier
to improve the generalization properties of the trained classifiers. The
statistical features, such asmean, max, std, var, were extracted from
the fish eye images. Part of the data set was used for training and the other
part for testing of the proposed SVM-based classifier.Evaluation carried out on 232
fish eye images shows
that the performance of the two-level classifiers using linear and MLP kernel
was good with an overall classification rate of 96.46%. For the identification
of some special cases of fishes with no visible eyes, such as flounders and
soles, fish skin images may be used instead of fish eye images.The use of
commercial E-nose in food industry and quality control such as assessment of
fish freshness is the subject of great interest. After the fish is caught, it
starts degrading. Environmental conditions such as temperature and water
contents play a significant role in maintaining the freshness of the fish.
E-nose can be employed not only to estimate the freshness but also it can be
used to predict how fast it degrades in a given environmental condition. E-nose
can be also employed to control the environmental conditions via a certain
feedback mechanism to prolong fish freshness.In this paper,
we presented a fish freshness assessment method based on a portable E- nose and
ANN classifiers. The array of chemical sensors, forming the basis of the
E-nose, responded to the signal pattern specific to the different days of each
fish type. The ANN-based classifiers processed these signals and assigned the freshness
level to the individual fish.The proposed
method was tested on the samples of four fish over different days. The results
were obtained using similar architecture for all four networks (50, 15, and 2
neurons in the input, hidden, and output layers, resp.). The networks were
trained with different training data sets, collected from individual fish,
therefore the weight values among their layers were different. After training
the networks five times, we selected the best trained network for each fish. By
randomly selecting the learning and testing sets, the average learning error
equalled2.50E-02 (for Net1), 1.25E-02 (for Net2), 5.44E-02 (for Net3), and 4.17E-03 (for Net4).While all
four fish could be classified using a single network, the purpose of using four
different networks was to increase the accuracy of the system. The assessment
of freshness for individual fish was performed by a specific network which was
trained for the specie.A fairly
accurate assessment of fish freshness has been obtained for the selected
samples. The averages of the right classification of the number of days after
catching the fish were
95.83%, 93.75%, 91.67%, and 95.83% for the selected fish types. The results
confirm that the proposed assessment method can be considered as a suitable
technique for freshness assessment. However, it has to be put on record that
the samples were kept in a refrigerator which is the practical way to store
them and freshness is related to the refrigerator stored environment.The common
disadvantages with respect to quality control within industrial environment are
that the process is slow and the analysis requires laboratory facilities and
qualified staff. Moreover, there is arising difficulty in establishing standards
for use in various locations. The key obstacle in the way of this system being
considered as a commercial solution lies in the fact that quality
controllers/inspectors need to evaluate the freshness of a batch of fishes
rather than individual sample. Employment of this system to process batch of
fishes is still being explored.In
conclusion, E-nose can be employed not only to estimate the freshness but also
it can be used to predict how fast it degrades in a given environmental
condition. E-nose can be also employed to control the environmental conditions
via a certain feedback mechanism to prolong fish freshness. Using four
different networks can increase the accuracy of the system. Further research is
being undertaken to make the system commercially viable.



---

*Source: 628585-2008-04-22.xml*