225
Bulgarian Journal of Agricultural Science, 21 (No 1) 2015, 225-236
Agricultural Academy

GRAIN SAMPLE QUALITY ASSESSMENT FUSING THE RESULTS FROMCOLOR IMAGE AND SPECTRA ANALYSES
M. I. MLADENOV1, M. P. DEJANOV2 and R. TSENKOVA3
1
University of Ruse, Department of Automatics and Mechatronics, BG – 7017 Ruse, Bulgaria
2
University of Ruse, Department of Automatics and Mechatronics, BG - 7017 Ruse, Bulgaria
3
Biomeasurement Technology Laboratory, Kobe University, Japan

Abstract
MLADENOV, M. I., M. P. DEJANOV and R. TSENKOVA, 2015. Grain sample quality assessment fusing theresults from color image and spectra analyses. Bulg. J. Agric. Sci., 21: 225-236
The paper presents the approaches, methods and tools for assessment of main quality features of grain samples which are
based on color image and spectra analyses. Visible features like grain color, shape, and dimensions are extracted from theobject images. Information about object color and surface texture is obtained from the object spectral characteristics. The
categorization of the grain sample elements in three quality groups is accomplished using two data fusion approaches. The
first approach is based on the fusion of the results about object color and shape characteristics obtained using image analysis
only. The second approach fuses the shape data obtained by image analysis and the color and surface texture data obtained byspectra analysis. The results obtained by the two data fusion approaches are compared.

Key words: grain quality assessment, color image analysis, spectra analysis, classification, data fusion
Abbreviations: ANN – Artificial Neural Networks; CA - Cluster Analysis; CDRBE - classifier with decomposing
RBEs; CRBEP - classifier with RBEs which takes into account the class potentials; CSRBE - classifier withstandard RBEs; CVS – Computer Vision System; HIS - Hyperspectral Imaging System7; INTECHN – Intelligent
Technologies for Assessment of Quality and Safety of Food Agricultural Products; KNN - K-Nearest Neighbors;
LDA – linear discriminant analysis; NIR spectra analyses – Near infrared spectra analyses; NIRS - A NIR
spectroscopy; PCA - Principle Component Analysis; QDA - quadratic discriminant analysis; RBEs - Radial Basis
Elements; SIMCA - Soft Independent Modeling of Class Analogy; SVM - Support Vector Machines; VIS spectra
analyses – Visible spectra analyses; 1cc – first color class; 1csh – first shape class; 1cst – first class standard

Introduction
One of the main factors of human life quality is the foodquality and safety. The food provides the energy, needed forthe human body for movement, physical and intellectual activity. It is a source of proteins, fats, carbohydrates, vitamins
and minerals, due to them the cells and tissues are renovated.
As a result of the feeding the human organism produces hormones, enzymes and other regulators of the metabolic processes.
The assessment of food quality and safety is an importantpart of food production chain. The grain is a main part ofCorresponding author: mladenov@uni-ruse.bg

the human and animal food. The higher food quality requirements demand development of new, objective, intelligent
technologies, methods and tools for assessment of main foodquality and safety features.
The grains and the cereals are an essential part of the humanfood. The cereals assure the half of the daily energy ration of thepeople in the developed countries and 80% in the developing
countries. The grains of the wheat, maize, rice, barley, oats andmillet contain about 60 - 80% carbohydrates, 8 - 15% proteinsand 1.5 - 2% fats according to Emes et al. (2003).
The problem for rapid, objective, automated, express andnondestructive grain quality assessment is a complex and226

M. I. Mladenov, M. P. Dejanov and R. Tsenkova

multilevel task, related to the analysis of the appearance, thevisible features as well as the grain contents, smell, flavor,
moisture content, infections, non-grain impurities, etc. of thegrain sample elements.
This investigation is focused on the assessment of main
corn grain quality features. There are proposed and investigated methods and tools for feature extraction and data
dimensionality reduction, analysis and identification of thegrain sample elements. They are based on the analysis ofcolor images and spectral characteristics of the investigatedobjects, as well as the fusion of the results of the two kindsof analyses. This approach is realized in the frame of INTECHN project DО 02-143/16.12.2008 (2008) “Developmentof Intelligent Technologies for Assessment of Quality and
Safety of Food Agricultural Products” founded by Bulgarian
National Science Fund.
According to the Bulgarian national standards the main
quality features are as follow: the appearance, shape, color,
smell, taste, moisture and impurities typical for the variety.
Some of these features of a corn grain sample are presented
in Table 1. The assessment of grain quality features presented
in Table 1 is mainly related to the visible features of the grain
sample elements and features related to the grain content, drymatter content, moisture content, starch, protein, glutenin,
vitamins, toxins and mineral content according to Bulgarian
Government Standard 607-73 (1973).
It is obvious that all of the features mentioned above cannot
be evaluated using the information extracted from one sensor
source only. A huge part of these features (for example grain
appearance, shape and color) are evaluated by an expert on thebase of visual assessment only. That’s why such features can be
efficiently evaluated using a Computer Vision System (CVS).
A review of the progress of computer vision in the agriculturaland food industry is given by Brosnan and Sun (2003).
Some features like grain composition, content, infections
etc. cannot be evaluated by means of CVS. Spectra analysis
is mainly used for assessment of such features. Other features
like moisture content, specific weight etc., are evaluatedby other standard physicochemical methods. Acording toTable 1Corn grain quality groups
Grain quality groups
First group - standard kernel
Second group -grain impurities
Third group – non-grain impurities

Mladenov (2011) obtaining a complex assessment of the grain
quality using data about color, shape and dimensions of thegrain sample elements is a complicated and multilevel task.
This is because the color, the shape and the dimensions of theelements in a sample vary within a wide range.
There are many publications related to the assessment ofsome particular quality features using color image analysis. A
digital image analysis algorithm based on the textural features
is developed for classification of individual kernels of cereal
grains (Mahesh et al., 2010). Color analyses are used to assess
variety (Majumdar and Jayas, 2000a,b), infections (Ning etal., 1988; Mladenov et al., 2011b), germination (Mladenov
and Dejanov, 2008a), weed identification (Aitkenhead et al.,2003), etc.
The grain variety is usually assessed by means of different
morphological features related to the shape and geometricalparameters. A set of eight morphological features namelyarea, perimeter, length of major axis, length of minor axis,
elongation, roundness, Feret diameter and compactness areused to recognize five different kinds of cereal grains (Paliwal
et al., 2001). A broader investigation about classificationof barley, Canada Western Amber Durum wheat, Canada
Western Red Spring wheat, oats, and rye is presented in(Paliwal et al., 2003a). It is based on a total of 230 features (51morphological, 123 color and 56 textural). A profile analysis
through one-dimensional digital signals (Liao et al., 1993),
by modeling the shape by means of a set of morphologicalfeatures (Paliwal et al., 2003b) and by shape curvature
analysis (Mladenov et al., 2011) is performed for assessmentof grain purity. Computer vision methods are also used todetermine kernel mechanical damage, mold damage (Ng etal., 1998), broken kernels in threshing process (Schneider etal., 1999), etc.
A preliminary investigation (Mladenov et al., 2011) showsthat we can’t get a precise assessment of some of the grain
sample elements like smutty grains, infected grains and nongrain impurities, using an image analysis. It is difficult to
detect the small changes of surface texture through a CVS.
That’s why we expect a more accurate assessment of such

Grain quality features
Whole grains and broken grains bigger than the half of the whole grain, withappearance, shape and color typical for the variety
Broken grains smaller than the half of the whole grain, heat-damaged grains, small
grains, shriveled grains, green grains, sprouted grains, infected (with Fusarium)
grains, smutty grains.
Corn-cob particles, leaf and stem fractions, pebbles, soil and sand, as well as
harmful elements

Grain Sample Quality Assessment Fusing The Results from Color Image and Spectra Analyses
features to be gotten by spectra analysis. Unfortunately
information about shape and dimensions cannot be extracted
from spectra.
Visible (VIS) and Near Infrared (NIR) spectra analyses areapplied in the assessment different food products (Tsenkova
et al., 2010; Nakakimura et al., 2012; Damyanov et al., 2006;
Mladenov et al., 2011), as well as of grain quality features
like grain composition, dry matter content, moisture content,
starch, protein, glutenin, vitamins, toxins, mineral content,etc (Huang et al., 2008; Mladenov et al., 2011; Huang et al.,2008; Dowell et al., 2002; Girolamo et al., 2009). Different
calibration models are developed for predicting graincomposition and content (Peiris, 2001; Paulsen et al., 2003;
Paulsen et al., 2004; Huang et al., 2008; Wesley et al., 2001;
Miralbés et al., 2003).
Modified partial least squares models on NIR spectra (850
− 1048.2 nm) are developed to predict grain quality features(Paulsen et al., 2003). The best models are obtained forprotein, moisture, wet gluten, and dry gluten with r2 = 0.99,
0.99, 0.95, and 0.96, respectively.
The spectra analysis is also used for detection of different
grain infections. Determination and prediction of the content
of ergosterols and different kinds of mycotoxins like aflatoxin,
fumonisin and others are very important tasks because
mycotoxins are toxic for animals and humans. Reflectanceand transmittance VIS and NIR spectroscopy are applied
to detect fumonisin in single corn kernels infected with
Fusarium verticillioides (Dowell et al., 2006). A method for
determination of Fusarium graminearum infection is proposedin (Paliwal et al., 2003b). The classification accuracy reaches
to 100% for individual samples. Transmittance spectra(500 to 950 nm) and reflectance spectra (550 to 1700 nm)
are suggested as tools for aflatoxin determination in single
whole corn kernels (Pearson et al., 2001). The authors use
discriminant analysis and partial least squares regression forspectral data processing. The best results are obtained using
two feature discriminant analyses of the transmittance data.
A NIR spectroscopy (NIRS) method for estimation of soundkernels and Fusarium-damaged kernels proportions in grain
and for estimation of deoxyinivalenol levels is proposed in(Peiris et al., 2010). The method classifies Fusarium damaged
kernels with an accuracy of 99.9%. A neural network based
method is developed for deoxynivalenol levels determination
in barley using NIRS from 400 to 2400 nm (Ruan et al.,2002). Fourier transform NIRS is applied for rapid and noninvasive analysis of deoxynivalenol in durum and common
wheat (Girolamo et al., 2009). A qualitative model for
discrimination of blank and naturally contaminated wheat
samples is developed. Classification accuracy of the model is
69% of the 65 validation samples.

227

A comparatively new approach for grain quality assessment
is based on the Hyperspectral Imaging System (HIS). A
HIS get data about object spectra at some regions (pixels)of the object area. Every pixel contains spectral reflection
data for many narrow situated spectral bands usually in VIS
and NIR spectrum. Spectral data is normally presented asa hyperspectral cube. HIS could be considered as a variantof a color image analysis, where the object image is divided
into pixels, every pixel is analyzed using multiband spectral
analysis instead of three band analysis (R, G, B).
The hyperspectral analysis is applied for assessment ofdifferent grain features. For example, Mahesh еt al. (2010)
use HIS for developing class models of different wheat
varieties in Western Canada. The grain samples are scanned
in NIR spectrum (960 – 1700 nm) at an interval of 10 nm. 75
different values of the intensity of the reflection are obtained
from hyperspectral images and they are used for class model
development. These models assure about 90% classification
accuracy.
NIR spectroscopy is applied for assessment of grainmoisture level too (Mahesh et al., 2008a,b; Mahesh etal., 2010). The authors present a new method using NIR
hyperspectral imaging system (960 – 1700 nm) to identify fivewestern Canadian wheat classes at different moisture levels.
They are found that the linear discriminant analysis (LDA)
and quadratic discriminant analysis (QDA) could classify
moisture contents with classification accuracies of 89 – 91%
and 91 – 99% respectively, independent of wheat classes.
Once wheat classes are identified, classification accuracies of
90 – 100% and 72 – 99% are observed using LDA and QDA,
respectively, when identifying specific moisture levels.
The HIS (350 – 2500 nm) is used for assessment of protein
content in wheat grains (Wang, 2004) too.

Materials and Methods
Color image analysis. Grain groups and subgroups
Some features of grain sample elements, which arein principle evaluated by an expert on the basis of visual
estimation, are assessed using CVS within the framework of this investigation. These features are related to theappearance, the color, the shape and the dimensions of thegrain sample elements.
Groups (classes) and subgroups (subclasses) in which thecorn grain sample elements are distributed are presented inTable 2. The tree normative quality classes are based on thecorresponding color and shape subclasses presented in thesame row of the Table 2.
Because the color and shape features are extracted andrepresented in a different manner, it is expedient their

228
assessment to be made separately. After that the results fromthe two assessments have to be fused to obtain the finalclassification to one of the normative classes. Color and shapegroups are divided in several subgroups in order to simplify
the classification procedure. Color features are divided
in 8 basic classes corresponding to the typical of different
sample elements color zones and one additional class that
corresponds to the non-grain impurities (it is impossibly todefine a compact class for non-grain impurities). The sample
elements are divided in 3 basic shape classes corresponding
to the whole grains, broken grains bigger then the half ofwhole grain and broken grains smaller than the half of thewhole grain, and one additional class that corresponds to thenon-grain impurities. Each of the three basic shape classes isdivided in 6 shape subclasses.
Features extraction from images
RGB, HSV, XYZ, NTSC and YCbCr color models areused for extracting the object area from background and for
different color zones extraction in the frame of object area.
These zones are typical of the standard grains, heat-damaged
grains, green grains, smutty grains, infected (with Fuzarium)
grains, bunt and non-grain impurities. Furthermore fourcolor texture models (Mladenov, 2008b) are development
for this purpose. It is expected they will better underline thedifference between the color zones in the input RGB image.
Ten–dimensional descriptions are applied to represent
the shape of the grain sample elements (Mladenov et al.,2011a,c). The following procedure is realized to obtain the
shape description. First, the binary image of the object area iscreated. After that the object’s peripheral contour is extracted
and the bisection line of the object contour is found. An odd

Table 2
Corn grain sample classes and subclasses
Normative (quality) classes
1cst - standard kernel (whole grains andbroken grains bigger than the half of thewhole grain,) with appearance, shape andcolor typical of the variety

M. I. Mladenov, M. P. Dejanov and R. Tsenkova
number of cross-sections perpendicular to the bisection line
are built (Figure 1).
The relative length hi = si/D of the cross–sections, as wellas the size and the sign of the difference between two neighborcross–sections ∆ i = ∆1 − ∆ 2 are calculated. Finally, the object
shape description is presented in the following form:()

X sh = h1, h2 ,, hn , ∆1, ∆ 2 ,, ∆ n		

(1)

The contour line of the corn kernels has a huge asymmetry along to the bisection line. It is easy to locate the germ in
the whole grain and to build contour descriptions and models
with proper orientation. For broken grain, depending on whatpart of the whole grain is remained (with the germ or without

Fig. 1. Object shape description: D – length of thebisection line; hi = si/D – length of a cross–section;∆ i = ∆1 − ∆ 2 - difference between neighbor cross–sections

Color classes
Shape classes
1cc- grains with color typical of the 1csh- with typical of the whole grainsvariety, back sidevariety shape
2cc- grains with color typical of the 2csh- broken grains bigger than thevariety, germ side
half of the whole grain
3cc- heat-damaged grains
2cst-grain impurities: broken grains smaller
4cc- green grains
than the half of the whole grain, heat3csh- broken grains smaller than the
5ccmouldy grains
damaged grains, small grains, shrivelled
half of the whole grain and small andgrains, green grains, sprouted grains,
6cc- smutty grains
shrivelled grains
infected (with Fusarium) grains, smutty
7ccinfected(withFusarium)grainsgrains.
8cc- sprouted grains
3cst-non-grain impurities: corn-cob
particles, leaf and stem fractions, pebbles,
9cc- non – grain impurities
4csh- non – grain impurities
soil and sand, as well as harmful elements

Grain Sample Quality Assessment Fusing The Results from Color Image and Spectra Analysesit) the contour descriptions could be sufficiently different. It
is necessary to define two types of descriptions and models
for 2csh and 3csh classes (for its corresponding subclasses):
for broken grain where the germ exists in the remaining partof the grain and where it does not exist. The all shape groups
are divided into 18 subgroups (subclasses).
Spectra analysis. Grain groups and subgroups
The spectra analysis is used to evaluate the color and texture features of grain groups like Fusarium infected grains,
shriveled grains, sprouted grains, smutty grains and non
grain impurities. The color image analyses doesn’t enable toobtained sufficiently precise assessment of such features.
The groups and subgroups in which the corn grain sample elements are distributed based on the spectra analysis areshown in Table 2.
Features extraction from spectra and data
dimensionality reduction
Different methods like Principal Component Regression, Partial Least Squares Regression, Principal Component
Analysis, Hierarchical Cluster Analysis and other methods
are applied for developing a model to predict a property of interest, as well as for feature extraction and large and complex
spectra data reduction. Methods like K-Nearest Neighbors(KNN), Linear Discriminant Analysis (LDA), Quadratic
Discriminant Analysis (QDA), Cluster Analysis (CA), Support Vector Machines (SVM), Neural Networks (ANN), andSoft Independent Modeling of Class Analogy (SIMCA) aremainly used for assessment of different grain features using
data from grain spectra.
The spectral characteristics are obtained using QE65000
spectrophotometer. Each characteristic is a vector with about
1500 components. Principle Component Analysis (PCA) andcombination of Wavelet descriptions and PCA are applied
for extracting typical features from object spectra and forspectral data dimensionality reduction. Тhе Wavelet1(detailcoefficients) and Wavelet2 (approximation coefficients) andthe Haar, Daubechies2, Coiflet2, Symlet2 wavelet functions
are used in this investigation. The level of decomposition isvaried from m = 1 to m = 4. The most informative wavelet
coefficients are chosen using PCA method.
Grain quality assessment fusing data from image andspectra analyses
Because the color and shape features are extracted anddescribed in a different manner, the assessment of these characteristics is separately done. After that the results from thetwo assessments are fused in order to obtain the object’s final
categorization to one of the normative classes.

229

Different variants of data fusion schemes are developed
at different stages of the study (Mladenov et al., 2011c), theschemes developed could be associated with hierarchicalclustering algorithms. Their typical feature is that different
criteria for class merging are used at different levels of data
fusion.
Variant 1 - The First scheme uses a simplified fusion
scheme. It is presented in Figure 2.
The input data (input classes) are separated in two groups– data about object color characteristics and object shapedata. The first group consist of 10 color zones typical of thecorn grains. These are regions of pixels in the grain image
with similar color characteristics. The color zones were extracted from kernel images within the framework of a preliminary investigation.
The second group consists of 18 shape subclasses (1scsh,
2scsh,…, 18scsh). The first six of them correspond to different shape models of whole grains, the next six – to modelsof broken grains bigger than the half of whole grain and thelast six – to models of broken grains smaller than the half ofwhole grain.
The color class (1cc, 2cc, …, 8cc) is determined on the basis of preliminary defined combinations of color zones at thefirst stage of fusing the results from the color analysis. The
shape subclasses are merged into one of the three main shapeclasses (1csh, 2csh and 3csh).
At the second stage of the analysis the fusion of color andshape classes is made in order to form the final decision of
object classification in one of the three normative classes(1cst, 2cst and 3cst). The assessment whether the shape of theobject is typical of one of the three classes or not, is used as afusion criterion for color classes 1cc to 5cc. For 6cc, 7cc and8cc classes the shape is not important at all.
Variant 2 - The second scheme (Figure 3) uses color andcombined topological models of typical color zones. The

Fig. 2. Data fusion of the color and shape characteristics
based on an image analysis only

230

M. I. Mladenov, M. P. Dejanov and R. Tsenkova

topological models represent the plane distribution of thecolor zones within the object area. A set of color topology
models (when 3 or more typical for the kernels color zones
are found Figure 4a) and combined topology models (whenonly 2 typical color zones are found Figure 4b) is preliminary
defined. The combined topological models represent theplane distribution of some shape element (kernel tipcap orcrown region) and the color zones found.
The final categorization when such topology is found is
performed on the base of the object area only. The object
shape and the object area are important for the final categorization, when one typical color zone is found. For 6cc, 7cc
and 8cc classes the shape is not important at all.
Variant 3 - The third scheme (Figure 5) fuses color
characteristics extracted from spectra and shape characteristics extracted from images. This is because the color class recognition is more precise when we use spectra

analysis instead of image analysis. The main criterion
for final categorization is the object color class. The
correspondence of the object shape and/or the object area
to the typical for the grain sample elements shapes is anadditional criterion.
The variants 1 and 2 of color and shape data fusion could
be associated with the first level of multisensory data fusion(Direct fusion of sensor data (Liggins et al., 2008), we canconsider the results from color and shape analyses as signals
obtained from two different sensors (the shape and the color
are extracted and presented in a different way).
The third variant of data fusion is a typical example ofthird level of multisensory data fusion (Decision level fusion). The color and shape data are obtained from two different sensors – spectrophotometer and RGB camera. Thisdata is separately processed and after that the results are
combined to get the final decision.

Fig. 3. Data fusion of color and shape characteristics basedon an image analysis and grain topological models

Fig. 5. Data fusion of color characteristics extracted fromspectra and shape characteristics extracted from images

Fig. 4. Color (a) and combined (b) topology

Grain Sample Quality Assessment Fusing The Results from Color Image and Spectra Analyses
It is expected that the data fusion procedures can improve
the final classification in normative classes in comparison withthe variants, when the categorization is made using object
color data or object shape data only. This expectation is basedon the assumption, that the fusion procedure can ignore or
decrease some of the factors, which determine big errors of the
final classification, when we use object color or object shapedata only. The effectiveness of the proposed methods for data
fusion is confirmed with the results presented in the tables.
Classification of the grain sample elements
Specific classification strategy, classifiers, and validation
approach (Mladenov, 2011a,b) are applied for the categorizationof the grain sample elements, which are conditioned by the
specificity of the classification tasks.
Classification approach
If the classes (related with the color, shape, PCA and
Wavelet + PCA descriptions) are presented in the feature
space, a part of them (1cc, 2cc,…8cc) will form comparatively compact class regions. The sets of descriptions extracted
from the grain sample training sets are used for developing
the models of these grain sample groups. Each class model ispresented by the class centre (the average value of the classtraining data) and the class boundary surface. The boundary surface is determined through a threshold value of thecovariance of the class training data. A correct model for the
9cc class (non-grain impurities) could not be created because
the characteristics of the elements of this class could be sufficiently different in each subsequent grain sample.
As a correct model for the 9th grain group could not becreated, a part of the descriptions of such objects of the testing
set could get into the boundaries of the other eight classes.
A big part of them would get outside the class regions andcould be located in a random place in the feature space. These
descriptions could be considered as noisy vectors. It could
be assumed that the comparatively compact class regionsof the objects from the first eight groups are submerged ina noisy environment. Therefore the task for categorizationof the grain sample elements can be interpreted as a task
for classification in classes, whose boundaries have definite
shapes, dimensions and location in the feature space, andthey are situated in a noisy environment (Mladenov, 2011a).
Under this formulation, the use of popular strategies like
LDA, CA, SVM, KNN and some other methods, which build
boundaries between class regions, is obviously not a good
choice. This is due to the fact that for the class 9cc a correctmodel cannot be created.
Furthermore if there are too big deviations of the actual
values of the object characteristics and intensive measure-231

ment noise, the class areas can be overlapped. Very often
correct information about prior probabilities of the classes
is missing. This makes the classification problem more complex. If we use a classifier, which demands the prior probabilities to be known (for example Bayesian classifier), thetraining procedure has to be implemented using the priorprobabilities obtained from the number of elements in training sets. When we assess quality of an unknown sample, theratio of the number of elements from different classes could
be sufficiently different from this ratio in the training sets.
The classifier decision can be sufficiently different from theoptimal decision under these circumstances. In this case theclassification task is reduced to a task for approximation of
overlapping class areas when the classes are situated in noisy
environment and correct information for class a priori probabilities is missing.
Classifiers. The task for grain class modelling is reduced
to a task for approximation of the boundaries of the grainclass regions. For this purpose classifiers based on Radial
Basis Elements (RBEs) could be used. Such classifiers will
easily perform the approximation of the class regions andwill simplify the classification procedure.
The following classifiers (Mladenov et al., 2011c) are used
for class area approximation: CSRBE, CDRBE and CRBEP.
Classification accuracy
The accuracy of classification procedure is evaluated on
the bases of the following classification errors:
ei 

FN i
TPi  FN i 

,				

(2)

where ei gives the relative part of objects from some classi, which are assigned incorrectly to other classes k = 1...N,
where FNi is the number of elements from the ith classclassified incorrectly to other classes, TPi is the number ofcorrectly classified elements from the ith class;
gi 

FPi
TPi  FPi 

,				

(3)

where gi gives the relative part of objects from other classes,
which are assigned to class i, where FPi is the number of
elements from other classes assigned to the ith class;Neo  FN ii 1NN ,			
  TPi   FN i i 1
 i 1
(4)

where eо (classification error rate) gives the relative part of
all incorrectly classified objects, were N is the number ofclasses.

232

M. I. Mladenov, M. P. Dejanov and R. Tsenkova

Test Setup
The hardware system consists of the following main
components: computer vision system (CVS) (5) andspectrophotometer (4) (Figure 6). The CVS includes twocolor CCD cameras (1) which give a possibility to obtain color
images of investigated object (2) in two planes (horizontal
and vertical). The illuminant system (3) is used for direct
object illumination. The reflectance spectral characteristics
are obtained using spectrophotometer type QE65000 (4). The
specifications of the camera, lenses, spectrophotometer andcomputer are presented in Table 3.
All investigations are carried out in laboratory conditions
with a constant artificial lighting. Investigations in industrial
environments are not made. The object color images are ob-

Fig. 6. Test Setup. 1- color CCD camera; 2- investigatedobject; 3- illuminant system; 4- spectrophotometer;
5- computer vision system (CVS)tained in a dark room. The objects are separately placed ona color pad with a color different from the grain sample elements. It allows the object zone to be accurate extracted fromthe background.

Results and Discussions
Training and testing sets
The developed procedures for grain sample quality
assessment are trained, validated and tested with sets presented
in Table 4. The training and testing sets include a number ofdifferent elements (whole grains, broken grains, etc.)
The results from objects classification in shape and color
classes are presented in Table 5. All classifiers described in
the section “Classifiers” are used in the investigation. The “selected classifier” is the classifier with the best performance.
The results from objects classification in normative classes
using the three variants of data fusion are presented in Table
6. Two classifiers are placed in the field “Selected classifiers”.
The first classifier is applied in color class recognition and thesecond classifier is used in shape class recognition.
• The test results in shape class recognition show that the rateof objects from class 1csh assigned to other classes is comparatively small (4.9%). On the other hand, the rate of objects assigned to this class which actually belong to otherclasses is sufficiently bigger (35.2%). The rate of objects
from class 3csh assigned to 2csh and 4csh is big too.
• The classification error rate of objects from class 3csh (partsof kernels) is large. This is an expected result because it is
impossible to define some standard shape for objects from

Table 3
Technical specifications of camera, lens, spectrometer, computer and lighting system
Name
Information
Color camera
Color digital video camera with USB interface, 1/3” Sony CCD sensor with progressive
DFK 31AU03
scanning, resolution – 1024х768 pixels;
Camera lensLens with variable focal length – 3.5 to 8 mm, diaphragm – from 1.4 to infinity,
T2Z 3514 CS
CS – assembly, MOD 0.3m;
CPU
Intel
Core
2
Duo
P8700 (2.53 GHz, 3MB L2 Cache, 1066 MHz FSB)
Computer system
RAM- 4 GB (2x2048 GB) DDR2, 800 MHz
Dell Vostro 1720
Video card - NVIDIA GeForce 9600M GS 512MB

Spectrometer
QE65000

Lighting system
Fluorescent lamps

Detector: Hamamatsu S7031-1006;
Range: 350-1000 nm;
Resolution: 1024 x 58 pixels;
Optical resolution: ~0.14-7.7 nm FWHM;
S/N ratio: 1000:1;
ADC: 16 bits;
Dynamic range: 7.5 x 109, 25000:1 for single measurement;
Integral time: 8 ms to 15 min ;Adjusted linearity: >99.8%;
The lighting system is compound of two ring-shape fluorescence sources with different
diameters. It is used for direct illumination of investigated objects. They are placed so thatthe light can uniformly illuminate the object.

233

Grain Sample Quality Assessment Fusing The Results from Color Image and Spectra Analyses
this class. In many cases even a qualified expert will notrecognize such objects if no color characteristics but only
shape is taken into consideration. During the classifierstraining models of broken kernels are created on the basisof whole kernels models and that is why the training sample
classification error rates for classes 2csh and 3csh are small.
This explains the big difference between training and testing classification results for these two classes.
• The testing error in color class recognition using spectra analysis (7.3%) is acceptable bearing in mind the specific investigation conditions and the diversity of grain sample elements.
• The comparative analysis of the results obtained using
different variants of classifier validation, training andtesting confirms the effectiveness of the classification
strategy, classifiers, validation approach and data models.
For example, if we use the three data models: PCA, Wavelet1+ PCA and Wavelet2 + PCA the training errors are 6.8%,
6.3% and 10.3% respectively using the CDRBE classifier.
The validation approach (when the non – grain impurities
are included in validation procedure, but are excluded fromtraining sets) decreases the testing error 3.8 times (from27.6% to 7.3%) in comparison with the traditional validation
approach (when the non – grain impurities are simultaneously
excluded or included in validation and training sets). The
choice of an appropriate classifier for specific classification
task has an influence over the classification accuracy too.

Table 4Training and testing sets
Classes
Training sets
Testing sets

1cc
10
47

Classes
Training sets
Testing sets

1csh
120
122

Classes
Training sets
Testing sets

1cc
120
30

Color classes recognition using CVS
2cc
3cc
5cc
7cc
10
12
15
18
81
44
24
74
Object shape recognition
2csh
3csh
4csh
135
135
63
11
256
Color classes recognition using spectra analysis
2cc
3cc
5cc
7cc
120
80
53
192
30
20
13
48

Table 5Classification errors in shape and color classes recognition
Color and shape class recognition
Color class recognition
Shape class recognition
using CVS
Selected classifiers
Errors
Class
1
2
3
4
5
6
7
8
9

Test. errors

e0=35.6%

168

8cc
42
11

9cc
536
134

ei,%
8
4.9
15.9

79.4

8.3

4.5

12.5

35
0
23.2

13.5
5.3
68.4

19.5
0
12.0

5.4
10.3
6.6

CDRBE
ei,%
5.7
69.8
77.8
43.1

9cc

Color class recognition
using spectra analysis
CDRBE
Wavelet1+PCA model
Test. Errors
gi,%
ei,%
0
15.2
0
5.1
0
0

CRBEP
gi,%
39.5
62.0
87.5
21.9

8cc
19
39

gi,%
8.9
8.3
0

Test. errors

e0=30.6%e0=7.3 %234

M. I. Mladenov, M. P. Dejanov and R. Tsenkova

For example, the training errors obtained using the CDRBE,
CSRBE and CRBEP classifiers and PCA data model are
6.8%, 72%, and 7.3% respectively.
• The classification errors in color class recognition using
spectra analysis are sufficiently smaller than the errors using image analysis. For example, the testing errors are 1.3%
and 10% respectively using the two approaches when the
non grain impurities are excluded from the validation andtesting sets. When we include the non grain impurities invalidation and testing sets these two errors are 7.3% and42%. The big difference between the two errors can be explained by the fact that the object spectral characteristics
contain not only information for objects color characteristics, but for their surface texture too. Although typical forsome grain groups color zones are found in a big part of non– grain impurities, the surface texture of these elements is sufficiently different from the typical for the grains.
• Object classification in normative classes (1cst, 2cst and 3cst)
includes complex assessment of color and shape characteristics of the investigated objects. For this purpose color dataand shape data are fused. The data fusion procedure improves
sufficiently the final classification results. The classification
error rate e0 in normative classes using CVS (Selected variant CDRBE–CRBEP) is 15.3% when data fusion Variant 1
is used and 8.6% when Variant 2 is used, while the errors ofobject color zones extraction and object shape recognition are
30.6% and 35.6% respectively. When we use Variant 3 forclasses’ recognition the classification error rate e0 decreases
about 1.6 times in comparison with the better result obtainedusing CVS. This is due to the fact that the spectra analysis
gives the best result in color classes’ recognition.

Conclusions
The results from the investigation at this stage of the INTECHN project implementation concerning grain sample

quality assessment using complex assessment on the basisof color image and spectra analyses can be summarized asfollow:• The developed approaches, methods and tools for grain
samples quality assessment based on the complex analysis
of object color, object surface texture and object shape
give an acceptable accuracy under specific experimental
circumstances. The error rate e0 = 5.3% of the final
categorization in the normative classes can be accepted as agood result at this stage of project implementation.
• The data fusion procedure improves sufficiently the finalclassification results. The classification error rate e0 using
CVS is 15.3% when Variant 1 is used and 8.6% when Variant 2 is used, while the errors of object color zones extraction and object shape recognition are 30.6% and 35.6% respectively.
• According to the minimal quality requirements of grain samples defined in Bulgarian Standard Regulation 1272/2009
the impurities (grain class 2cs and non-grain 3cs) in a grain
sample cannot be more than 12%. About 28% of standard
grains are recognized as impurities using Variant 1. Thismeans that the error is bigger than the permissible percentage of the impurities and this variant is not applicable forthe analyses of real grain samples. The errors of the Variant
2 and 3 are permissible from the point of view of the Regulation 1272/2009. Variant 3 is preferred because the error(these are objects from third and second class recognized
as objects from first class) is about 2 times smaller than thesame error of the Variant 2.
The results obtained show that the choice of an appropriate
procedure for fusion the results from color characteristics
and objects shape analysis has a significant influence overthe final classification accuracy. When we use the second
algorithm (Variant 2) which is based on color or combined
topology assessment the classification error rate decreases
about 1.8 times compared to the first algorithm (Variant 1) in

Table 6Classification errors in normative classes recognition
Color and shape data fusion
Fusion variant
Variant 1
Variant 2
Selected classifiers
Errors
Class
1cst
2cst
3cst

CDRBE-CRBEP
gi,%
7.7
27.9
12.7

Test. errors

e0=15.3%

ei,%
28.2
32.2
0.8

CDRBE-CRBEP
gi,%
4.2
17.2
6.2

Test. errors

e0=8.6%

ei,%
1.7
14.4
9.1

Variant 3
CRBEP-CDRBE
Wavelet1+PCA model
Test. Errors
gi,%
ei,%
2.7
6.8
0.8
12.7
8.4
0.4
e0=5.3 %Grain Sample Quality Assessment Fusing The Results from Color Image and Spectra Analyses
which color class assessment is based on the registration of
the typical color zones combinations only. When we fuse theresults from color classes recognition obtained on the basisof spectra analysis and shape classes recognition obtained on
the basis of image analysis (Variant 3) the final classification
accuracy is increased 2.9 and 1.6 times in comparison with
Variant 1 and Variant 2 respectively.
Acknowledgements
This investigation is a part of implementation of the research project “Intelligent Technologies for Assessment of
Quality and Safety of Food Agricultural Products”, fundedby the Bulgarian National Science Fund.

References
Aitkenhead, M. J., I. A. Dalgetty and C. E. Mullins, 2003.
Weed and crop discrimination using image analysis andartificial intelligence methods. Computers and Electronics inAgriculture, 39: 157-171.
Brosnan, T. and D. W. Sun, 2003. Inspection and grading ofagricultural and food products by computer vision systems - areview. Computers and Electronics in Agriculture, 36: 193-213.
Bulgarian Government Standard, Grain maize for purchase andmarketing. Pp. 607-73.
Damyanov, Ch. I., 2006. Non desatructive recognition of quality
In system for food products automatic sorting. Monography.
Academic Publishing of the University of Food Technologies,
Plovdiv.
DО 02-143/16.12.2008. Project INTECHN - Intelligent Technologies for Assessment of Quality and Safety of Food Agricultural Products, 2008-2012, funded by the Bulgarian National
Science Fund.
Dowell, F. E. E., B. Maghirang, F. Xie, G. L. Lookhart, R. O.
Pierce, B. W. S. Seabourn, R. Bean, J. D. Wilson and O. K.
Chung, 2006. Predicting Wheat Quality Characteristics andFunctionality Using Near-Infrared Spectroscopy. Cereal Chem.,
83 (5): 529-536.
Dowell, F., T. Pearson, E. Maghirang, F. Xie and D. Wicklow,
2002. Reflectance and Transmittance Spectroscopy Applied
to Detecting Fumonisin in Single Corn Kernels Infected with
Fusarium verticillioides. Cereal Chemistry, 79 (2): 222-226.
Emes, M. J., C. G. Bowsher, C. Hedley, M. M. Burrell, E. S.
F. Scrase-Field and I. J. Tetlow, 2003. Starch synthesis andcarbon partitioning in developing endosperm. J. Exp. Bot., 54(382): 569-575.
Girolamo, A. D., V. Lippolis, E. Nordkvist and A. Visconti, 2009.
Rapid and noninvasive analysis of deoxinivalenol in durum andcommon wheat by Fourier – Transform infrared spectroscopy.
Food Additives and Contaminants, 26 (6): 907-917.
Huang, H., H. Yu, H. Xu and Y. Ying, 2008. Near infrared
spectroscopy for on/in-line monitoring of quality in foods andbeverages: A review. Journal of Food Engineering, 87 (3): 303-313.

235

Kos, G., H. Lohninger and R. Krska, 2003. Development of a
Method for the Determination of Fusarium Fungi on Corn Using
Mid-Infrared Spectroscopy with Attenuated Total Reflection
and Chemometrics, Anal. Chem., 75 (5): 1211-1217.
Liao, K., M. R. Paulsen, J. F. Reid, B. C. Ni and B. E. P.
Maghirang, 1993. Corn kernel breakage classification bymachine vision using a neural network classifier. Transactionsof the ASAE, 36 (6): 1949-1953.
Liggins, II M. , D. Hall and J. Llinas, 2008. Handbook ofMultisensor Data Fusion: Theory and Practice, Second Edition,
CRC Press, 856 pp.
Mahesh, S., D. S. Jayas, J. Paliwal and N. D. G. White., 2008a.
Identification of western Canadian wheat classes at different
moisture levels using near-infrared (NIR) hyperspectral
imaging. CSBE Paper, (08): 196.
Mahesh, S., D. S. Jayas, J. Paliwal and N. D. G. White, 2010b.
Identification of wheat classes at different moisture levels
using near-infrared hyperspectral images of bulk samples.
Sensing and Instrumentation for Food Quality and Safety, 2(3-4): 1007-1015.
Mahesh, S., A. Manickavasagan, D. S. Jayas, J. Paliwal and N.
D. G. White, 2008a. Feasibility of near-infrared hyperspectral
imaging to differentiate Canadian wheat classes. Biosystems
Engineering, 101 (1): 50-57.
Majumdar, S. and D. S. Jayas, 2000a. Classification of cereal grainsusing machine vision, Part 3: Texture models. Transactions of
the ASAE, 43 (6): 1681-1687.
Majumdar, S. and D. S. Jayas, 2000b. Classification of cereal
grains using machine vision: Color Models. Transactions of
ASAE, 43 (6): 1677-1680.
Miralbés, C., 2003. Prediction Chemical Composition and Alveograph Parameters on Wheat by Near-Infrared TransmittanceSpectroscopy. J. Agric. Food Chem, 51 (21): 6335-6339.
Mladenov, M., 2011a. Pattern classification in a noisy environment.
Information Technologies and Control, (1): 23-33.
Mladenov, M. I., 2011b. Grain quality analysis and assessment.
Monograph, Academic publishing of the University of Ruse,
Ruse.
Mladenov, M. and M. Dejanov, 2008. Application of neuralnetworks for seed germination assessment. Proceedings of the
9th WSEAS international conference on Neural Networks. pp.
67–72.
Mladenov, M., S. Penchev, M. Dejanov and M. Mustafa, 2011.
Quality assessment of grain samples using colour image
analysis. Proc. оf тhe 8th IASTED International conference on
Signal Processing, Pattern Recognition and Applications.
Mladenov, M., Ts. Draganova, P. Daskalov, R. Tzonev and M.
Dejanov, 2008. Application of color and texture models andwavelet transformations in seed sowing properties assessment
using computer vision. Agricultural Engineering, 6: 2-7.
Nakakimura, Y., M. Vassileva, T. Stoyanchev, K. Nakai, R.
Osawa, J. Kawano and R. Tsenkova, 2012. Extracellular
metabolites play a dominant role in near-infrared spectroscopic
quantification of bacteria at food-safety level concentrations.
Analytical Methods, 4: 1389-1394.

236
Ng, H. F. and W. F. Wilcke, 1998. Machine vision evaluation ofcorn kernel mechanical and mold damage. Transactions of the
ASAE, 41 (2): 415-420.
Ning, S., R. Ruan, L., Luo, X. Chen, P. L. Chen and R. K.
Jones, 1988. Automation of a machine vision and neural network based system for determination of scabby rate of wheat
samples. ASAE Annual International Meeting, Orlando,
Florida.
Paliwal, J., N. S. Visen and D. S. Jayas, 2003a. Comparison of
a neural network and non-parametric classifier for grain kernel
identification. Biosystems Engineering, 85 (4): 405-413.
Paliwal, J., N. S. Visen and D. S. Jayas, 2001b. Evaluation of
Neural Network Architectures for cereal grain classification
using morphological features. Journal of Agricultural Engineering Research, 79 (4): 361-370.
Paliwal, J., N. S. Visen, D. S. Jayas and N. D. G. White, 2003.
Cereal grain and dockage identification using machine vision.
Biosystems Engineering, 85 (1): 51-57.
Paulsen, M. R. and M. Singh, 2004. Calibration of a near-infrared transmission grain analyzer for extractable starch in corn.
Biosystems Engineering, 89 (1): 79-83.
Paulsen, M. R., L. O. Pordesimo, M. Singh, S. W. Mbuvi andB. Ye, 2003. Corn starch yield calibrations with near-infrared
reflectance. Biosystems Engineering, 85 (4): 455-460.
Pearson, T. C., D. T. Wicklow, E. B. Maghirang, F. Xie and F.
E. Dowell, 2001. Detecting aflatoxin in single corn kernels by

M. I. Mladenov, M. P. Dejanov and R. Tsenkova
transmittance and reflectance spectroscopy. Transactions of theAmerican Society of Agricultural Engineers, 44: 1247-1254.
Peiris, K. H. S., M. O. Pumphrey, Y. Dong, E. B. Maghirang,
W. Berzonsky and F. E. Dowell, 2010. Near-Infrared Spectroscopic Method for Identification of Fusarium Head Blight Damage and Prediction of Deoxynivalenol in Single Wheat Kernels,
Cereal Chem., 87 (6): 511-517.
Ruan, R., Y. Li, X. Lin and P. Chen, 2002. Non-destructive determination of deoxynivalenol levels in barley using nea-infrared spectroscopy. Applied Engineering in Agriculture, 18 (50): 549-553.
Schneider, H. and H. Kutzbach, 1999. Determination of broken kernels in threshing process using image analysis and machine vision.
In: Proceedings of 27th Symposium, Actual Tasks on AgriculturalEngineering, Croatia.
Tsenkova, R., H. Meilina, S. Kuroki and D. Burns, 2010. Near infrared spectroscopy using short wavelengths and leave-one-cowout cross-validation for quantification of somatic cells in milk.
Journal of Near Infrared Spectroscopy, 17 (6): 345-352. 38
Wang, Z. J., J. H. Wang, L. Y. Liu, W. J. Huang, C. J. Zhao and
C. Z. Wang, 2004. Prediction of grain protein content in winterwheat (Triticum aestivum L.) using plant pigment ratio (PPR).
Field Crops Research, 90 (2-3): 311-321.
Wesley, I. J., O. Larroque, B. G. Osborne, N. Azudin, H. Allen
and J. H. Skerritt, 2001. Measurement of gliadin and glutenincontent of flour by NIR spectroscopy. Journal of Cereal Science, 34: 125-133.

Received July, 8, 2014; accepted for printing December, 2, 2014.