Patent Description:
Known methods and systems of inspection use transmission of radiation through a container to determine the nature of its load. It is sometimes difficult to determine the nature of the load, as some different items and/or different compositions appear to be similar on inspection images generated by the known methods and systems of inspection.

A relevant prior art is disclosed in <NPL>, which an automatic detection of potential threat objects in x-ray images of luggages, where objects in the image are being segmented and applied an edge detection process using kernels, where each 64x64 edge map is divided into 16x16 cells and the number of edge pixels in each of these cells is counted based on edge orientation to create a feature vector used for classification.

In the Figures like reference numerals are used to indicate like elements.

Embodiments of the disclosure relate to a method for inspecting a load in a container by extracting one or more texture descriptors from patches of a digitized inspection image of the load, and by classifying the corresponding patches into reference classes of items using the extracted one or more texture descriptors. The classifying may involve comparing the extracted one or more texture descriptors with one or more reference texture descriptors corresponding to the reference classes. The classifying may enable e.g., detection and/or identification of the load, or may enable identification of a composition of at least a part of the load. The extraction of the texture descriptors may enable the detection and/or identification where mere analysis, without using the texture of at least a zone of interest of the image, does not allow discrimination between different items or compositions on the inspection image. The detection and/or identification may particularly be advantageous in cases where the load and/or the composition to be detected and/or identified include high value and/or contraband items and/or compositions, such as cigarettes, bank notes, drugs (such as cannabis or cocaine), medications, pills, beans (such as coffee), etc..

Embodiments of the disclosure use texture descriptors, texture extractors and/or texture classifiers which may allow detection and/or identification of the load and/or its composition, even with a relatively small number of reference texture descriptors in the reference classes (for example a number of an order of magnitude of <NUM>). Alternatively or additionally, embodiments of the disclosure may enable detection and/or identification of the load and/or its composition, even with a relatively high amount of noise and/or diffusion in, and/or a relatively low resolution of, the inspection images. Alternatively or additionally, embodiments of the disclosure may enable identification and/or detection in a relative short period of time, i.e. in an order of magnitude of a second.

Embodiments of the disclosure use classifying one or more zones of the inspection image, and thus may enable identification and/or detection, even when at least a part of the load is superimposed with a screen blocking the transmission of the inspection radiation.

As illustrated in <FIG>, a texture in an image <NUM> is a quasi-periodic or random repetition of basic elements <NUM> having identical macroscopic visual properties. The basic elements <NUM> are called 'texels' (a short version for 'texture elements'). The texture in an image <NUM> of an item is one of the parameters which may be used to identify the item and/or its composition. As a non-limiting example, a texture may be identified in an image of cigarettes, since:.

The texture of the image <NUM> is described by at least one texture descriptor comprising texture features. As described in further detail below, the texture descriptor is extracted from values of pixels <NUM> of the image <NUM> and/or from spatial mutual relationships between the pixels <NUM> of the image <NUM> corresponding to the structure of the image <NUM>.

<FIG> illustrates an analyser <NUM> configured to classify one or more patches <NUM> of one or more digitized inspection images <NUM> of a load <NUM> in a container <NUM>.

Each of the images <NUM> is generated by an inspection system <NUM>.

As will be apparent in more detail below, the analyser <NUM> may be configured to receive the one or more images <NUM> from the system <NUM>, for example over a communication network <NUM> which may be wired and/or may be wireless. The analyser <NUM> conventionally comprises at least a processor and a memory in order to carry out an example method according to the disclosure.

As explained in further detail below in relation to <FIG> and <FIG>, the inspection system <NUM> is configured to inspect the container <NUM> by transmission of inspection radiation <NUM> from an inspection radiation source <NUM> to an inspection radiation detector <NUM> through the container <NUM>.

<FIG> and <FIG> illustrate that the container <NUM> may be a trailer and/or a boot of a vehicle such as a truck, a van and/or a car, and/or may be a shipping container. It is appreciated that the container <NUM> may be any type of container, and thus may be a suitcase in some examples. The radiation source <NUM> is configured to cause inspection of the load <NUM> through the material (usually steel) of walls of the container <NUM>, e.g. for detection and/or identification of high value and/or contraband items and/or compositions, such as cigarettes, bank notes, drugs, medications, pills, beans (such as coffee beans), etc..

The system <NUM> is configured to, in the inspection mode, cause inspection of the container <NUM>, in totality (i.e. the whole container <NUM> is inspected) or partially (i.e. only a chosen part of the container is inspected, e.g., typically, when inspecting a vehicle, a cabin of the vehicle may not be inspected, whereas a rear part of the vehicle is inspected).

In the example illustrated by <FIG>, the inspection system <NUM> may be mobile and may be transported from a location to another location (the system <NUM> may comprise an automotive vehicle), and in the example illustrated by <FIG>, the inspection system <NUM> may be static with respect to the ground and cannot be displaced.

A type of the inspection system <NUM> is characterized by an energy and/or a dose of the inspection radiation <NUM>.

In the examples illustrated by the Figures, the inspection radiation source <NUM> comprises an X-ray generator. The energy of the X-rays may be comprised between 1MeV and 15MeV, and the dose may be comprised between 2mGy and 20Gy (Gray). In the example illustrated by <FIG>, the power of the X-ray source <NUM> may be e.g., between 500keV and <NUM>. 0MeV, typically e.g., 2MeV, <NUM>. 5MeV, 4MeV, or 6MeV, for a steel penetration capacity e.g., between <NUM> to <NUM>, typically e.g., <NUM> (<NUM>. In the example illustrated by <FIG>, the dose may be e.g., between 20mGy and 50mGy. In the example illustrated by <FIG>, the power of the X-ray source <NUM> may be e.g., between 4MeV and 10MeV, typically e.g., 9MeV, for a steel penetration capacity e.g., between <NUM> to <NUM>, typically e.g., <NUM> (<NUM>. In the example illustrated by <FIG>, the dose may be 17Gy.

In the examples illustrated by the Figures, the inspection radiation detector <NUM> comprises, amongst other conventional electrical elements, radiation detection lines <NUM>, such as X-ray detection lines. The inspection radiation detector <NUM> may further comprise other types of detectors, such as optional gamma and/or neutrons detectors, e.g., adapted to detect the presence of radioactive gamma and/or neutrons emitting materials within the container <NUM>, e.g., simultaneously to the X-ray inspection. In the example illustrated in <FIG>, the inspection radiation detector <NUM> may also comprise an electro-hydraulic boom <NUM> which can operate in a retracted position in a transport mode (not illustrated in the Figures) and in an inspection position (<FIG>). The boom <NUM> may be operated by hydraulic activators (such as hydraulic cylinders). In the example illustrated in <FIG>, the inspection radiation detector <NUM> may also comprise a structure and/or gantry <NUM>. The detection lines <NUM> may be mounted on the boom <NUM> (<FIG>) or structure and/or gantry <NUM> (<FIG>), facing the source <NUM> on the other side of the container <NUM>.

In order to inspect the container <NUM>, in the example illustrated by <FIG>, the system <NUM> may comprise a motion generation device so that the system <NUM> may be displaced, the container <NUM> being static (this mode is sometimes referred to as a 'scanning' mode). Alternatively or additionally, the motion generation device may cause the container <NUM> to be displaced, the system <NUM> being static with respect to the ground (<FIG>). Alternatively or additionally, in a 'pass-through' mode the system <NUM> does not comprise a motion generation device and the container moves with respect to the system <NUM>, the system <NUM> being static with respect to the ground.

In some examples, the radiation <NUM> may be transmitted through the container <NUM> (the material of the container <NUM> being thus transparent to the radiation), while the radiation may, at least partly, be reflected by the load <NUM> located in the container <NUM> (the material and/or composition of the load located in the container <NUM> being thus only partly transparent to the radiation <NUM>, and partly reflective to the radiation <NUM> - in that case, detectors may be placed to receive the radiation reflected by the load <NUM>).

The example method illustrated by <FIG> may comprise receiving, at S1, one or more digitized images <NUM> from the inspection system <NUM>. As illustrated in <FIG>, each digitized image <NUM> comprises a plurality of pixels <NUM>. At least a value Ng representative of the transmission of the inspection radiation <NUM> through the container <NUM> is associated with each pixel <NUM>. Each pixel <NUM> is associated with a portion of the space and may be associated with a <NUM> of the load <NUM>.

At S2, the analyser <NUM> classifies one or more patches <NUM> of the digitized inspection image <NUM>. Each of the patches <NUM> is a division of the image <NUM> the image <NUM> being divided into successive patches <NUM>. In some examples, the analyser <NUM> may divide the received image <NUM> into the patches <NUM>. Alternatively or additionally, the analyser <NUM> may receive the image <NUM> from the inspection system <NUM> already divided into the patches <NUM>. The patches may be part of a selection <NUM> of patches <NUM>, such as to reduce processing time by only classifying patches of interest, as explained in further detail below.

In some examples, successive patches <NUM> may be overlapping each other. The overlap between two successive patches <NUM> may take any extent, such as for example the overlap may be comprised between <NUM>% and <NUM>%. It is appreciated that an overlap of <NUM>% generally reduces the number of patches <NUM> to be processed in an image <NUM>, and thus reduces the time of processing, at a cost of a possible less accurate result, whereas an overlap of <NUM>% generally increases the number of patches <NUM> to be processed in an image <NUM>, and thus increases the time of processing, but gives a more accurate result. As an advantageous compromise, the overlap may be equal to <NUM>%.

Each of the patches <NUM> may be of any dimension or form. However, as will be apparent in the specification below, each patch <NUM> may form a square, with a same number of pixels on each side of the patch <NUM>. This may be advantageous in some examples of the disclosure, some of the algorithms requiring square patches. In an example, the number of pixels on each side of the patch <NUM> may be <NUM>, and the dimension of each patch may be then 64x64 pixels.

In some examples, the method may further comprise selecting, for further processing, only patches <NUM> having a mean value Mg (Mg being the mean of the values of the pixels <NUM> of a patch) such that: <MAT> where:.

Pixels <NUM> with values Ng above T2 usually correspond to pixels associated with background <NUM> in the image <NUM>, and pixels <NUM> with values Ng below T1 are usually zones <NUM> too dark for any detection. The selection of the patches <NUM> decreases the number of patches <NUM> to process, and thus saves some processing time. For example, for values Ng encoded on <NUM> bits (from <NUM> to <NUM>), T1 may be for example equal to <NUM> and T2 may be for example equal to <NUM>.

As explained in greater detail below, the method illustrated in <FIG> may comprise, at S3, an optional assessing of a size of a part <NUM> of the load <NUM> corresponding to a plurality of selected patches <NUM> classified in a same class, and only items with parts <NUM> having a size above a threshold size may be processed. This may reduce the number of false alarms. It is appreciated that the size may depend on the item. For example, it might be that parts <NUM> of cigarettes below a threshold corresponding to half the size of a pallet (i.e. approximately <NUM>/<NUM><NUM> in Europe) may not be processed. The threshold size may be smaller in the case of drugs or bank notes for example.

At S4, the method comprises triggering an action based on the result of the classifying performed at S2.

As explained in greater detail below, the action may be:.

In the example illustrated by <FIG>, the classifying performed at S2 may comprise, for one or more patches <NUM>, for example in a selection <NUM> of one or more patches <NUM> as explained in further detail below:.

As already explained, the reference items <NUM> may be, as non-limiting examples, cigarettes, bank notes, drugs (such as cannabis or cocaine), medications (such as pills or tablets), pills, beans (such as coffee beans), etc. There may thus be a class <NUM> for cigarettes, a class <NUM> for bank notes, a class <NUM> for cannabis, etc..

As explained in greater detail below, the reference texture descriptors of each class <NUM> of reference item <NUM> are extracted from one or more reference images <NUM> of the one or more reference items <NUM>, for example inspected by the inspection system <NUM>.

In the example illustrated by <FIG>, the reference texture descriptors Vr, Wr or Pr, for example forming the classes <NUM>, are stored in a reference database <NUM>. The database <NUM> comprises respective reference texture descriptors Vr, Wr or Pr (and thus for example the corresponding classes <NUM>) for each type of inspection system <NUM>.

In the example illustrated by <FIG>, the reference texture descriptors of the class cigarettes <NUM> may be extracted from patches <NUM> of reference images <NUM> of cigarettes, as inspected by the inspection system <NUM>.

It will be appreciated that in examples of the method in accordance with the disclosure, the analyser <NUM> may be configured to retrieve the reference texture descriptors Vr, Wr or Pr from the database <NUM> over a communication network <NUM>, thanks to a communication server <NUM> configured to provide a remote data management system. Alternatively or additionally, the database <NUM> may be at least partially located in the analyser <NUM>.

In the example illustrated by <FIG>, the server <NUM> may also provide access to the database <NUM> to a plurality of geographically distributed analysers <NUM>, over the network <NUM>.

As already stated, in some examples, the database <NUM> may be populated from reference items <NUM> inspected by the inspection system <NUM>.

In the example illustrated by <FIG>, the inspection system <NUM> may thus send reference images <NUM> to the controller <NUM> which may be further configured to extract the corresponding reference texture descriptors and thus populate the database <NUM>, for example during a phase of setting up.

In the example illustrated by <FIG>, one or more inspection systems <NUM> (of same or different types) may also send reference images <NUM> of reference items <NUM> to the controller <NUM> in order to further populate the database <NUM>.

Alternatively or additionally, the analyser <NUM> may also send one or more selected patches <NUM> to the controller <NUM> once they have been classified in a class <NUM>, in order to further populate the database <NUM> (the one or more selected patches <NUM> may be then considered as reference patches). This allows enriching and/or updating the database <NUM>.

It will be appreciated that the disclosure may be applied to detection and/or identification of any type of item and/or composition. However, a non-limiting example for detection and/or identification of cigarettes will now be described.

First the reference classes <NUM> are built, and they may be referred to as class <NUM> for "cigarettes" and class <NUM> for "non-cigarettes".

In order to build the class <NUM>, a set of reference images <NUM> containing only images of cigarettes may be divided into patches <NUM> having a size of 64x64 pixels. This size of patch is advantageous, at least because a pallet of cigarettes usually occupies about <NUM> pixels on the detection lines <NUM> of the inspection system <NUM>. A texture descriptor (such as a vector having texture features as its dimensions, as described in more detail below) may then be extracted for each reference cigarettes patch <NUM>. Examples of methods of extraction of the descriptors will be described in more detail below. The set of all these texture descriptors (such as vectors) is the reference class <NUM>.

Similarly, a set of reference images <NUM> containing only images of non-cigarettes items may be divided into patches <NUM> with the size 64x64 pixels. A texture descriptor may then be extracted for each reference non-cigarettes patch <NUM>. The set of all these descriptors is the reference class <NUM>.

In some embodiments, the method may comprise a validation step of the reference classes and/or of the method by cross-validation. The cross-validation may comprise a step of classifying a subset of one or more patches <NUM> of the reference images <NUM> using the other patches <NUM> of the reference images <NUM>.

Then, when one wants to determine if an inspection image <NUM> contains cigarettes or not, the image <NUM> is divided into patches <NUM> of the same size as the patches <NUM> constituting the two reference classes <NUM> (i.e. 64x64 pixels). Each patch <NUM> is considered as being an object to be classified.

The analyser <NUM> may extract the texture descriptor as for the reference descriptors. Then a classification method may be applied in order to classify the patches <NUM>. Examples of methods of classification will be described in more detail below.

As already stated, the method may comprise classifying one or more overlapping successive patches <NUM>. The overlap between two successive patches may be comprised between <NUM>% and <NUM>%, and may be preferably equal to <NUM>%.

Once the patches <NUM> are classified, the image may be equalized, and an action may be triggered.

The action may comprise determining a composition of the part <NUM> of the load <NUM> corresponding to the selected patch <NUM> of the inspection image <NUM>, such as cigarettes or drugs. The action may also be displaying the digitized inspection image with one or more determined compositions for one or more parts <NUM> of the load <NUM>, as illustrated in <FIG>. The display may use a code of colours for the different items and/or compositions for ease of inspection by a user. An alarm (visual and/or aural) may be issue as a response to the identification and/or detection of an item and/or composition.

Each of the texture descriptor may be a generic descriptor and/or a descriptor dedicated to a load <NUM> and/or a reference item <NUM>.

As explained in greater detail below, the generic descriptor may be extracted using one or more extractors among the following: a response to a bench of filters; a Haralick method implementing a matrix of co-occurrence; and/or a quadrature mirror filter, QMF.

In some examples, the response to a bench of filters may implement, for example, Gabor filters, i.e. filters generated from a Gabor function on a plurality of scales and orientations. Gabor filters may allow extraction of textures, contours, lines and points with different orientations in patches <NUM> and/or <NUM>.

In some examples, each patch <NUM> and/or <NUM> may be filtered by a plurality of bi-directional Gabor filters, which correspond to a convolution with a cosine kernel, weighted by a Gaussian window. The bi-directional Gabor filters may be equivalent to a local Fourier transform using a Gaussian window. The response to a bench of filters may thus enable to filter locally a bandwidth of frequencies.

Then a texture descriptor V may be calculated by convolution of pa by g, g being a Gabor filter such that: <MAT> with g being a Gabor filter in 2D (i.e. on the frequencies (i,j)) such that: <MAT> with: <MAT> <MAT> <MAT> <MAT> and θ and λ being chosen filter parameters.

In some examples, two benches of filters g may be used, with the same parameters θ and λ as the parameters θ and λ set out in the article "<NPL>. In one of the two benches, the Gabor kernel may be symmetrical (Φ=<NUM>), and in the second of the two benches, the Gabor kernel may be anti-symmetrical (Φ=-π/<NUM>). Each bench contains <NUM> Gabor filters of frequency <NUM>/λ (with <NUM>/λ being successively equal to <NUM>, <NUM> and <NUM>) and of orientation θ=kπ/<NUM> (with k being successively equal to <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> and <NUM>).

Therefore in some examples, as a response to each of the two benches of <NUM> filters as described above, a descriptor under the form of a vector V with <NUM> dimensions may be extracted, for each pixel <NUM> in the patch <NUM> and/<NUM> of the image <NUM> or <NUM>. The obtained generic descriptor V may thus comprise <NUM> texture features.

The two vectors V obtained from the symmetric and anti-symmetric benches may also be combined into a single texture descriptor which may be called "Gabor energy" Ve (the Gabor energy may be related to a modelling of human visual cortex and an ability of human neurones to perceive an orientation of contours). The Gabor energy Ve may such that: <MAT>.

Matrixes of co-occurrence (short for "matrixes of co-occurrence of gray levels") may be implemented in a method of Haralick, as described in the article "<NPL>. The matrixes may be based on the statistical analysis of the distribution of the value Ng of the pixels <NUM> (linked to the intensity) in the patch <NUM> and/or <NUM>. Rather than using statistics based on individual pixels <NUM>, the matrixes of co-occurrence use second-order statistics obtained by considering pairs of pixels <NUM>. A co-occurrence matrix P(i, j / d, Θ) (which may be analogous to a bi-directional histogram) illustrates the probability (or number of occurrences) that two pixels <NUM>, remote from each other by a distance d in a direction having an angle Θ, have intensities i and j.

The generic descriptor V obtained by a matrix of co-occurrence may comprise a plurality of texture features, such as contrast, mean and standard variation, etc. In some examples, the matrixes of co-occurrence used in a Haralick method may allow extraction of, e.g., <NUM> texture features for each angle Θ, such as the contrast, the mean and the standard variation already mentioned and:.

Table <NUM> below illustrates an example of an initial patch <NUM> and/or <NUM> with pixels <NUM> having values Ng e.g., comprised between <NUM> and <NUM>:.

Table <NUM> shows a matrix of co-occurrence obtained for (d,Θ)=(<NUM>,<NUM>°).

Therefore, for six angles Θ (such as: <NUM>°, <NUM>°, <NUM>°, <NUM>°, <NUM>°, <NUM>°), each texture descriptor V may thus be a vector having <NUM> dimensions (6x13=<NUM>).

In some examples, the descriptor may comprise a descriptor obtained by QMF. The QMF method may allow a textural analysis, both in the spatial domain and in the frequency domain. Alternatively or additionally, the QMF method may allow consistency of results between different types of systems <NUM>. Alternatively or additionally, the QMF method may allow better analysis for signal having discontinuities or local oddities (e.g., blurred contours).

In examples using the QMF as an extractor, each patch <NUM> or <NUM> forms a square, with a same number of pixels on each side of the patch <NUM> or <NUM>.

A wavelet transform used by the QMF decomposes a patch <NUM> or <NUM> into lower resolution sub-images. Each of the sub-images is called a wavelet coefficient and may highlight information on textures and also on contour, location and orientation in the patch <NUM> or <NUM>.

In the example schematically illustrated in <FIG>, the coefficients may be obtained by using quadrature mirror filters. A pair comprising a digital low-pass filter (H0) and a high-pass filter (H1) may be applied at each level of decomposition on an output of a previous low-pass output. In the case of a patch <NUM> or <NUM>, each level of the wavelet transform by the QMF may be calculated by applying, successively on the lines and on the columns of the pixels <NUM> of the patch <NUM> or <NUM>, the pair of low-pass H0 and high-pass H1 filters mentioned above. In other words, at each level of the wavelet transform, the recursive decomposition has, as an input, the sub-image resulting from a low-pass filter.

H0 in the two spatial directions (horizontal, i.e. lines, and vertical, i.e. columns).

Mathematically, the wavelet transform used in the QMF may decompose a signal using a set of elementary functions generated from a single wavelet (called the 'mother wavelet') by translation and scaling, i.e. by an affine transformation of the 'mother wavelet'. Each of the wavelets has a mean which is equal to <NUM>, and is defined on a compact interval (contrary to a Fourier transform, which uses periodic sinusoid functions defined from [-∞; +∞]).

Let the family of wavelets ψs,u be defined by the mother wavelet ψ such that: <MAT> where.

The wavelet transform transforms a function pa into coefficients cs,u such that: <MAT>.

The generic descriptor may comprise thus one or more texture features. The one or more texture features may comprise the mean m and the standard deviation σ of the coefficients cs,u.

In some examples, the wavelet transform can be implemented efficiently in a pyramid-structured wavelet transform (PSWT) or in a packet wavelet transform (PWT). In the example illustrated in <FIG>, at each decomposition level, the patch <NUM> or <NUM> may be decomposed into four frequency regions LL, LH, HL and HH. The PWT decomposes the signal only in the region of low frequencies LL, while the PSWT decomposes the signal in both low and high frequency regions. As mentioned above, the obtained generic descriptor comprises texture features, such as the mean m of each frequency region, and the standard deviation σ of each frequency region.

The initial patch <NUM> or <NUM> may be decomposed into any number of decomposition scales strictly superior or equal to <NUM>. It is appreciated that the more levels, the more accurate the texture descriptor, since the obtained texture descriptor will comprise more texture features (such as the mean and the standard deviation), at the cost of a longer processing time.

In the example illustrated in <FIG>, the initial patch <NUM> or <NUM> may be decomposed into three decomposition scales, which is a good compromise between the processing time and the number of texture features (i.e. <NUM> features, such as the mean m and the standard deviation σ of LH1, HH1, HL1, LH2, HH2, HL2, LH3, HH3, HL3 and LL3). <FIG> illustrates an intermediate state with only two levels of decomposition of the initial patch <NUM> or <NUM>.

In the examples illustrated by <FIG>, each of the texture descriptor of each patch <NUM> or <NUM> may define a vector Vr or Wr, with each of the texture features of the descriptor Vr or Wr defining a dimension of the vector. For a decomposition into three levels, it is appreciated that each of the texture descriptor Vr or Wr has <NUM> dimensions (noted m1, σ1, m2, σ2,.

In the example illustrated by <FIG>, the n texture descriptors Vr1, Vr2,. , Vrn correspond to respective reference texture descriptors Vri (with i=<NUM>. n in the example of <FIG>). The reference texture descriptors Vri correspond to the class <NUM> of reference for cigarettes <NUM>. The reference texture descriptors Vri of the class of reference cigarettes <NUM> may be extracted from one or more reference images <NUM> of cigarettes <NUM>, inspected by the inspection system <NUM> as illustrated for example by <FIG>.

In the example illustrated by <FIG>, the n texture descriptors Wr1, Wr2,. , Wrn correspond to respective reference texture descriptors Wri (with i=<NUM>. n in the example of <FIG>). The reference texture descriptors Wri correspond to the class <NUM> of reference corresponding to any item <NUM> but cigarettes. The reference texture descriptors Wri of the class <NUM> of reference for any item <NUM> but cigarettes may be extracted from one or more reference images <NUM> of any item <NUM> not containing cigarettes, inspected by the inspection system <NUM>.

The examples illustrated by <FIG> show that the texture descriptors Vr1, Vr2,. , Vrn and Wr1, Wr2,. , Wrn may be stored in the database <NUM>.

The reference classes <NUM> may contain any number n of reference descriptors Vri or Wri. It is appreciated that the more reference descriptors, the more accurate the classification.

The classes <NUM> do not need to contain the same number of descriptors between them.

Each of the patches <NUM> or <NUM> may have any number of pixels <NUM>, such as 32x32 pixels or 128x128 pixels. It is appreciated that the more pixels <NUM>, the more accurately the obtained descriptor, since the descriptor may comprise more pixels <NUM> in each of the regions for the purpose of the calculation of the mean m and the standard deviation σ, even in the regions LH3, HH3, HL3 and LL3, at the cost of a longer processing time. In the example illustrated in <FIG>, the initial patch has 64x64 pixels, which is a good compromise between the processing time and the number of pixels in the LH3, HH3, HL3 and LL3 regions (i.e. 8x8 pixels in the regions LH3, HH3, HL3 and LL3 of <FIG>) for the calculation of mean m and the standard deviation σ in the corresponding regions.

In some embodiments, texture descriptors dedicated to a load <NUM> and/or a reference item <NUM> (such as cigarettes, as a non-limiting example) may be created.

In some examples, the dedicated descriptor may comprise one or more descriptors such as:.

As a non-limiting example, the above-mentioned dedicated descriptors may describe advantageously patches <NUM> or <NUM> corresponding to cigarettes, as explained below.

As shown in the example illustrated by <FIG> (on the left-hand side), on a vertical strip of the patches <NUM> or <NUM>, the grey level is either uniform or decreases gradually. This is because, as seen the example illustrated by <FIG>, the radiation <NUM> usually passes through less material at the top of the container <NUM>, which is shown on the patches <NUM> or <NUM>. The right-hand side of <FIG> shows that the created dedicated descriptor Pr, such as the profile P corresponding to a projection of the values Ng of the pixels <NUM> of the selected patch <NUM> on an axis (such as a horizontal axis), may be resilient to diffusion.

Furthermore, as shown in the example illustrated by <FIG> (on the left-hand side), the patches <NUM>/<NUM> may be impacted by noise. The right-hand side of <FIG> shows that the created descriptor P, such as the profile P corresponding to a projection of the values Ng of the pixels <NUM> of the patches <NUM> or <NUM> on an axis (such as a horizontal axis), may be resilient to noise.

Clustering may be advantageously used with dedicated texture descriptors, such as the profiles P described above.

Finding clusters C may aim to extract the profiles which are the most characteristics of the reference items <NUM> (such as cigarettes as a non-limiting example), and thus may aim to obtain patterns M for the reference items <NUM>.

In some examples, the clustering may use K-means. As illustrated in <FIG>, the profiles P are inherently periodic, and only the phase varies between them. Accordingly, a measure c of correlation of the patterns, which is invariant to translation of the patterns, may be used: <MAT> <MAT> where P1 and P2 are the profiles of two patches, for example two patches <NUM>. The centre of the clusters C may then be defined as the mean of the corresponding profiles P, after a translation maximizing the correlation c. An idea of the patterns M extracted by clustering is illustrated by <FIG>.

The advantages of clustering may include:.

In the example illustrated by <FIG>, the classifying of the patches S22 may comprise using any one of the following classifiers:.

In the example illustrated by <FIG>, S22 may comprise a classifier using SVM classification as for example described in the article "<NPL>, and/or in the article "<NPL>. The SVM may aim to predict a class to which a texture descriptor, such as a vector, belongs to. The SVM may use:.

During the learning step, the SVM may aim to find an optimal separator SP (such as a hyperplane) between the classes <NUM> (such as a separation SP between the class <NUM> for cigarettes and the class <NUM> for non-cigarettes) based on the learning vectors. The SVM may aim to find the optimal separator SP while the distance (also called margin) between the hyperplane SP and the nearest example is kept at a maximum. The search for the maximum margin hyperplane SP corresponds to an optimization problem in which support vectors (i.e. in the examples of <FIG>, the closest learning vectors to the hyperplane SP) are selected, as explained below.

The SVM may be used, as a non-limiting example, for a binary classification such as cigarettes/non-cigarettes, as explained below.

Let D be a training set of vectors such that: <MAT> where Vi is the descriptor (vector) of a patch <NUM> in the space Rn of the texture features generated from S21; and
yi ∈ {<NUM>,-<NUM>} is the reference class <NUM> of Vi, i.e. (<NUM>) for cigarettes and (-<NUM>) for non-cigarettes.

The SVM process may start by injection of Vi in a Hilbert space F, F having a high dimension, by a non-linear application Φ such that: <MAT>.

First, let us consider that the data is linearly separable in F. Thus there exists a vector ω <MAT> and a scalar b <MAT> such that: <MAT>.

Thus, the SVM may build the separator hyperplane SP defined by the equation: <MAT> and which maximizes the error margin as illustrated by <FIG>.

It can be shown that the vector w defining the optimal hyperplane SP is defined by: <MAT>.

The vectors Vi for which αi≠<NUM> are called support-vectors.

The scalar product in F can be replaced by a kernel function K in Rn such that: <MAT>.

In the example illustrated in <FIG>, the separator SP illustrated in <FIG> is thus better than the separator SP illustrated in <FIG>.

However, the data may not be linearly separable in any of the spaces of characteristics generated in S21, and in some examples a Gaussian or a polynomial kernel may be used.

The SVM may use functions as set out in a libsvm library by C. Chang and C. Lin, available online at:
http://www. tw/~cjlin/libsvm/.

In some examples, the Gaussian kernel may be such that: <MAT> where τ may be obtained by k-fold cross-validation.

In some other examples, the polynomial kernel function may be: <MAT> where.

The determination of the parameters (y,v) may be performed by cross-validation. The cross-validation may comprise dividing the available data into multiple subsets (e.g., <NUM>), and evaluating the SVM on each subset using the remaining data as a learning set. The evaluation may be performed several times by varying (y,v), then keeping the best couple (y,v).

Once the classifier (such as the separator SP) is determined, to classify a texture descriptor (vector) of a patch <NUM>, the learning vectors used in the search of the hyperplane SP are no longer useful, only the found support vectors are used. The SVM method may be thus effective in terms of complexity and processing time.

In some examples, a classifier may use a k-NN, which may aim to assign a texture of a patch <NUM> represented by a vector V to the most representative class <NUM> Vri in its immediate vicinity, as illustrated in <FIG>. The k-NN method can be formally defined by: <MAT> <MAT> where Si(x) is the number of neighbours of V belonging to the class Vri, among its k nearest neighbours.

The k-NN method may thus select the k closest learning vectors Vri or Wri to the vector V to be classified. The technique of classification by k-NN may thus be a method of classification based on distances. The k-NN classifier may use any type of distance, such as the L1 distance (Manhattan), <MAT> the L2 distance (Euclidian) <MAT> and/or the L3 distance (Minkowski) <MAT> with p=<NUM>.

It is appreciated that other values of p may be used.

The patch <NUM> will then be classified in the class <NUM> of the majority of the k-nearest neighbours. The choice of k may depend on the images, since a large value of k may reduce noise but may also make the decision borders less distinctive. k is an odd integer. An optimal k may be estimated by cross-validation, and k may be preferably equal to <NUM>.

In the example illustrated by <FIG>, it may thus be determined whether the vector V of the patch <NUM> of <FIG>, to be classified, has, among its <NUM> closest neighbours (using for example L3):.

In some examples, a classifier may use a Bayesian inference. In the example illustrated in <FIG>, for each patch <NUM> to be classified, the correlation with each pattern M of the bench of filters is determined, and the best response Br is selected. If the selected best response Br goes over a threshold T, then it will be classified in a corresponding class <NUM>, for example as cigarettes, i.e. class <NUM>. If the selected best response Br does not go over the threshold T, it will be classified as non-cigarettes, i.e. class <NUM>.

The classifier using Bayesian inference may be based on the principle of the classifiers using margins, as illustrated in <FIG>. In other words, the distance of the selected best response Br to the threshold T determines a value of confidence Vc of the classification, i.e. the further the selected best response is from the threshold T, the less the classifier is certain of its decision.

In some examples, the thresholds T may be determined by Bayesian inference for each pattern M as explained below.

Let Mi be the ith pattern, and paik the k-th patch of all the patches <NUM> of the learning set, for which Mi has obtained the best responses Brik.

The decision threshold Ti is the one which minimizes the rate of wrong classification. or <MAT> <MAT> where G1 and G2 correspond to all the patches <NUM> cigarettes and non-cigarettes, respectively.

<FIG> illustrates a patch <NUM> to be classified and its profile P. As already stated, <FIG> illustrates an example of an obtained bench of filters f, the patterns Mi as extracted by clustering, and the decision thresholds Ti. <FIG> illustrates the classification of the patch <NUM> and the determination of the value of confidence VC.

The classification of the patches <NUM> as performed in S21 and S22 is performed locally, i.e. that any selected patch <NUM> is classified using only its own texture descriptor. In other words, at the end of the classification S22, there may be isolated patches <NUM> falsely detected as cigarettes (sometimes referred to as "false positives" <NUM>, as illustrated in the example of <FIG>), but also patches "cigarettes" not recognized as such in large zones of cigarettes.

In the example illustrated in <FIG>, the classifying may thus further comprise classifying, at S23, one or more zones <NUM> of patches <NUM>, each zone <NUM> comprising a plurality of selected patches <NUM> using an uniformization algorithm. The uniformization may take into account zones <NUM> of patches <NUM> to determine their class.

In some examples, the uniformization algorithm may be at least one of the following:.

In a binary morphological closing, elements which are both connected to classified patches, e.g., cigarettes patches, and below a certain size may be classified in the same connected class.

In a segmentation process, the image is first segmented into zones <NUM>, and each zone <NUM> is classified based on a ratio ra, such that ra=(number of patches "cigarette"/number of patches "non-cigarettes") in the zone <NUM>.

A regularization type of process may be based on the assumption that a patch <NUM> surrounded by patches "cigarettes" is very likely to be also a patch "cigarettes".

In some examples, the regularization may be based on at least one of the following:.

In some examples, the image <NUM> of <FIG> containing the classified patches <NUM> may be modified into an intermediate modified image, so that the intermediate modified image may show a value of confidence (such as a distance to the nearest neighbour in the case of k-NN, or a distance to the hyperplane SP in the case of the SVM), indicating how sure the classifier is about its classification. In some examples, the values of confidence may be probabilities that a patch <NUM> is, for example, cigarettes or non-cigarettes.

Once the initial image <NUM> has been modified in a probabilistic space in the intermediate modified image, Markov fields may be applied to measure a probability of an overall final result image <NUM>, as illustrated in <FIG>, e.g., by using a local probability of each patch <NUM>, using only the information of each patch's neighbourhood.

The probability of I will thus be: <MAT>.

This shows that it is unlikely that a "cigarettes" patch is in the middle of "non-cigarettes" patches and vice-versa.

As illustrated in the example of <FIG>, the final result image <NUM> may be the most probable image <NUM>.

Alternatively or additionally, the images <NUM> and/or <NUM> may be processed using Gibbs fields.

Alternatively or additionally, once the initial image <NUM> has been modified in a probabilistic space in the intermediate modified image, Gibbs fields may be applied to measure a probability of an overall final result image <NUM>, as illustrated in <FIG>. In some examples, Gibbs fields may have a smaller computational cost compared to Markov fields. A regularization type of process may minimize an Energy function. An Energy function U of Gibbs may be such that: <MAT> <MAT>.

Maximizing the likelihood of an image may amount to minimizing the global energy U. There may be several methods to minimize the global energy.

In some examples minimizing the global energy may be performed using an Ising model.

Let <MAT> the binary image B generated from the classification, as illustrated in <FIG>, and <MAT> the image I (referred to as image <NUM> in <FIG>) of probability indicating the confidence in the classification of the patch <NUM> (referred to as "pa"). Considering the <NUM>-connexity as a neighbourhood, there are only first-order potentials (i.e. the patch <NUM> "pa" itself) and second-order potentials (i.e. the patch <NUM> "pa" and its neighbour N(pa)). The potentials U are defined as follows: <MAT> <MAT> otherwise <MAT> <MAT> and the global energy to minimize is <MAT> where.

When β is positive, the most probable configurations (i.e. with the lowest energies) are those for which neighbouring patches are of the same class <NUM>. The absolute value of β determines the strength of regularization of the model.

In some examples minimizing the global energy may be performed using an Energy function.

In some examples, a simulated "annealing" may be used, for example for a Gibbs field.

In some examples, a Gibbs distribution may be introduced with a temperature TO, such that: <MAT>.

The algorithm may build a sequence of images thanks to a sampler (such as the "sampler of Metropolis") which may converge towards the distribution PTO. A subsequent lowering of the temperature TO in the algorithm may ensure that the last image generated by the sampler may be a global minimum of the image.

Alternatively or additionally, other algorithms, such as iterated conditioned modes, may also be used during the regularization.

The action may be triggered using the result image <NUM>. Regularization may enable to cancel false positives and/or false negatives <NUM> in the final image as illustrated in <FIG>.

Regularization may also enable identification and/or detection of items and/or compositions, even when at least a part of the load is superimposed with a screen blocking the transmission of the inspection radiation.

Other variations and modifications of the system or the analyser will be apparent to the skilled in the art in the context of the present disclosure, and various features described above may have advantages with or without other features described above.

For example, the analyser <NUM> may, at least partly, form a part of the inspection system <NUM>.

It is understood that the inspection radiation source may comprise sources of other radiation, such as gamma rays or neutrons. The inspection radiation source may also comprise sources which are not adapted to be activated by a power supply, such as radioactive sources, such as using Co60 or Cs137.

As one possibility, there is provided a computer program, computer program product, or computer readable medium, comprising computer program instructions to cause a programmable computer to carry out any one or more of the methods described herein. In example implementations, at least some portions of the activities related to the analyser <NUM> and/or the communications networks <NUM> and/or <NUM> herein may be implemented in software. It is appreciated that software components of the present disclosure may, if desired, be implemented in ROM (read only memory) form. The software components may, generally, be implemented in hardware, if desired, using conventional techniques.

In some examples, components of the analyser <NUM> and/or the communications networks <NUM> and/or <NUM> may use specialized applications and hardware.

As will be apparent to the skilled in the art, the server <NUM> should not be understood as a single entity, but rather refers to a physical and/or virtual device comprising at least a processor and a memory, the memory may be comprised in one or more servers which can be located in a single location or can be remote from each other to form a distributed network (such as "server farms", e.g., using wired or wireless technology).

In some examples, one or more memory elements (e.g., the database <NUM> and/or the memory of the processor) can store data used for the operations described herein. This includes the memory element being able to store software, logic, code, or processor instructions that are executed to carry out the activities described in the disclosure.

A processor can execute any type of instructions associated with the data to achieve the operations detailed herein in the disclosure. In one example, the processor could transform an element or an article (e.g., data) from one state or thing to another state or thing. In another example, the activities outlined herein may be implemented with fixed logic or programmable logic (e.g., software/computer instructions executed by a processor) and the elements identified herein could be some type of a programmable processor, programmable digital logic (e.g., a field programmable gate array (FPGA), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM)), an ASIC that includes digital logic, software, code, electronic instructions, flash memory, optical disks, CD-ROMs, DVD ROMs, magnetic or optical cards, other types of machine-readable mediums suitable for storing electronic instructions, or any suitable combination thereof.

The communications network <NUM> and the communications network <NUM> may form only one network.

The data received by the analyser <NUM> may be typically received over a range of possible communications networks <NUM> and/or <NUM> at least such as: a satellite based communications network; a cable based communications network; a telephony based communications network; a mobile-telephony based communications network; an Internet Protocol (IP) communications network; and/or a computer based communications network.

Claim 1:
A method for inspecting a load (<NUM>) in a container (<NUM>), comprising:
receiving (S1) a digitized image (<NUM>) from an inspection system (<NUM>) of a type of inspection system, wherein the type of inspection system (<NUM>) is characterized by an energy and a dose of the inspection radiation (<NUM>),
wherein the digitized image (<NUM>) was generated by the inspection system (<NUM>) by transmission of inspection radiation (<NUM>) from an inspection radiation source (<NUM>) of the inspection system (<NUM>) to an inspection radiation detector (<NUM>) of the inspection system, through the container (<NUM>), and
wherein the digitized image (<NUM>) is divided into a plurality of successive patches (<NUM>), each patch (<NUM>) of the plurality of patches (<NUM>) being a division of the image (<NUM>); and classifying (S2) one or more patches (<NUM>) of the plurality of successive patches (<NUM>), J
wherein the classifying (S2) comprises:
extracting (S21) one or more texture descriptors (V, P) of a patch (<NUM>),
wherein the one or more texture descriptors (V, P) are configured to describe a texture in the patch (<NUM>), the texture being a repetition of texels (<NUM>) having identical macroscopic visual properties, and
wherein the texture descriptor is extracted from values of pixels (<NUM>) of the patch (<NUM>) and/or from spatial mutual relationships between the pixels (<NUM>) of the patch corresponding to the structure of the patch (<NUM>), and
classifying (S22) the patch (<NUM>), by comparing the one or more extracted texture descriptors (V, P) of the patch (<NUM>) to one or more reference texture descriptors (Vr, Wr, Pr) corresponding to one or more classes (<NUM>) of reference items (<NUM>),
wherein the one or more reference texture descriptors (Vr, Wr, Pr) of each class of reference items (<NUM>) has been extracted from one or more reference images (<NUM>) of the one or more reference items (<NUM>) on a type of inspection system corresponding to the inspection system (<NUM>) generating the digitized image,
wherein the one or more reference texture descriptors (Vr, Wr, Pr) corresponding to one or more classes of reference items (<NUM>) are stored in a reference database (<NUM>), and
wherein the reference database (<NUM>) comprises one or more reference texture descriptors (Vr, Wr, Pr) for each type of inspection system, comprising said same type of inspection system (<NUM>) and a different type of inspection system.