Patent Description:
Various methods for measuring the size distribution of small particles exist. These methods include using sieves, sedimentometry, laser diffraction, dynamic light scattering, and use of a microscope. Using sieves involves separating particles and measuring their size distribution based on varying mesh sizes of sieves. Sedimentometry, which is measurement using a sediment meter, measures the rate particles fall through a viscous medium, which rate is then correlated to particle size. Laser diffraction and dynamic light scattering use laser light directed at the particles. For laser diffraction, particle size distribution is determined based on the diffraction pattern of light scattered by the particles; for dynamic light scattering, particle size distribution is determined based on the changes in intensity of light scattered by particles in a solution. Microscopes can be used to directly measure particle sizes.

A problem with the above methods is that they are laborious and time-consuming, require specialized and expensive equipment such as lasers or an appropriate microscope, or both. Each method also typically requires trained personnel to use the equipment accurately. Such requirements limit these methods to industrial and laboratory applications. A method for measuring the size distribution of small particles that could be more readily used in a casual environment (e.g., in an average home) by individuals would be desirable.

The document "<NPL> discloses a review of the entire line of Baratza grinders. It puts the grinders "through a variety of tests, to come up with an experimental scoring system (subject to review and changes) to properly gauge the grinders". In this document, photographs showing grind sizes in comparison to other items are provided to help determine correct grind particle sizes for various grinding needs.

The document "<NPL> discloses a website post about coffee grinders.

The document "<NPL> discloses a method of measuring the grain sizes of river gravel using a screen and a camera. The automated procedure is divided into four keys stages: (<NUM>) image collection; (<NUM>) image preprocessing; (<NUM>) image processing and analysis; and (<NUM>) the derivation of a grain-size distribution. Regarding the image collection, data input to the procedure is an image of a patch of sediment collected approximately vertically with a digital camera. Reference points must be placed at each corner of the rectangular sample patch to provide a scale and define the boundary of the patch in the image. Regarding the image preprocessing, a radial lens correction and a perspective effect correction are applied to the image. The grain-size distribution obtained by the automated procedure is compared with the grain-size distribution obtained from sieve control data.

<CIT> relates to a method for measuring a size distribution of particles, in particular for optical online measurement of individual particles of bulk materials having wide particle-size distributions. In this document, at least one process parameter of the particle production method can be subjected to open-loop and/or closed-loop control such that it causes the size distribution of the particles to be changed. It is thus possible to regulate an actual size distribution toward a predetermined set point size distribution.

In one aspect, the inventive concept pertains to a computer-implemented method of determining a particle size distribution according to claim <NUM>.

In another aspect, the inventive concept pertains to a non-transitory computer-readable medium according to claim <NUM>.

A method for determining the size distribution of small particles using an image of such particles is disclosed. The method entails generating or otherwise obtaining an image (e.g., a bitmap) that includes the small particles and a calibration pattern, wherein the dimensions of the calibration pattern are known. The image is produced using any conventional camera to image the small particles positioned on or around a calibration pattern. Where particle size is small (e.g., in the order of <NUM>-<NUM> inches), corrections are made to account for distortions caused by the camera lens.

In the method that is disclosed, particle size is determined using a calibration pattern that includes marks of known dimensions. The calibration pattern and the particles are imaged together, with the assumption that the same distortions will apply to both the calibration pattern and the particles. Using the known dimensions of the calibration pattern, a transformation is generated for removing the distortion effects and converting the image to a corrected image that is free of distortion effects. The transformation is then applied to determine particle size distribution from the corrected image.

Advantageously, any camera can be used to record the bitmap, and specialized imaging or recording equipment is not required. Furthermore, because the calibration pattern and particles are recorded together in a single image, the camera, lens and other image parameters do not need to be known to correct for distortions and extract accurate particle sizes. Capturing the particles and the calibration pattern together in one image eliminates the need to record additional, separate calibration images to obtain a set of parameters to correct for distortions before recording the measurement image. In the method that is disclosed, the information necessary to accurately determine the sizes of the particles is embedded in a single image.

<FIG> illustrates an embodiment of an apparatus <NUM> that may be used to implement the particle size determination technique disclosed herein. As shown, the apparatus <NUM> includes a camera <NUM> that is positioned to capture an image of particles <NUM> lying on a calibration pattern <NUM>. As shown, the camera <NUM> stores the captured image in a memory <NUM> and communicates with a processor <NUM>, either by direct connection or via a network. In one embodiment, the camera <NUM> may be part of a mobile device such as a smart phone, tablet, or a laptop. A "camera," as used herein, is intended to mean any imaging device capable of generating an electronic image (e.g., a bitmap) of physical objects using one or more lenses.

A stand [not shown] may be used to hold the camera <NUM>, although the stand is not required and the camera <NUM> may be held by a person. The distance of the camera <NUM> from the calibration pattern <NUM> does not need to be known or specifically set, and should be in a range that allows the calibration pattern <NUM> and particles to be distinguished with appropriate pixel coverage of calibration marks and particles. The distance may be, for example, about <NUM> inches when using an iPhone® manufactured by Apple Inc.

<FIG> depicts an example embodiment of the calibration pattern <NUM>. The calibration pattern <NUM> includes a background <NUM> and calibration marks <NUM>. In the example calibration pattern <NUM> shown in <FIG>, the calibration marks <NUM> are the outlines of squares having consistent dimensions x and y, which are drawn with lines of a substantially constant thickness and repeated at a consistent interval w in rows and columns. The dimensions of the calibration marks <NUM> are known (e.g., by measuring the physical pattern). The calibration marks <NUM> can be of any dimensions that provide enough information for producing a corrected image that allows accurate measurement in the desired particle size range.

The calibration pattern <NUM> is most useful if the main color or shade of the background <NUM> has a high contrast to the color of the particles to be measured and the color of the calibration marks <NUM>. It is also useful to have the color of the calibration marks <NUM> be different than the color of the particles whose size is to be determined. For example, if the particles are brown or black, the background <NUM> may be white and the calibration patterns <NUM> may have a blue hue. In general, any material may be used for the calibration pattern <NUM>, and it may be useful if the surface of the calibration pattern <NUM> is impervious to the material of the particles, so that the particles do not damage the calibration pattern <NUM>. In one example, the calibration pattern <NUM> is a pattern printed on a sheet of paper and the particles are sprinkled on it. A digital image of the calibration pattern <NUM> may be obtained by the user and printed at home.

The smallest particle size that can be determined using the technique disclosed herein depends on a number of factors, one of which is the ratio between the smallest measurable object's pixel coverage and the calibration mark's pixel coverage. There also should be enough camera resolution to capture one or more calibration marks and enough particles of the sample to gather statistics. The camera that is part of an iPhone ® that is currently available in the market would allow determination of particle size as small as <NUM> × <NUM>-<NUM> cm (<NUM> × <NUM>-<NUM> inches). In one embodiment, the dimensions of the calibration marks <NUM> depend on the digital resolution of the camera and the size of the smallest particle to be measured. As shown for example in <FIG>, for a camera having a resolution of <NUM> × <NUM> pixels, a square <NUM> of <NUM> pixels by <NUM> pixels is needed to measure a particle with a diameter of <NUM> × <NUM>-<NUM> cm (<NUM> × <NUM>-<NUM> inches), and a calibration pattern <NUM> that is <NUM> pixels per side (i.e., surrounds an area of <NUM>,<NUM> pixels) is used. In the apparatus shown in <FIG>, such a calibration pattern <NUM> would include calibration patterns <NUM> that are <NUM> inch square.

Patterns other than the repeating squares shown in <FIG> may be used for the calibration patterns <NUM>. However, using a regular pattern that is mathematically easy to model, e.g., that includes orthogonal lines, for the calibration patterns <NUM> may simplify the processing of the recorded image. Additionally, as shown in <FIG>, the calibrations mark <NUM> uses maximum use of the color that has a high contrast to the color of the particles whose size is being determined. A pattern such as a "checkerboard" pattern, for example, may not work as efficiently because half of the boxes would be dark and, where the particles are dark, too many of the particles would lie on the dark areas where there is not much contrast between the particles and the surface they are on.

<FIG> shows an example of particles <NUM> on the calibration pattern <NUM>. To take the measurement, a sample of the particles to be measured is randomly placed (e.g., sprinkled) on the calibration pattern <NUM>. It may be desirable to make the calibration marks <NUM> a complementary color to the particles <NUM> whose sizes are being determined. For example, if particles <NUM> are blue, the calibration marks <NUM> may be red.

The method of determining the particle size distribution from the image is described with respect to <FIG> and <FIG>.

<FIG> illustrates an embodiment of a method for determining particle size. After the particles to be measured are distributed on the calibration pattern <NUM>, the camera <NUM> is used to capture a digital image (e.g. record a bitmap) of the particles on the calibration pattern <NUM> (step S510). The Image generation may include preparation of the image to facilitate object recognition, for example by applying image processing techniques (including, but not limited to noise reduction and/or thresholding and/or filtering) to improve the clarity/definition of the image. Any suitable noise reduction techniques may be used.

Any technology for capturing or generating two-dimensional or three-dimensional images may be used in step S510, including techniques that utilize photographic, ultrasound, x-ray, or radar. The inventive concept is not limited to any particular way of capturing the bitmap.

At step S520, the image of calibration pattern <NUM> is corrected to remove distortions. One of the problems of using imaging for determining particle size is that various distortions in the recorded image will lead to inaccuracies in the size determination. These distortions are particularly deleterious when trying to make accurate measurements of small particles, in the size ranges of, for <NUM> × <NUM>-<NUM> cm (<NUM>-<NUM> inches). Such distortions can be the result of imperfections in lens geometry or alignment, which may cause, for example, straight lines to be captured in the image as non-straight lines. Distortion can also be the result of perspective distortion, which occurs when the camera's optical axis is not perpendicular to the center of the object being imaged, causing parallel lines to appear non-parallel in the image. As particle sizes are determined using the dimensions of the calibration marks <NUM>, distortion in the image of the calibration marks <NUM> will likely result in error in size determination. Hence, distortions in the calibration marks <NUM> are corrected before particle size determination is done. At the end of step S520, a "corrected image" is produced that includes the distortion-free calibration marks <NUM> and the original (uncorrected) image of the particles.

In one approach to removing distortions in step S520, calibration marks <NUM> are extracted from the image. A domain filtering technique may be used for this extraction. For example, where the calibration marks <NUM> are blue, a technique that uses a chroma range to extract patterns in the blue hue range may be used (the background <NUM> of the calibration pattern <NUM> is a different color from the marks <NUM>). The result is a calibration mark object array that is built by extracting contours from the blue portions of the image. The calibration mark object array is used to generate a uniform scale that is useful for mapping the image, as in orthophotographs. This generation of a uniform scale is an ortho-transformation, which may be reused with the same calibration pattern <NUM> repeatedly once it is generated.

At step S530, particle sizes are determined using the corrected image of the calibration pattern <NUM>. This process entails extracting the particles from the corrected image using domain filtering. For example, domain filtering may be accomplished using a chroma range to extract the particles that are known to be in the brown/black color range. A particle object array is generated by extracting contours of the particles from the processed image. Each element of the particle object array can then be measured to determine particles dimensions, for example the extent (diameter), area and circularity of each particle. From these measurements, a size distribution of the measured particles is obtained. In measuring the particle sizes, the calibration pattern acts as a dimensional reference.

As mentioned above, the dimensions of the calibration marks <NUM> in the physical world (as opposed to in the image) are known. Optionally, the accuracy of the calibration mark distortion correction used in S520 can be cross-checked by measuring the calibration marks in the corrected image and calculating the discrepancy from known sizes of the calibration marks <NUM> in the calibration pattern <NUM>.

As step S540, the measurements of the particle object array are characterized. A size distribution histogram, an area distribution histogram, volume distribution histogram, minimum, maximum and standard deviations and distribution peak analysis may be used to determine a set of parameters that characterize the size and shape of the particles. A "set" of parameters, as used herein, is intended to mean at least one parameter.

The particle size determination is made with the objective of processing (grinding) the particles to achieve a desired size. A profile of target particles is available, and the profile may be defined in terms of the set of parameters that is used in step S540. Based on a comparison of the measured parameters (result of step S540) against the profile of the target particles, one determines may what action needs to be taken to bring the two profiles closer. This determination is made automatically by a processor. Where the determination is made automatically, the processor indicates to an operator (e.g., visually and/or by sound) that further grinding would bring the measurement closer to the target profile.

One possible application of the particle size determination method described above is in preparing coffee, particularly in grinding coffee beans. It is known that the taste of a coffee drink is affected by several parameters, one of which is the fineness or the coarseness of the coffee grinds. A user who desires to make coffee of a certain flavor may obtain the profile of target coffee grinds that he knows would produce the flavor he wants when used with a specific setting in his coffee maker. Such user could sprinkle some coffee grinds on the calibration pattern <NUM> and use the above method to obtain the set of parameters that characterize his grinds, then compare the parameters to the target profile. For example, if the set of parameters indicate that measurement distribution for his grinds is centered on <NUM> inch and the target profile is for <NUM> inch, the user would know that he ground the beans too fine and that he needs to restart with a coarser preparation.

Now, further details will be provided about the distortion correction step S520 of <FIG>, and particularly about the generation of a transformation matrix for converting the original, distorted image to a distortion-free corrected image. The actual position of the calibration patterns is known from the physical, real-world calibration pattern. So, a comparison of the two (i.e., the actual position and the position in the image) can be used to determine a transformation matrix (or any other scalar) that converts the imaged calibration pattern back to the actual calibration pattern. This transformation matrix for the calibration pattern shows how to convert the image to real world measurements, and vice versa. The transformation matrix may be extrapolated across the set of pixels and applied to either the whole image or to a selected portion of the image in which, again using a chroma range extraction, just the particles are shown (obtained in step S530). After application of the transformation matrix to the image, a partially-corrected image that is less distorted (and perhaps substantially free of perspective or geometric distortions) is obtained. This approach generates a correction factor to each calibration mark <NUM> to correct for distortion.

<FIG> depicts another approach <NUM> for distortion correction. Unlike the first approach using the empirically-generated transformation matrix, this second approach entails correcting the image for lens distortion and perspective distortion. Using the calibration mark object array described above and regions of the image referenced by the calibration mark object array at steps S610 and S620, distortion coefficients are obtained. For example, a method described in the calibration method of <NPL>, may be used. The method described in Zhang uses a checkerboard pattern to obtain calibration parameters that can be used in subsequent photographs. For the inventive concept disclosed herein, however, the calibration pattern <NUM> extracted from the image as described above is used. The calibration result obtained will include the lens distortion parameters as k<NUM> and k<NUM>.

The technique proposed by Zhang uses the camera to observe a planar pattern shown at a few (at least two) different orientations. The pattern can be printed on a laser printer and attached to a "reasonable" planar surface (e.g., a hard book cover). Either the camera or the planar pattern may be moved and it is not necessary for the details of the motion to be known. The proposed approach lies between the photogrammetric calibration and self-calibration because 2D metric information is used rather than 3D or purely implicit one. Both computer simulation and real data have been used to test the proposed technique.

The technique described herein advances 3D computer vision from laboratory environments to the real world.

Constraints on the camera's intrinsic parameters are provided by observing a single plane. A 2D point is denoted by m = [u,v]T. A 3D point is denoted by M = [X, Y, Z]T. The symbol ~ denotes the augmented vector by adding <NUM> as the last element: m̃= [u, v, <NUM>]T and M̃ = [X, Y, Z, <NUM>]T. A camera is modeled by the usual pinhole: the relationship between a 3D point M and its image projection m is given by: <MAT> where s is an arbitrary scale factor; (R, t), called the extrinsic parameters, is the rotation and translation which relates the world coordinate system to the camera coordinate system; and A, called the camera intrinsic matrix, is given by: <MAT> with (u<NUM>, υ<NUM>) the coordinates of the principal point, α and β the scale factors in image u and υ axes, and γ the parameter describing the skewness of the two image axes. The abbreviation A-T is used to represent (A-<NUM>)T or (AT)-<NUM>.

The model plane may be assumed as being on Z = <NUM> of the world coordinate system. The ith column of the rotation matrix R will be denoted as ri. From Equation (<NUM>), we have: <MAT>.

The symbol M is still used to denote a point on the model plane, but M = [X, Y]T since Z is equal to <NUM>. In turn, M~ = [X, Y, <NUM>]T. Therefore, a model point M and its image m is related by a homography H: <MAT>.

As is clear, the <NUM> × <NUM> matrix H is defined up to a scale factor.

Given an image of the model plane, a homography can be estimated. Denoting the homography by H = [h<NUM> h<NUM> h<NUM>], from Equation (<NUM>), we have <MAT> where λ is an arbitrary scalar. Using the knowledge that r<NUM> and r<NUM> are orthonormal, the following constraints are obtained: <MAT> <MAT>.

These are the two basic constraints on the intrinsic parameters, given one homography. Because a homography has <NUM> degrees of freedom and there are <NUM> extrinsic parameters (<NUM> for rotation and <NUM> for translation), <NUM> constraints are obtained on the intrinsic parameters. The parameter A-TA-<NUM> actually describes the image of the absolute conic. A geometric interpretation will now be provided.

The model plane, under the convention used herein, is described in the camera coordinate system by the following equation: <MAT> where w = <NUM> for points at infinity and w = <NUM> otherwise. This plane intersects the plane at infinity at a line, and we can easily see that <MAT> and <MAT> are two particular points on that line. Any
point on it is a linear combination of these two points, i.e., <MAT>.

Computing the intersection of the above line with the absolute conic, and knowing that by definition, the point x∞ (the circular point) satisfies x∞Tx∞= <NUM>, i.e., <MAT>.

The solution is b = ± ai, where i<NUM> = -<NUM>. That is, the two intersection points are <MAT>.

Their projection in the image plane is then given, up to a scale factor, by <MAT>.

Point m̃∞ is on the image of the absolute conic, described by A-T A-<NUM>. This gives <MAT> requiring that both real and imaginary parts be zero yields (<NUM>) and (<NUM>).

The details on how to effectively solve the camera calibration problem will now be provided. An analytical solution will be presented, followed by a nonlinear optimization technique based on the maximum likelihood criterion. Finally, both analytical and nonlinear solutions will be provided, taking lens distortion into account.

Consider the following Equation (<NUM>): <MAT>.

Noting that B is symmetric and defined by a 6D vector <MAT>.

Let the ith column vector of H be hi = [hi1, hi<NUM>, hi3]T. Then, we have <MAT> with <MAT>.

Therefore, the two fundamental constraints (<NUM>) and (<NUM>), from a given homography, can be rewritten as <NUM> homogeneous equations in b, as shown below in Equation (<NUM>): <MAT>.

If n images of the model plane are observed, by stacking n such equations as (<NUM>), the result can be expressed as: <MAT> where V is a <NUM>n × <NUM> matrix. If n ≥ <NUM>, generally, a unique solution b defined up to a scale factor is obtained. If n = <NUM>, the skewless constraint γ = <NUM>, i.e., [<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>]b = <NUM>, can be imposed and added as an additional equation to Equation (<NUM>). (If n = <NUM>, two camera intrinsic parameters, e.g., α and β, may be solved assuming u<NUM> and υ<NUM> are known (e.g., at the image center) and γ = <NUM>. The solution to Equation (<NUM>) is well-known as the eigenvector of VTV associated with the smallest eigenvalue (equivalently, the right singular vector of V associated with the smallest singular value).

Once b is estimated, values in camera intrinsic matrix A can be calculated. Once A is known, the extrinsic parameters for each image may be computed. Using Equation (<NUM>), for example, the following may be obtained: <MAT> <MAT> <MAT> <MAT> with λ = <NUM> / ∥A-<NUM>h<NUM>∥= <NUM> / ∥A-<NUM>h<NUM>∥. Due to the presence of noise in data, the so-computed matrix R = [r<NUM>, r<NUM>, r<NUM>] does not satisfy the properties of a rotation matrix.

Suppose there are n images of a model plane and m points on the model plane. Suppose also that the image points are corrupted by independent and identically distributed noise. The maximum likelihood estimate can be obtained by minimizing the following functional: <MAT> where m̂ (A, Ri, ti, Mj) is the projection of point Mj in image i, according to Equation (<NUM>). A rotation R is parameterized by a vector of <NUM> parameters, denoted by r, which is parallel to the rotation axis and whose magnitude is equal to the rotation angle. R and r are related by the Rodrigues formula. Minimizing Value (<NUM>) is a nonlinear minimization problem, which may be solved with the Alevenberg-Marquardt Algorithm. It uses an initial guess of A, {Ri, ti|i = <NUM>. n} which can be obtained using the technique described above.

The solutions above do not consider lens distortion of a camera. However, a desktop camera usually exhibits significant lens distortion, especially radial distortion. Now, first two terms of radial distortion will be discussed. It is likely that the distortion function is dominated by the radial components, and especially by the first term.

Let (u, υ) be the ideal (distortion-free) pixel image coordinates, and (ǔ, υ̌) the corresponding real observed image coordinates. The ideal points are the projection of the model points according to the pinhole model. Similarly, (x, y) and (x̌, y̌) are the ideal (distortion-free) and real (distorted) normalized image coordinates. <MAT> <MAT> where k<NUM> and k<NUM> are the coefficients of the radial distortion. The center of the radial distortion is the same as the principal point. From <MAT> <MAT>.

Estimating Radial Distortion by Alternation. As the radial distortion is expected to be small, one would expect to estimate the other five intrinsic parameters, using the technique described above reasonably well by simply ignoring distortion. One strategy is then to estimate k<NUM> and k<NUM> after having estimated the other parameters, which will give the ideal pixel coordinates (u, υ). Then, from Equations (<NUM>) and (<NUM>), we have two equations for each point in each image: <MAT>.

Given m points in n images, we can stack all equations together to obtain in total 2mn equations, or in matrix form as Dk = d, where k = [k<NUM>, k<NUM>]T. The linear least-squares solution is given by <MAT>.

Once k<NUM> and k<NUM> are estimated, one can refine the estimate of the other parameters by solving Equation (<NUM>) with m̂(A, Ri, ti, Mj) replaced by Equations (<NUM>) and (<NUM>). We can alternate these two procedures until convergence.

The convergence of the above alternation technique may be slow. A natural extension to Equation (<NUM>) is to estimate the complete set of parameters by minimizing the following functional: <MAT> where m̌ (A, k<NUM>, k<NUM>, Ri, tj, Mj) is the projection of point Mj in image i according to Equation (<NUM>), followed by distortion according to Equations (<NUM>) and (<NUM>). This is a nonlinear minimization problem, which is solved with the Levenberg-Marquardt Algorithm. A rotation is again parameterized by a <NUM>-vector r, as disclosed above. An initial guess of A and {Ri, ti | i = <NUM>. n} can be obtained using the technique described above. An initial guess of k<NUM> and k<NUM> can be obtained with the radial distortion solutions described above, or simply by setting them to <NUM>.

At step S630, the lens distortion parameters k<NUM> and k<NUM> are then used on the original image <NUM> to obtain an image <NUM> that is corrected for lens distortion. Alternatively, instead of using the image, a particle object array that describes the particle's geometries, may be corrected. The lens distortion parameters k<NUM> and k<NUM> are used to correct the image for lens distortion using the following equations (<NUM>) and (<NUM>).

Let (xcorrect, ycorrect) represent the correct locations in the image if there were no distortion due to the lens. Then: <MAT> <MAT> where r<NUM> = (xcorrect -u<NUM>)<NUM> + (ycorrect - v<NUM>)<NUM>, and u<NUM> and v<NUM> are principal points (i.e., the intersection points of the cameral optical axis and image plane).

The lens distortion can then be corrected by warping the recorded bitmap with a reverse distortion. For each pixel in the corrected image, its corresponding location is mapped in the distorted image using equation (<NUM>) and (<NUM>) above. The section below titled Geometric Image Transformations describes how the two-dimensional image coordinates and the three-dimensional world coordinates are related by a camera's intrinsic parameters (e.g., focal lens, principal points, distortion coefficients) and extrinsic parameters (rotation and translation matrix). For each integer pixel coordinates in the destination (corrected bitmap), trace back to the source (recorded bitmap) and find the corresponding float coordinates, using the surrounding integer pixels to interpolate the float coordinates. Bilinear interpolation may be used in this process.

In Summary, the lens-distortion correction procedure proposed here, part of which incorporate Zhang's technique, is as follows:.

At step S640, correction for perspective distortion may be achieved using the four corner points of a rectangular calibration mark to solve the homology H using the following process:.

Suppose we have a point in world coordinates
<MAT>
and we write this in homogeneous coordinates as
<MAT>.

Similary, the corresponding point in image coordinates is
<MAT>
and we write this in homogeneous coordinates as
<MAT>.

The relationship between these two can be expressed in the following equation:
<MAT>
where <MAT> is the Homography we want to solve.

Expanding both sides of the equation using matrix multiplication, we get: <MAT> <MAT> <MAT>.

Plug in the third equation to the first two, we get two equations from this pair of points: <MAT> <MAT>.

Since there are eight unknowns in H, we need <NUM> pairs of points to solve H. We write the eight equations in matrix form: <MAT>.

Thus, for each image suffers from the projective distortion, we pick four points in this image, and given the world coordinates of these four points, we are able to solve H.

The "k" in the above equation is a scalar for the homogeneous representation of the two-dimensional coordinates, different from the lens coefficients k1, k2. Four points (e.g., four corners where the shape is rectangular) of multiple calibration marks <NUM> may be used to account for nonuniformity of distortions. As shown in <FIG>, the starting point for perspective correction in this approach is not the original image, but the corrected image where lens distortion effects have been removed. The homography H is determined using four points from a set of calibration marks identified in the image corrected for lens distortion, and then is applied to the corrected image to determine a bitmap <NUM> (the corrected, or true-size, bitmap) that has been corrected for both lens and perspective distortion.

A perspective distortion usually occurs when the camera's optical axis is not perpendicular to the center of the object. Using the image of particles captured on a grid-patterned background, multiple (e.g., five) pairs of orthogonal lines in the scene may be used to find the homography to correct the perspective distortion. This correction will often make parallel lines in the physical world also parallel in the image, orthogonal lines in the physical world also orthogonal in the image, squares in the physical world have unit aspect ratio in the image, and/or circles in the physical world circular in the image.

To summarize the perspective distortion correction process detailed above, the process entails the following steps:.

The lens distortion correction and the projection distortion correction mayb e tested separately and cascaded, so that Shepard's interpolation may only be done once.

The method illustrated in <FIG> and <FIG> may be implemented in a processing device. As discussed with respect to <FIG>, once the camera <NUM> captures the image, the data processing of the image can occur using a processor directly connected to the camera <NUM> or the image data may be transmitted to a separate processor.

This section will now talk about some known image transformation functions that may be used to correct and manipulate the image of the particle. More specifically, the functions in this section perform various geometrical transformations of 2D images. They do not change the image content but deform the pixel grid and map this deformed grid to the destination image. In fact, to avoid sampling artifacts, the mapping is done in the reverse order, from destination to the source. That is, for each pixel (x, y) of the destination image, the functions compute coordinates of the corresponding "donor" pixel in the source image and copy the pixel value: <MAT>.

In case the forward mapping is specified as 〈gx, gy〉 : src → dst, the functions described below first compute the corresponding inverse mapping 〈fx, fy〉 : dst → src and then use the above formula.

The actual implementations of the geometrical transformations, from the most generic Remap and to the simplest and the fastest Resize, need to solve two main problems with the above formula.

This function calculates an affine matrix of 2D rotation. Some parameters used for this process are as follows:.

The function calculates the following matrix: <MAT> where.

The transformation maps the rotation center to itself. If this is not the target, the shift should be adjusted.

This function calculates the affine transform from <NUM> corresponding points. Some parameters used for this process are as follows:.

The function calculates the <NUM> × <NUM> matrix of an affine transform so that: <MAT> where <MAT>.

This function calculates a perspective transform from four pairs of the corresponding points. Some parameters used for this process are as follows:.

The function calculates a matrix of a perspective transforms so that: <MAT> where <MAT>.

This process retrieves a pixel rectangle from an image with sub-pixel accuracy. Some parameters used for this process are as follows:.

This function extracts pixels from src at sub-pixel accuracy and stores them to dst as follows: <MAT> where <MAT> and <MAT>.

The values of the pixels at non-integer coordinates are retrieved using bilinear interpolation. When the function needs pixels outside of the image, it uses replication border mode to reconstruct the values. Every channel of multiple-channel images is processed independently.

This function retrieves the pixel rectangle from an image with sub-pixel accuracy.

This function extracts pixels from src: <MAT> where the values of the pixels at non-integer coordinates are retrieved using bilinear interpolation. Every channel of multiple-channel images is processed independently. While the rectangle center must be inside the image, parts of the rectangle may be outside. In this case, the replication border mode is used to get pixel values beyond the image boundaries.

This function remaps an image to a log-polar space.

This function transforms the source image using the following transformation:.

The function emulates the human "foveal" vision and can be used for fast scale and rotation invariant template matching, for object tracking and so forth. The function cannot operate in-place.

This function applies a generic geometrical transformation to the image.

This function transforms the source image using the specified map: <MAT> where values of pixels with non-integer coordinates are computed using one of available interpolation methods. mapx and mapy can be encoded as separate floating-point maps in map<NUM> and map<NUM> respectively, or interleaved floating-point maps of (x, y) in map<NUM>, or fixed-point maps created by using ConvertMaps function. The reason one might want to convert from floating to fixed-point representations of a map is that they can yield much faster (~2x) remapping operations. In the converted case, map<NUM> contains pairs (cvFloor(x), cvFloor(y)) and map<NUM> contains indices in a table of interpolation coefficients. This function cannot operate in-place.

To shrink an image, it will generally look best with INTER_AREA interpolation, whereas to enlarge an image, it will generally look best with INTER_CUBIC (slow) or INTER_LINEAR (faster but still looks OK).

This function applies an affine transformation to an image.

The function warpAffine transforms the source image using the specified matrix: <MAT> when the flag WARP_INVERSE_MAP is set. Otherwise, the transformation is first inverted with InvertAffineTransform and then put in the formula above instead of M. The function cannot operate in-place.

The function is similar to GetQuadrangleSubPix but they are not exactly the same. WarpAffine requires input and output image have the same data type, has larger overhead (so it is not quite suitable for small images) and can leave part of destination image unchanged. While GetQuadrangleSubPix may extract quadrangles from <NUM>-bit images into floating-point buffer, has smaller overhead and always changes the whole destination image content. The function cannot operate in-place.

This function applies a perspective transformation to an image. Parameters useful for this function include the following:.

This function transforms the source image using the specified matrix: <MAT> if CV_WARP_INVERSE_MAP is not set <MAT> otherwise.

Note that the function cannot operate in-place.

While the embodiments are described in terms of a method or technique, it should be understood that the disclosure also covers an article of manufacture that includes a non-transitory computer readable medium on which computer-readable instructions for carrying out embodiments of the method are stored. The computer readable medium may include, for example, semiconductor, magnetic, opto-magnetic, optical, or other forms of computer readable medium for storing computer readable code. Further, the disclosure may also cover apparatuses for practicing embodiments of the inventive concept disclosed herein. Such apparatus may include circuits, dedicated and/or programmable, to carry out operations pertaining to embodiments.

Claim 1:
A computer-implemented method of determining a particle size distribution, comprising:
obtaining an image of a plurality of particles (<NUM>) and one or more calibration marks (<NUM>) by using a camera (<NUM>) that is positioned to capture an image of the plurality of (x, particles (<NUM>) lying on a calibration pattern (<NUM>) which includes a background (<NUM>) and the calibration marks (<NUM>), wherein the camera (<NUM>) includes at least one lens and wherein the plurality of particles (<NUM>) and the one or more calibration marks (<NUM>) were captured using the same lens (<NUM>);
correcting the image for distortion effects to generate a corrected image, wherein the same correction factor is applied to both the plurality of particles (<NUM>) and the one or more calibration marks (<NUM>);
determining a size of the plurality of particles in the image using the corrected image and the dimensions of the one or more calibration marks (<NUM>);
determining a particle size distribution for the plurality of particles;
comparing the particle size distribution with a target particle size profile to determine closeness of match; and
based on said comparison, indicating to an operator that further grinding would bring the particle size distribution closer to the target particle size profile.