METHOD AND APPARATUS FOR AUTOMATED MEASUREMENT

Abstract: The invention relates to a method and apparatus for automated measurement, particularly the use of automated measurement in retailing. As many as 50 % of all measurement-sensitive furnishing orders are inaccurate, mainly due to difficulties in making accurate measurements on site. There is therefore a requirement for reliable measurements in retailing. The invention provides a method of measuring the distance from a point in space to a reference plane, which may be a wall or other feature, in order to measure required dimensions. At least two different views of this scene are required. The scenes are captured and downloaded to a computer where they are processed using the method. The method offers: 1) speedy, unobtrusive and accurate data capture where furnishings are to be installed; 2) a permanent representation of the site and its key measurements; and 3) seamless integration into retailers' related systems such as CAD, stock control, and ordering systems.

Description: METHOD AND APPARATUS FOR AUTOMATED MEASUREMENT The present invention relates to a method and apparatus for automated measurement. It relates particularly, but not exclusively, to automated measurement in retailing. There is a strong requirement for reliability, quality and good customer service in the specification and delivery of fitted and soft furnishings. Accurate and verifiable measurements, which may be taken in the customers' homes, are important. As many as 50% of all measurement-sensitive furnishing orders (representing several thousand orders per month to a major retailer) are inaccurate because of errors, mis- communication, or difficulties in making or checking measurements on-site. This represents an extremely high overhead in terms of mis-spent retailer and manufacturer time, materials, communication and travel costs. A preliminary calculation reveals cost inefficiencies measured in millions of pounds per annum for a typical major national retailer due to inaccurate measurements. A solution to the aforementioned problem is required which offers: 1) speedy, unobtrusive and accurate data capture where the furnishings are to be installed; 2) systematic and permanent representation of the site, and its key measurements; and 3) seamless integration into retailers' related systems such as CAD, estimating, stock control, and ordering systems. A measurement system offering such a solution will greatly enhance the productivity of peripatetic measuring and fitting staff, and avoid the costs and sources of error/confusion which translation from paper records and/or frequent re-keying of data incur. Creation of a permanent measurement record will also allow frequent checking at all points in the transaction chain, from early customer scrutiny and verification through order placement and to pre-delivery final checking. An aim of the present invention is to provide a method of automated measurement, and an apparatus for implementing the method. According to a first aspect of the invention there is provided a method of measuring a dimension h r relative to a predetermined reference plane, where h r is the Euclidean distance of a point R of an object from the reference plane, the method including the steps of: a) acquiring at least two different views of a scene, the reference plane and the point R being visible in both views; b) determining the parallax vector of point R; c) determining the parallax vectors of at least two reference points P and Q which do not lie on the reference plane, reference points P and Q being visible in both views; d) determining the Euclidean distance h and h q of respective reference points P and Q from the reference plane wherein h p is not equal to h , and h r , h p , h q are. all measured in the same direction; and e) determining h r using the expression h r where , 1 = h ph η , A is an area of a triangle on a plane and is defined herein, and pi, p2, ql, q2, rl and r2 are apices of the triangular area A. The object may be a tangible or real object, or an intangible object. Reference point R may be a point on (or very close to) a tangible object, so that the distance of the object from the reference plane may be calculated by calculating the Euclidean distance of point R from the reference plane. For example, the reference plane may be a wall, and the object a table. The point R may be associated with an object, or, alternatively, the point R may be associated with a protrusion (or depression) from the plane such as a clock, shelving, or an alcove. The method may include the further step of selecting a point R associated with the object. In both of the aforementioned methods, the reference plane is preferably calibrated by finding the location of its vanishing line in both views. This is typically achieved by determining the locations in each view of at least four calibration points on the reference plane whose world locations on the plane are known. However, if the calibration points chosen are corners of a rectangular object or feature, only two calibration points are required. Such methods of calibration are known in the prior art. According to another aspect of the invention there is provided apparatus for measuring the distance of a point R from a plane using the aforementioned methods, the apparatus comprising image capture means for capturing at least two views of a scene from different viewpoints, image storage means for storing or saving the captured images, image display means for displaying the images, point selection means for selecting calibration points, reference points and the point R, and calculation means for calculating the Euclidean distance of the point R from the reference plane. Preferably the image capture means is a camera. The camera may be a digital camera, or an analogue camera. Preferably there is also provided an image transfer means. Where the camera is a digital camera, the image transfer means may be a serial connection from the camera to a computer. Where the camera is an analogue camera, the analogue image may be transferred to a computer by way of a scanner. A computer may be used as a means of storing and/or displaying the captured images. Preferably the calibration (basis) points are selected by a user. This usually takes the form of clicking on the image displayed on a computer screen with a computer mouse in order to select the points. Alternatively, the calibration points may be automatically generated from a calibration plate located in the image. The distance between the calibration points is required in order to enable the calculation of a reference distance. Preferably the point R is selected by a user. As for the case of the selection of the calibration points, point R may be selected by clicking on an image of the scene displayed on a computer screen with a mouse. Preferably there is also provided edge detection means. The edge detector fits a series of straight lines to straight lines (or curves) of features in the images. This enables a user to more accurately select points to be measured. A threshold may be set to limit the number of straight lines to be fitted. The methods and apparatus may be used for: kitchen measurements (facilitating ordering of fitted kitchens), living room or bedroom measurements (facilitating ordering of suites and other free-standing furniture), and window measurements (for curtain, net and blind quantity/price estimation). It may also be used for making measurements of key dimensions in external scenes such as building and civil engineering sites. It may also be used for determining the height of individuals appearing on CCTV pictures. Additionally, simple visualisation of potential purchases may be performed. This may, for example, determine if a sofa of given dimensions will fit in a customer's living room. The measurements calculated from the images may be used to form a 3D representation of the view. This may be in the form of a VRML model, or other virtual reality coding language. The invention will now be described, by way of example only, with reference to the accompanying Figures, in which :- Figure la shows a diagram of the general configuration of parallax geometry; Figure lb shows a diagram of parallax geometry viewed on the distinguished plane, where the parallax vector (p l , p 2 ) passes through the epipole e n . Figure 2 shows a diagram of a planar homology defined by a vertex and an axis; Figure 3a shows a diagram of the geometry of three points from two different views, where the triangle p,#,r, is the "shadow" of PQR under the camera O , ; Figure 3b shows a diagram of the geometry of three points from two different views, where the axis of homology is given by the intersection of the plane PQR with the world plane, and the vertex (epipole e n ) by the intersection of the baseline with the world plane; Figure 4 shows a diagram of three points in two views relative to a known plane, which leads directly to a Desargues configuration on the plane; Figure 5 shows a diagram of the geometry of three views with two points off the plane, where the three epipoles are collinear, lying on the line which is the intersection of the plane O , O 2 O 3 with the world plane, and the point a pq is the intersection of the line PQ with the world plane; Figure 6 shows a diagram of the planar geometry in the three view two point case which is a Desargues configuration and point correspondences P j — > q , p 2 — q 2 and 3 — >q 3 are related by a homology; Figure 7 shows a diagram where the ratios of distances between r , , r 2 and e 12 and three epipoles define the transfer of the point r to r 3 ; Figure 8 shows a diagram of the parallax geometry of two points; Figure 9 shows a diagram of the relative structure y expressed as a ratio of areas; Figure 10a shows a lecturn with reference points indicated by white squares; Figure 10b shows the lecturn of Figure 10a, with measurements; Figure 11a shows a view of a dining hall with reference points indicated by white squares; Figure lib shows the view of Figure 11a, with added measurements; and Figure 12 shows a scene with added measurements. The invention, as herein described, enables the replacement of conventional measurement techniques (e.g. tapes and rules) with an accurate automated measurement system. During the method of the invention, a set of images of a scene acquired from different viewpoints is captured using a camera. Once the images are captured, the system may compute a three-dimensional (3D) representation of the scene. This 3D representation may be available for immediate or subsequent analysis, reworking or checking, including the visibility of all dimensions. Using the method of the invention, 3D measurements of the size and location of any features present in at least two views from two or more images from disparate viewpoints can be made. This method is based on a branch of mathematics known as projective geometry, and will now be described in more detail. The method requires the use of 3D-2D projective geometry. In this case, a single view of a point does not provide enough information to locate the point in 3D, but it does constrain the 3D point to lie on a ray containing the camera's optic centre and the image point. A second view of the same point from a different viewpoint provides a second ray, and the 3D location is determined as the intersection of the two rays. So, two image measurements are sufficient to make a single 3D measurement. The steps of the method are as follows: 1. Acquire two images of a plane and protrusions from the plane or objects off the plane. Each image must be taken from a different viewpoint. At least 4 points must be identified on the plane (or 2 points if the points are corners of a rectangle), along with the distances from the plane of two reference points off the plane; 2. Transfer both images to a computer. A serial connection or other suitable method may be used; 3. Calibrate the image pair. Calibration points and the points relative to the two distances from the plane may be selected by a user, or generated automatically; 4. Compute measurements on the plane and distances from the plane. To measure the relative distance of an object (protrusion) from the plane, corresponding points on the two images are selected (distances between points on the plane may also be computed). A detailed description of the mathematics behind the method will now be given. In the description, the geometry of two views of seven points (four of which are co-planar) and three views of six points (four of which are co-planar) is investigated. In addition to the mathematical background of the method, a number of practical contributions are made, including formulae for computing the distance of the cameras from a distinguished world plane, and formulae for structure computations. Examples of these computations on real images is given. Additional details of the method are described in "A Plane Measuring Device" by Criminisi, Reid and Zisserman (British Machine Vision Conference). Introduction In their recent work, Carlsson and Weinshall et al. ("Duality of reconstruction and positioning from projective views", IEEE Workshop on Representation of Visual Scenes, Boston, 1995) have demonstrated the fundamental duality of the 3D reconstruction problem. They show that for points and camera in general position, the problem of computing camera positions from n points in m views is mathematically equivalent to the problem of reconstructing m + 4 points in n - 4 views. In this description we investigate the case where the points are not in general position, but where four of the space points are coplanar (which we refer to as the plane + points configuration). We show that in this case there exists an additional dual relationship which is described by a planar homology, which encapsulates the fundamental geometric constraints which can be obtained. A summary of the duality results contrasted with the general position cases is shown in Table 1. The plane + points configuration has received significant attention in the past, not least because it arises frequently in everyday scenes. A useful and popular approach to the problem decomposes the image motion into a planar homographic transfer plus a residual image parallax vector. This decomposition has the advantage that it partially factors out dependence on the camera relative rotation and internal parameters. Furthermore, it can be shown that the relative structure of points, and the rigidity of a scene, can be determined directly from the image measurements - i.e. the parallax vectors - without needing to compute the epipolar geometry. We show that these constraints, and other equivalent and dual ones, are consequences of the planar homology. Table 1: Camera point duality results for (i) points in general position and (ii) four points lying on a distinguished plane. The fundamental matrix F has 7 degrees of freedom (d.o.f.) and the trifocal tensor T has 18 d.o.f. In fact, the work here unifies a number of previously diverse results related to planar parallax, duality and planar homologies. In addition to this theoretical contribution, we make a number of practical contributions, including formulae for computing the distance of the cameras from a distinguished world plane, formulae for structure computations, and we derive the trifocal tensor in the plane + points case, showing that it is obtained uniquely. We begin with a discussion of background material - notation, parallax geometry and planar homologies. We then turn to the geometry of two views, seven points, four of which are coplanar. We show that there exists a homology on the plane relating the two views and derive necessary conditions for the homology directly in terms of the parallax measurements. We also show the duality of the geometry of three views, six points (four coplanar) to the two view, seven point case, and hence obtain analogous necessary conditions. We also derive the trifocal tensor and show that it is over-constrained. We then derive expressions for the height of the cameras from the distinguished plane and the structure of points in terms of affine invariants, and give examples of various applications. Background The mathematical notation used herein will now be described. We denote 3D points in general position by upper case bold symbols (e.g. P) and image positions and vectors by lower case bold symbols (e.g. p, μ ) and scalars by lower case normal symbols (e.g. d, h p ). Matrices are denoted by typewriter style capitals (e.g. A, B). The area of a triangle on a plane with vertices p, q, and r is denoted A pqr , and can be determined via the formula A pqr = — |pqr| where the points p, q, and r are represented as homogeneous 3-vectors with last component equal to one. Numbered subscripts are used to distinguish different views, with the first camera centre given by O, , the second by O 2 and the third by O 3 . The projection of an image point onto the distinguished world plane from the i' h view is denoted p ( . . Planar parallax The underlying parallax geometry is shown in Figure 1. The distinguished world plane induces a homography between the views meaning that the images of points on the plane can be transferred via the homography between views 1 and 2. The homography can be determined from a minimum of four correspondences in the two views of points (or lines) on the distinguished plane. The parallax vector in the first view is the vector joining the image of a world point P with the transferred location of P's image in the other view (i.e. the image of p 2 ). Furthermore, since the three planes (distinguished world plane and two image planes) are equivalent up to a plane projectivity, we can also measure parallax in the second view, or - if we know the image to world plane homographies - on the distinguished world plane. In fact it is particularly elegant to work with the world plane. In this case all dependence on the rotational and internal parameters of the cameras is removed (aggregated into the image plane to world plane homographies) leaving only a dependence on the camera centres. Since the clarity of the underlying geometry is greatly increased, we depict all relevant points and vectors on the world plane in all of the figures. However, the computations, in general, do not require the image to world homographies to be known. The parallax vector is directed towards (or away from) the epipole, so two such vectors are sufficient to compute its position, and the full epipolar geometry follows. The magnitude of the parallax vector is related to the distance of the world point and cameras from the world plane. Planar homologies A planar homology is a plane projective transformation with five degrees of freedom, having a line of fixed points, called the axis and a distinct fixed point not on the axis known as the vertex (Figure 2). Algebraically, such a transformation has one distinct eigenvalue, with corresponding eigenvector being the vertex, and two repeated eigenvalues, whose corresponding eigenvectors span the axis. Planar homologies arise naturally in an image when two planes related by a perspectivity in 3-space are imaged. If two triangles on a plane are related such that the lines joining their corresponding vertices are concurrent, then they are said to be in a Desargues configuration, and Desargues' Theorem states that the intersections of their corresponding sides are collinear (Figures 4 and 6). Such triangles are related by a planar homology, with the common point of. intersection being the vertex of the transformation, and the axis being the line containing the intersections of corresponding sides. Conversely, any triple of points in correspondence under a homology must be in a Desargues configuration. The projective transformation representing the homology can be parametrized directly in terms of the 3-vector representing the axis a, the 3-vector representing the vertex v, and the characteristic cross-ratio μ as: va T H = I + (μ - l) v. Having five degrees of freedom (the scales of v and a have no effect), a homology can be determined by 2.5 point correspondences. Three point correspondences, therefore, provide an additional constraint. In the next section we derive the link between homologies and the structure and motion. Geometry of two views We consider the case of imaging seven points, four of which are coplanar from two distinct viewpoints. Each of the three points P, Q and R not on the plane gives rise to a parallax vector, which is depicted on the world plane in Figure 3. The plane PQR intersects the world plane in a line, and the camera baseline intersects the world plane in a point. It can be seen by inspection of Figures 3 and 4 that the geometry under consideration (seven points, two views) leads directly to a Desargues configuration in which the epipole is the vertex of the homology and the intersection of plane PQR with the world plane is the axis of the homology. The two triangles in the Desargues configuration are the two images of the space triangle PQR. This key observation underpins the results which follow. As stated in the previous section, a homology has five degrees of freedom, and therefore three point correspondences over-determine the homology. The extra constraint available can be used as a test for the rigidity of the scene and is equivalent to the epipolar constraint. Clearly the constraint can be tested geometrically by using point correspondences either to construct the intersections of corresponding sides and testing their collinearity, or testing the concurrence of the parallax vectors. Alternatively an algebraic test could, for example, compute the epipole using two point correspondences, use the epipole plus the three point correspondences to solve for a general homography, then test the homography to determine if it is a homology. The former has the disadvantage of requiring the construction of features which may be far removed from the measured image features themselves, while the latter gives little insight into the underlying geometry. Below we derive novel bilinear and trilinear constraints which are necessary conditions on the homology. We refer to these as motion constraints and they are equivalent to the epipolar constraint, but have the advantage that the computations involve only those features which can be measured directly, namely the parallax vectors. Motion constraints Here we give necessary conditions for the homology (which are therefore necessary for scene rigidity in two views) in the form of an identity involving only areas computable from the parallax vectors. Two such conditions and their symmetric forms can be determined. The first is derived from the collinearity of the points a , a pr and a pq and leads to a constraint which is trilinear in the areas. The second is derived from the collinearity of the epipole e 12 and corresponding points and is bilinear in the areas. The results are summarised in Table 2. The areas A can be computed either in the image or on the distinguished plane. The latter requires knowledge of the world to image homography, the former only the homography between images. Violation of any of the motion contraints (1) - (5) shown in Table 2 is a clear indication that there has been non-rigid motion between the views. However, if any (or all) of the points P, Q, R moves in its own epipolar planes, then the equations are still satisfied and non-rigidity is not detected. Table 2: Two view bilinear (B , ) and trilinear (T , ) motion constraints equivalent to the epipolar constraint. Geometry of three views We now consider the geometry of six points, four of which are coplanar, in three views. The geometry is shown in Figure 5. We begin by demonstrating the duality of this case to the two view case, and obtain a structural constraint directly from the measured image features. We then derive the trifocal tensor for the three view, six point (four coplanar) case. Since the trifocal tensor is over-constrained by six points, four of which are coplanar, we also obtain another form of the structure constraint. Duality It is clear by inspection of Figure 5 that the three view geometry is dual to that of Figure 3 in which we have directly exchanged points off the plane for camera positions. The vertex of the homology is given by the intersection of the line PQ with the world plane, and the axis by the intersection of the trifocal plane containing the three camera centres O, , O 2 , O 3 , with the world plane (Figure 6). Having established the duality of the two situations, we are now in a position to invoke duality in order to prove further results. We make the substitutions: and the dual trilinear and bilinear constraints given in Table 3 follow. Table 3: Three view bilinear (Bi) and trilinear (Ti) structure constraints. The trifocal tensor In this section it is shown that the trifocal tensor is uniquely determined from three views of six points, four of which are coplanar. We begin with a familiar form of the trifocal tensor in which we consider the image projection matrices and image point locations. We then show how the form of the tensor is simplified when we consider all geometric objects (lines, points, etc) projected onto the distinguished plane. Image form: In order to discriminate between points (and epipoles) in images, as opposed to their projections onto the distinguished plane, we use primes to indicate the image in which the points appear; i.e., the images of P in views 1, 2 and 3 are, respectively, p, p' and p". Suppose the homographies induced by the plane of the points are A and B, so that p 1 = Ap and p" = Bp for images of points on the plane. These homographies are computed from the images of the four coplanar points. The images of the first camera centre in the second and third images, denoted e' and e" respectively, are the epipoles. They are determined using parallax vectors, as previously described, so that F 12 = [e' ]x A, and F 13 = [e"j x B. It can be shown that the three camera projection matrices can be chosen as p = [i | oi, P' = [A | eι, P " = [B μe'τ (11) up to a homography of 3-space, where λ is an unknown scalar. This unknown scalar is determined by line transfer. The line through the (non-coplanar) points P, Q, is imaged as / = p x q, I' = p' x q', I" = p" x q" in the first, second and third views respectively. It is then straightforward to show that lines transfer as follows (we could alternatively consider point transfer): 1 = λ(e V')K τ l' - (e'.Z')B r I" (12) The scalar λ is the only unknown in this equation. It is determined by taking the vector product with I. λ(e".l")l x (A r 0 = (e'I)l x (B r /") (13) This provides two equations in the one unknown λ , and so we can solve uniquely for the trifocal tensor and obtain one further constraint, namely that the rigidity condition that the imaged intersection of the line through P, Q is the same when computed from views one and two ( 1 x (A τ I')) as views one and three ( 1 x (B r l 1 )) . This is yet another form of the constraints (6) - (10). The scale factor lambda is obtained by normalising both sides of (13): Distinguished plane form: On the distinguished plane, A = B = I, so the projection matrices have the very simple form: P, = [I | 0], P 2 = [I | e 12 ], P 3 = [l U e I3 ] (15) It then follows that the equivalent of (12) for point transfer is r 3 = ^ 13 (/ 2 .r 1 )-(e 12 ./ 2 )r 1 (16) where Z 2 is any line through r 2 . This equation depends only on the positions of the epipoles on the distinguished plane, with all dependence on camera internals and relative rotations having been factored out into the image to plane homographies. Then, using the projection matrices defined in (15) for a general point R which is represented as R = r x + p e, = P 3 R r, +pλe l 3 . We now give an interpretation of p and λ on the distinguished plane (see Figure 7). The ratio λ depends only on the camera centres, and it can be obtained from the baseline ratio. On the distinguished plane it can be determined as λ = d(e l2 ,e 23 )l d(e ,e 23 ) where d() is the distance between the points on the distinguished plane. The parameter p is the relative affine invariant of Shashua, and is related to the point depth. On the distinguished plane it is obtained as p = d(r 1 , r 2 )ld(r 2 ,e n ). So point transfer using the trifocal tensor simply involves computing the ratio p from r, , r 2 and e l2 and employing λ to define the transferred point r 3 on the line between e 13 and r, as (16). In the case that the three camera centres are collinear, the ratio λ is still defined and can be obtained analogously to the previous section, and there is no degeneracy in the point/line transfer. Structural computations and applications In this section we discuss a number of useful structural computations which can be achieved using ratios of areas. We require affine measurements on the world plane, which can be obtained either from four world plane points known up to an affinity (and hence the image to world plane homographies), or from the inter-image homography and vanishing line of the world plane in each image. In either case we obtain results for the scene structure without resorting first to computing the epipolar geometry. A significant novel aspect of the formulae given in the sections entitled "Camera distance to the world plane" and "Measuring the structure of other points", is that the vanishing point for the direction of measurement need not be known. The results are derived for the two view, seven point case. However, because of the fundamental duality, they are equally valid (with appropriate symbol substitutions) in the three view, six point case. For example, Equation (21) can be used to compute the height of a third point given two other known heights in the two view, seven point case; dually, in the three view, six point case, it can be used to obtain the height of a third camera given the other two camera heights. We begin by re-deriving the basic parallax relationship for the case where the parallax is measured on the distinguished world plane (17). Considering Figure 8 and writing p = O + — - — (P -O x ), p 2 = 0 2 — (P -0 2 ) d l - h d 2 —h and e 12 = 0 2 H — (θ x -0 2 ). Then eliminating O, and O 2 yields d 2 -h where μ p - p 2 - p is the planar parallax vector and A d = d x - d 2 is the component of camera translation towards the plane. Let γ be the ratio of the distance of a point to the plane and the point to the first camera h. h 0 (measured in the same direction), i.e., γ = — and γ = — - — then combining the d x -h p d -h q basic parallax equation (17) for two points P and Q gives Ya p -Yp pA~q ■■ r P r q -j-{pι -^) (18) Finally, taking the cross product of both sides of the equation with p 2 - q 2 and taking γ magnitudes yields an expression for — - as a ratio of areas (see Figure 9) of the form r P (19) This ratio is computable solely from the parallax measurements, and is clearly affine invariant, being a ratio of areas. In this formulation we have used only affine constructs. No perpendicularity has been assumed and the formulae are homogeneous. Camera distance to world plane Here we show that, given the parallax vectors of two world points P and Q, the Euclidean distances of these points from the world plane (measured in the same but arbitrary direction), h p and h q and affine measurements on the world plane, then we can determine the Euclidean distance of either camera to the world plane (measured in the same direction as ft and h ). We can rearrange (19) to give an expression for the distance of the first camera from the plane (a similar formula can be derived for the second camera): d In Figure 11 we have used the formulae to compute the heights of the camera above the floor as 566cm and 586cm (left and right views respectively). Measuring the structure of other points Here we show that, given the parallax vectors of three world points P, Q and R, the Euclidean distances of two of these points from the world plane (measured in the same but arbitrary direction), h p and h q (h p ≠h q ), and either condition I or II above, then we can determine the Euclidean distance of the third point from the plane (measured in the same direction as h p and h q ). A. A From (19) we have that γ r = γ p —^ x - and γ r = γ q rXr2ql . After eliminating the - ■ lp2rl ™-q\q2r\ camera distance d between these equations we obtain an expression for h r : (21) Note that if h and h q are equal, then the formulae above degenerate. However, this situation can be avoided in practice. The degeneracy is understood in terms of the geometry as follows. In general, we obtain projective structure (since F is determined uniquely). If in addition h p ≠h q , the line PQ intersects π ∞ in a point which can be identified in both images (details omitted). This point and the vanishing line of the world plane, I ∞ , determine π ∞ , hence we can obtain affine structure. When h p - lι q then the line PQ intersects π on 1 and so no additional information about π„ is obtained - it is determined only up to a one parameter family (the pencil of planes with 1 ^ as its axis). Figure 10 shows an example in which the heights of points have been estimated. The floor tiling and the perpendicular heights of two other points were measured by hand with a tape measure. A second example is shown in Figure 11. As before, we have used the patterned floor tiling to compute the image to world plane homography, and measured the heights of two points with a tape measure as reference heights. An actual example of the an image of a room taken with a digital camera and processed using the method and apparatus of the invention is shown in Figure 12. As a result of this processing, the height of a door and the width of the room were measured automatically. Discussion We have considered the geometry of two views of seven pints, four of which are coplanar, and the geometry of three views of six points, four of which are coplanar. These configurations were shown to be dual to one another, and that the fundamental geometric constraints are captured by a planar homology relating the images of points across views. Consequently, a number of previously diverse results related to planar parallax, duality and planar homologies have been unified. The constraints derived from the homology are easily computed from directly measured image features, and have been tested on real imagery. Formulae for determining the height of the cameras from the world plane, and the heights of points from the plane have been developed and tested on real imagery. There are several benefits of the methods described above for retailers and customers: • The technique is considerably less invasive within customers' home than manual measuring methods - a few minutes to capture room views on camera will suffice. • In turn, this raises the efficiency of the peripatetic operator, whose calling rate can rise appreciably. • The 3D model can be easily queried either immediately or at a later date, to determine specific measurements, or a verification before delivery is made. • Measurement data are stored digitally, and can therefore be seamlessly integrated into existing stock control and ordering systems, speeding data entry and avoiding typographical errors. • The acquisition of a 3D model facilitates the preparation of computer-generated graphics to represent potential purchases to customers, if desirable, in their own homes at the time of measurement. Further extensions may be added to the system: • Error analysis in 3D - analysis of the accuracy for distances measured off the plane. • Radial distortion correction - analysis and correction of the distortion introduced by the camera lenses.

Claims: Claims 1. A method of measuring a dimension h r relative to a predetermined reference plane, where h r is the Euclidean distance of a point R of an object from the reference plane, the method including the steps of: a) acquiring at least two different views of a scene, the reference plane and the point R being visible in both views; b) determining the parallax vector of point R; c) determining the parallax vectors of at least two reference points P and Q which do not lie on the reference plane, reference points P and Q being visible in both views; d) determining the Euclidean distance h p and h of respective reference points P and Q from the reference plane wherein h p is not equal to h q , and h r , h p , h q axe all measured in the same direction; and e) determining h r using the expression K ~ \ — τ where M = h p h q V Tl\rr22ppl\ A ** ,pιlp2rl M 2 A is an area of a triangle on a plane and is defined herein, and pi, p2, ql, q2, rl and r2 are apices of the triangular area A. 2. A method as claimed in claim 1 including the further step of selecting a point R associated with the object. 3. A method as claimed in claim 1 or claim 2 including the further step of selecting at least two calibrations points on the reference plane whose world locations are known. 4. A method as claimed in claim 3 wherein the calibration points are used to find the location of the vanishing line of the reference plane in both views thereby calibrating the reference plane. 5. Method according to claim 4 wherein the calibration points are selected by a user. 6. Method according to claim 4 wherein the calibration points are automatically generated from a calibration plate located in the image. 7. Apparatus for measuring the distance of a point R from a plane, the apparatus including: image capture means for capturing at least two views of a scene from different viewpoints; image storage means for storing or saving the captured images; image display means for displaying the images; point selection means for selecting calibration points, reference points and the point R; and calculation means for calculating the Euclidean distance of the point R from the reference plane. 8. Apparatus according to claim 7 further including image transfer means. 9. Apparatus according to claim 7 wherein the image capture means is a camera. 10. Apparatus according to claim 9 wherein the camera is a digital camera. 11. Apparatus according to claim 9 wherein the camera is an analogue camera. 12. Apparatus according to claim 7 wherein the image display means is a computer display. 13. Apparatus according to claim 7 wherein the image storage means is a computer. 14. Apparatus according to claim 7 further including edge detection means for fitting at least one substantially straight line to at least one straight line or curve in an image. 15. Method substantially as described herein with reference to the Figures. 16. Apparatus substantially as described herein with reference to the Figures.

Inventors: CRIMINISI ANTONIO, REID IAN, ZISSERMAN ANDREW, CRIMINISI, ANTONIO, REID, IAN, ZISSERMAN, ANDREW
Applicants: CENTRAL RESEARCH LAB LTD, CRIMINISI ANTONIO, REID IAN, ZISSERMAN ANDREW, CENTRAL RESEARCH LABORATORIES LIMITED, CRIMINISI, ANTONIO, REID, IAN, ZISSERMAN, ANDREW
Application year: 1999
Main classification: G01C11
IPC codes: G01C11