Measuring Squareness and Orientation of Shapes In this paper we propose a measure which defines the degree to which a shape differs from a square. The new measure is easy to compute and being area based, is robust-e.g., with respect to noise or narrow intrusions. Also, it satisfies the following desirable properties:-it ranges over (0, 1] and gives the measured squareness equal to 1 if and only if the measured shape is a square; -it is invariant with respect to translations, rotations and scaling.In addition, we propose a generalisation of the new measure so that shape squareness can be computed while controlling the impact of the relative position of points inside the shape. Such a generalisation enables a tuning of the behaviour of the squareness measure and makes it applicable to a range of applications. A second generalisation produces a measure, parameterised by δ, that ranges in the interval (0, 1] and equals 1 if and only if the measured shape is a rhombus whose diagonals are in the proportion 1 : δ.The new measures (the initial measure and the generalised ones) are naturally defined and theoretically well founded-consequently, their behaviour can be well understood. As a by-product of the approach we obtain a new method for the orienting of shapes, which is demonstrated to be superior with respect to the standard method in several situations.The usefulness of the methods described in the manuscript is illustrated on three large shape databases: diatoms (ADIAC), MPEG-7 CE-1, and trademarks. Shape descriptors are a powerful tool for shape classification tasks and many approaches have been developed [24] . There are a number of generic shape descriptors such as Fourier descriptors and moments that are capable of providing a high dimensionality feature vector that precisely describes specific shapes. Alternatively, other descriptors describe some single characteristic that is present over a variety of shapes, such as circularity [15] , ellipticity, rectangularity, triangularity [17] , rectilinearity [28] , complexity [14] and symmetry [27] . Even for a single characteristic there often exist many alternative measures which are sensitive to different aspects of the shape, e.g. convexity [4, 10, 16, 20, 21, 25, 29] . Such a need for alternative measures is caused by the fact that there is no single shape descriptor which is expected to perform efficiently in all possible applicationseven the computation of perimeter is not a straightforward task [5, 23] .. In this paper we consider measuring a single, specific aspect of shape: squareness; i.e. how square is a shape? There appears to be little in the literature on methods to measure squareness beyond Bowman et al.'s [3] method.. They describe the shape boundary by complex Fourier descriptors [8] c k = a k + ib k and use the normalised magnitude r −3 /r 1 to measure squareness, where r k = a 2 k + b 2 k . A modified version was also given in Rosin [18] .. In addition, there are some general approaches to defining shape descriptors that could be adopted to defining squareness [22] . For example, first, fit (somehow) a square K fit (S) to a measured shape S, and then estimate the squareness of the shape S by comparing K fit (S) and S. One straightforward possibility for a fitted square K fit (S) is the square whose centroid coincides with the centroid of S, whose area is equal to the area of S, and finally K fit (S) is rotated such that the area of K fit (S) ∩ S is maximised. Now, a squareness measure (i.e. the comparison between S and K fit (S)) can be given as. Of course, there are other variations of this approach.. It is easy to check that the above defined squareness measure satisfies the following desirable properties: Note Notice that instead of the property Q fit (S) ∈ [0, 1] in (a ) we would prefer the following property to be satisfied: (a) Q fit (S) ∈ (0, 1]. That is, we would prefer that there is no shape with a positive area with measured squareness equal to 0. But, a (slight) disadvantage of the Q fit (S) measure is that there are many shapes whose measured Q fit (S) squareness is equal to 0. An example is a shape containing a large hole such that its intersection with the fitted square K fit (S) is empty.. In this paper we define a new squareness measure which satisfies the basic requirements (a), (b), (c) and (d). A modification of the new measure is also described. Such a modification enables control of the impact of the points' positions inside the measured shape to the computed squareness, and, consequently, enables a tuning of the behaviour of the squareness measure depending on the application where it is applied.. In comparison to the previously defined squareness measures [3, 18] the proposed squareness measure can be applied to shapes represented either as boundaries or sets of internal pixels, and consequently is not restricted to single component shapes.. A by-product of squareness measures that are optimised over orientation, such as (1) above and the new measure which will be based on (19) , is that they can be used to provide an estimate of shape orientation. There already exist various approaches to determine the orientation of shapes, the most common being the principal axis method, determined from the shape's moments [24] . While computing orientation of elongated shapes is straightforward, many approaches break down or become unreliable for compact and/or symmetric shapes [30] . It will be shown that our squareness based orientation estimate is more effective than the standard principal axis method in such circumstances.. The paper is organised as follows. The next section introduces the new squareness measure and proves several desirable properties of it. Section 3 gives some illustrative examples that demonstrate the behaviour of the new measure Q(S) and compare it with Q fit (S).. In Sect. 4 we introduce a modification of Q(S) which gives different weights to the points inside the shape depending on their position with respect to the measured shape centroid. In such a way, the introduced squareness measure can be adopted to be successfully used in a wide range of applications. Illustrative examples and theoretical consideration of the modified measure are also given in Sect. 4. Section 5 uses the squareness measure for a variety of image processing applications. The suitability and effectiveness of all the methods introduced by the paper are demonstrated on three large shape databases: diatoms (ADIAC), MPEG-7 CE-1, and trademarks. Comments and conclusions are in the last section. This paper has described a measure to quantify the degree to which a shape is square-like. There appears to be little in the literature on methods to measure squareness, but some general methods for defining shape descriptors can be adopted to test shape squareness. A standard approach to defining shape descriptors is used here for comparison in testing the performance of the new squareness measures. We note that one of reasons for a lack of methods for measuring the squareness of shapes could be the fact that there is no 'simple' equation of the square boundary (e.g. the existence of such an equation leads to straightforward methods for measuring shape circularity). Another reason is that a square has no straightforward geometric property which could lead to a simple definition of the shape squareness measure. In comparison, simple definitions of the convex shapes have led to several definitions for shape convexity measures [4, 10, 16, 20, 21, 25, 29] .. Roughly speaking, we start from the integral S (|x| + |y|) dx dy and show that it reaches the minimal possible value if and only if S is a square which is suitably oriented. Such a minimum is 2Area(S) 3 /3. Based on this we define the shape squareness measure as • easy to compute;. • area based, and therefore robust;. • range in (0, 1]; • maximised only by a square;. • invariant to similarity transformations; • applicable to compound shapes, as well;. • naturally defined and theoretically well founded;. • results match human intuition.. In addition, the orientation of the shape that maximises the squareness measure has been shown to be a reliable estimator of the shape's orientation, while R(S), a modified version of Q(S) that provides a rhomboidal measure, was found on the MPEG-7 CE-1 database to provide even more reliable orientation estimates. Unlike the standard principal axis method Q(S) and R(S) are able to cope even with shapes that are compact and/or symmetric.. The behaviour of the introduced squareness measures and their usefulness, together with applicability to the computing of shape orientation, is verified on three large shape databases: diatoms (ADIAC), MPEG-7 CE-1, and trademarks. The obtained results have shown the benefits of the new measures in the majority of performed experiments.. The proposed measure of squareness has been developed and defined in the continuous space. Therefore some of the measure's properties are only strictly true for continuous shapes. In practise, any discrepancies are insignificant except for very small shapes. A topic for future research would be to develop a measure of squareness defined on discrete data. . Finally, taking into account that and a are strictly positive, and do not depend on β, the last proven inequality Q β (S) ≤ 4a 2 (2+β) gives the required: lim β→∞ Q β (S) = 0. The first experiment applies the squareness measure to the classification of the MPEG-7 CE-1 database of 1400 shapes-see Table 1 . A set of five features had been previously been selected [19] : triangularity, roundness, rectangularity, ellipticity, and convexity, 1 which gave a score of 71.05% on the bulls-eye test when applied with a minimum Mahalanobis distance classifier; this is similar accuracy to other global descriptors [11] -see Zernike moments in Table 1. If the shape measures are augmented by squareness Q β=2 (S) then the score improves to 72.97%. A better accuracy is provided by including Q fit (S) instead: 73.76%. Combining both along with the set of five features produces a further improvement in accuracy (74.74%), showing that the two squareness measures are providing mostly independent information. This is also shown when just the squareness measures are used without the other five shape features. Table 2 Circular variance of orientation estimates for 10 noisy shape instances, averaged over 1400 shapes from MPEG-7 CE-1 database. The rows show results for different normalisations and scalings (in the X direction) of the data before noise is added data moments Accuracies are Q β=2 (S): 23.07%, Q fit (S): 23.00% and the two combined gives 39.60%. We note that it is possible to obtain significantly better bull's eye scores, as demonstrated by state of the art methods such as that by Alajlan et al. [1] which achieves 87.13%. They use a curvature tree to model the shape and topology of objects, and perform dynamic space warping and the Hungarian bipartite matching algorithm to enable exact tree matching. A drawback with such approaches is that computational complexity increases, as does the difficulty in implementation. In Alajlan et al.'s method construction of the tree from a shape containing N boundary points is O(N 2 ), and matching between two trees is O(bnmN 2 ), where b is the maximum branching factor and the trees contain n and m nodes respectively. Another example is the inner distance shape context of Ling and Jacobs [12] , for which construction of the descriptor for one shape is O(N 3 ) and matching two shapes requires O(N 2 ) time. A great advantage of global descriptors such as ours is that the computational cost of using them for matching shapes is independent of the number of boundary points. For the next experiment we also used the MPEG-7 CE-1 database, but in this case to test the effectiveness of squareness for determining shape orientation. There are no groundtruth "correct" orientations for the shapes. In practise what is often required is a means for standardising the orientation of each shape in a consistent manner so that the pose of unseen objects can be set prior to matching against the dataset of models. Therefore we test effectiveness of the orientation estimators by measuring their stability under relatively small changes to the shapes. Shape orientations were computed using Q β (S) and Q fit (S) by determining the orientations that minimised the integral in (19) and the intersection area in (1) respectively. In addition the principal axis method, determined from the shape's moments [24] was applied; this method will be referred to as the 'moment based' method and is probably the most standard method for computing orientation.. Noise was added to the shapes as follows: points on the boundary were perturbed in their normal direction with random magnitude determined by a Normal distribution with standard deviation of 4. The X and Y extents of the shapes are typically in the range [300, 500]. A morphological opening with circular structuring element of diameter 3 was applied to tidy up the shape, following which only the largest connected component was retained. The latter step occasionally removes narrow parts of the shape, which can also be considered to be a typical segmentation error.. Ten noisy versions of each shape were generated, and the circular variance (which is in the range [0, 1]) [13] of the ten estimated orientations was calculated, where orientations modulo 90 • were used. Table 2 lists the mean circular variances averaged over all 1400 shapes. We note that for the squareness method the value of 8 is a good choice for β across the different versions of MPEG-7 CE-1 database. This can be considered to be the best trade off between increasing β so as to increase Q β (S)'s sensitivity to small deviations so that difficult (e.g. compact) shapes can be oriented, whilst avoiding increasing β so much that Q β (S) becomes over-sensitive to noise. Thus, if the test set had contained a different level of noise then a different value of β would be appropriate. It can be seen that, even on the original data, overall the squareness based orientation is slightly better than the moment based approach. Although the moment based approach is generally effective for many of the shapes, especially those that are elongated, there are also many other shapes contained in the MPEG-7 CE-1 database that are problematic. Some of those are symmetric shapes such as stars, squares, and other rotationally symmetric shapes. Others are compact such as the apple in Fig. 9 . However, it is obviously still potentially orientable, and it can be seen that squareness is successful in doing so. The octopus in Fig. 9 is neither rotationally symmetric or compact, but again orientation computed by squareness is much more consistent than that by moments. Fig. 9 Original data with added noise; upper row orientated using moments, lower row orientated using squareness (β = 8) The performance of the orientation estimators is further explored by normalising the data following the approach of Süße and Ditrich [26] in which a shearing and anisotropic scaling is applied to produce a shape that is unorientable by the moments based method. However, as can be seen from Table 2 the mean circular variance of the moments based method is still below unity (i.e. its performance is better than random) since the process of adding noise makes some of the shapes more orientable. Nevertheless, whereas most of the shapes are consistently oriented by both of the methods for the original data, with the normalised data the variances for the moments based method are roughly uniformly distributed in [0, 1] while the squareness method is only slightly affected; see Fig. 10 . We note that the MPEG-7 CE-1 database contains shapes such as stars with high orders of rotational symmetry (e.g. 8) that have intrinsic ambiguity if orientations modulo 90 • are used. This leads to the small proportion of shapes that have high variances for both methods even before the normalisation step. Examples of the moments based method producing inconsistent orientation estimates on the normalised data are given in Fig. 11 . For both shapes the squareness method is very consistent.. When the normalised data is stretched to make it more elongated then it naturally becomes more orientable, and the circular variances reduce (see Table 2 ). Rescaling all the data by 1.1 in the X direction ensures that even the previously problematic shapes (such as the symmetric shapes) are now orientable by the moments based method. Additional elongation further reduces the circular variances of the moments based method. In contrast, the squareness method is already consistent for compact shapes, and unlike the moments based method its circular variance does not decrease as substantially for elongated shapes. The moments based method can generally determine orientations reliably for the elongated shapes produced after the data is scaled in X by 1.1. However, even for such shapes the noise model used to generate the data can still cause problems. Figure 12 shows an example in which the thin parts of the rat (tail, feet, etc.) are prone to truncation when noise is added. It can be seen that the orientations estimated by the moments based method are more prone to disruption than those of the squareness method.. The square fitting method Q fit (S) provides similar advantages to Q(S) for the normalised data and also the close to normalised data (i.e. scaling in the X direction by 1.01). However, it is consistently outperformed by the moments based method in all other situations. Moreover, Q(S) produces better results than Q fit (S) for all versions of the dataset.. Furthermore, the version of Q(S) generalised to provide a rhomboidal measure provides further improvements, yielding more reliable orientation estimates for all the sets of database scalings except for the normalised data (i.e. the most extreme case). Even for the latter case only a small increase in error occurs. As shown in Table 2 , R β δ (S) consistently outperforms both the moments based method and Q fit (S), even for the largest scaling of the data X s = 1.25 for which all the shapes are distinctly elongated.. The second experiment on orientation tests the stability of the orientation estimates in a different way. A sequence of instances of a deforming object are used-despite the shape Fig. 11 Normalised data with added noise; upper row orientated using moments, lower row orientated using squareness (β = 8) Fig. 13 data moments squareness 12 Normalised data with a 1.1 scale factor in X and added noise; upper row orientated using moments, lower row orientated using squareness (β = 8) Fig. 13 Seven examples of shape data from three sequences of deforming objects after landmark based pose normalisation differences the aim is to consistently orient the shapes within the sequence. The data was provided by the Technical University of Denmark for the purpose of building statistical shape models, and consists of a set of landmarks which correspond over the shape sequence. Given such a point correspondence it is straightforward to estimate a similarity transformation to normalise the pose of the shapes. Following Cootes et al. [6] the best transformation, in the least squares sense, is determined for each shape against one arbitrary shape which is chosen as the reference. The mean of the transformed shapes becomes the reference shape, and this process is iterated until convergence. This pre-normalisation acts as a sort of ground-truthing, enabling us to assess the orientation estimates by their circular variances in the same manner as the first experiment. However, in this case the shape variations are more natural-although more severe-compared to the addition of synthetic noise. Figure 13 shows example shapes from each sequence; the two hand sequences each contain 20 frames, while the bottom sequence contains 14 left ventricles from cardiac MR images. Again, Table 3 demonstrates that the squareness based orientations with β = 8 are more consistent than the moment based orientation. In fact, for the hand sequences further improvements are obtained by increasing the value of β to β = 16. All the circular variances are higher than in the previous experiment with the MPEG-7 CE-1 database which is due to the greater variability of the shapes compared with just uniformly adding noise. Also, as expected, the compact shape of the ventricle is especially difficult, but while the moment based method fails the squareness based method performs reasonably well.

O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O


Adaptively quadratic (AQua) image interpolation Abstract-Image interpolation is a key aspect of digital image processing. This paper presents a novel interpolation method based on optimal recovery and adaptively determining the quadratic signal class from the local image behavior. The advantages of the new interpolation method are the ability to interpolate directly by any factor and to model properties of the data acquisition system into the algorithm itself. Through comparisons with other algorithms it is shown that the new interpolation is not only mathematically optimal with respect to the underlying image model, but visually it is very efficient at reducing jagged edges, a place where most other interpolation algorithms fail. This paper presented a novel method for adaptive image interpolation. The algorithm first determines the local quadratic signal class from local image patches and then applies optimal recovery to estimate the missing samples. Additionally, the new interpolation algorithm allows for integrating knowledge about the lens acquisition system into the interpolation itself by using weighted averages as functionals. This tends to produce somewhat sharper images. The general theory of optimal recovery allows estimation of any linear functional of the image. Arbitrary interpolation factors can be used and samples on any lattice can be estimated directly. The focus here has been the interpolation of images by arbitrary factors. Through visual examples this paper has shown that AQua interpolation performs better than several other published interpolation algorithms, especially in structured images and around edges where most other algorithms introduce objectionable artifacts or jaggedness. An extension of AQua interpolation is image rotation where the samples of the rotated image can be estimated directly on the rotated grid.

O O O O B O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O


Lower bounds for bayes error estimation We give a short proof of the following result. Let (X,Y) be any distribution on N/spl times/{0,1}, and let (X/sub 1/,Y/sub 1/),...,(X/sub n/,Y/sub n/) be an i.i.d. sample drawn from this distribution. In discrimination, the Bayes error L*=inf/sub g/P{g(X)/spl ne/Y} is of crucial importance. Here we show that without further conditions on the distribution of (X,Y), no rate-of-convergence results can be obtained. Let /spl phi//sub n/(X/sub 1/,Y/sub 1/,...,X/sub n/,Y/sub n/) be an estimate of the Bayes error, and let {/spl phi//sub n/(.)} be a sequence of such estimates. For any sequence {a/sub n/} of positive numbers converging to zero, a distribution of (X,Y) may be found such that E{|L*-/spl phi//sub n/(X/sub 1/,Y/sub 1/,...,X/sub n/,Y/sub n/)|}/spl ges/a/sub n/ often converges infinitely. THE pattern recognition problem may be formulated as follows: we are given n i.i.d. observations h n f I Y I Y F F F Y n Y n g, drawn from the common unknown distribution of Y on d Â fHY Ig. Given , one must estimate as best as possible by a function g n of and the observations. The best one can hope for is to make an error equal to the Bayes error, v Ã :. It is thus of great importance to be able to estimate v Ã accurately, even before pattern recognition is attempted. Also, a comparison of estimates of v n and v Ã gives us an idea how much room is left for improvement.. In a first group of methods, v Ã is estimated by an estimate v n of the error probability v n of some consistent classification rule g n . As such, this problem has been attempted by Fukunaga and Kessel [9] , Chen and Fu [2] , Fukunaga and Hummels [8] , and Garnett and Yau [10] , to cite just the early contributions. Concerning the error estimation of specific classification rules see Chapter 10 in McLachlan [11] . Clearly, if the estimate v n we use is consistent in the sense that v n À v n 3 H with probability one as n 3 I, and the rule is strongly consistent, then v n 3 v Ã with probability one. In other words, we have a consistent estimate of the Bayes error probability. The problem is that even though for many classifiers, v n À v n can be guaranteed to converge to zero rapidly, regardless what the distribution of Y is (see Chapters 8, 23, 24 , and 31 of Devroye et al. [7] ), in view of Cover [3] and Devroye [4] , the rate of convergence of v n to v Ã using such a method may be arbitrarily slow. Thus, we cannot expect a good performance for all distributions from such a method. The question thus is whether it is possible to come up with another method of estimating v. to zero rapidly for all distributions. Unfortunately, there is no method that guarantees a certain finite sample performance for all distributions. This disappointing fact is reflected in the following negative result (Theorem 8.5 of Devroye et al. [7] ). Theorem 1. For every n, for any estimate 0 n I Y I Y F F F Y n Y n of the Bayes error probability v Ã , and for every b H, there exists a distribution of Y , such that. The counterexamples in Theorem 1 vary with n, so it may still be possible that for every fixed distribution for Y , there exists a universal rate of convergence to zero for. The purpose of this note is to show that this too is impossible. We show the following:. Theorem 2. For any sequence f n g of positive numbers converging to zero, a distribution of Y on fIY PY QY F F Fg Â fHY Ig may be found such that. We note that for the v I error in density estimation, similar global lower bounds were obtained by Devroye [5] , [6] and Birge Â [1] . We also note that the phrase ªinfinitely oftenº cannot be dropped from Theorem 2. Indeed, there exist deterministic sequences n with j n À v Ã j an infinitely often for some constant : just consider the dyadic sequence. With 0 n n , we thus obtain a very good estimate along an (unknown) subsequence. In a standard pattern recognition design process, one takes a number of features, and evaluates whether these suffice or will do for discrimination. If so, a discrimination method is designed. If not, more or different features must be considered. The quality of a collection of features is measured by the Bayes probability of error, v Ã . Thus, the first phase of any pattern recognition method is based on estimates of v Ã (even before a discriminant is picked!). In this paper, we show that no one can trust any Bayes error estimate, and that it is futile to even let the sample size tend to infinity. It is only possible to give error bounds or confidence bands for v Ã under assumptions on the distribution of the data. In practice, one can thus never claim to have a universally superior feature extraction or Bayes error estimation method, no matter how many simulations are performed and no matter how large the sample sizes are.

B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O


Dynamic web page authoring by example using ontology-based domain knowledge Authoring dynamic web pages is an inherently difficult task. We present DESK, an interactive authoring tool that allows the customization of dynamic page generation procedures with no apriori tool-specific skill requirements from authors. Our approach consists of combining Programming By Example (PBE) techniques with an ontology-based representation of knowledge displayed in web pages. DESK acts as a client-side complement of a dynamic web page generation system, PEGASUS, which generates HTML pages from a formally structured domain model and an abstract presentation model. Authorized users can modify the internal presentation model by editing the generated HTML pages with DESK in a WYSIWYG environment. DESK keeps track of all user's actions and exploits the explicitly represented domain semantics to enhance the power of PBE techniques. An increasing proportion of web contents and functionality are accessed today through dynamically generated web pages, yet the development of dynamic pages remains a complex task that requires advanced programming skills. Languages and frameworks like XSL and JSP/ASP can greatly simplify the development and maintenance of dynamic web pages, but still require advanced technical knowledge that domain experts, graphic designers or even average programmers may lack. Development environments have been provided for these technologies that help manage projects and provide code browsing and debugging facilities, but one still has to edit the code. It is difficult to provide WYSIWYG tools for the development of dynamic pages because it is hard to describe procedural behavior visually.. DESK [13] (Dynamic web documents by Example using Semantic Knowledge) takes up the challenge of supporting the customization of page generation procedures in an editing environment that looks like an HTML editor from the author point of view. DESK acts as a client-side complement of a dynamic web page generation system, PEGASUS [5] , [12] , [14] , which generates HTML pages from a formally structured domain model and an abstract presentation model. The PEGASUS presentation model specifies which pieces of knowledge should be presented and how when a certain unit of information from the domain model is output to the user. Instead of using the PEGASUS modeling language, authorized users can modify the internal presentation model by editing in DESK the HTML pages generated by PEGASUS. DESK follows the Programming By Example (PBE) approach [6] , [7] , to infer changes that affect whole classes of knowledge from user's actions on the presentation of a specific unit.. DESK keeps track of all user's actions on edited documents, builds-up a model of the (HTML) document display structure (syntax), identifies domain model data fragments (semantics) in the document, and uses this information to give a semantic meaning to editing actions. DESK includes an inference agent used to recognize high-level transformations. The agent is constantly listening to the user's actions to infer complex modifications. The agent's behavior is configured using preactivation hints that activate different heuristics for recognizing global widget changes.. A tool like DESK allows widening the spectrum of authors that can participate in otherwise abstract and complex model-based environments like PEGASUS. Inversely, our work shows that PBE techniques can benefit from a knowledge-based approach, which provides knowledgeable models of the user interface and Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. The development and maintenance of dynamic interfaces for web applications is a costly task. Our work aims at combining the ease of use of an interactive authoring tool with the power of the model-based and knowledge-based approach. DESK provides the designer with an intuitive authoring environment capable of editing complex web page designs. Our authoring tool is based on the Programming by Example paradigm, where the user supplies the system with an example of what s/he wants to get and the system infers the changes to dynamic page generation procedures.. By tracking the user as s/he edits pages, DESK obtains enriched information from user's actions, comparing such elaborated information with semantic domain knowledge to provide the user with continuos assistance during the edition process. Changes are automatically carried out in the server by using domain knowledge from PEGASUS, using presentation templates for making generic changes to the way complex data structures are visualized. DESK tries to infer maximal information from user's actions and from existing semantic knowledge. All this semantic information is parameterized, editable and independent from the application domain.. In a sense DESK follows the inverse path of a dynamic page generation system, here PEGASUS. This path has a higher level of ambiguity because for a change to the document by the user several interpretations are possible and different generalizations can be considered. DESK solves this ambiguity by using heuristics like extracting structured knowledge from the domain model of PEGASUS. Sometimes this is not enough and the system can make mistakes. The purpose of this work is not to build a system that never fails, but to make a useful authoring tool capable of inferring correct actions in a reasonable number of cases. DESK can successfully infer the following types of changes in documents:. • Insertion, deletion and modification of HTML fragments along the presentation template and generation rules.. • Insertion, deletion and modification of HTML tags surrounding domain elements along the presentation template and generation rules.. • Creation, deletion, modification and automatic transformation between widgets (i.e. combo-boxes, selection lists, and tables).. • Insertion, deletion modification and moving around dynamic text and multimedia references from the domain model.. A working prototype of DESK has been developed where most of the features described in this paper have been implemented. We have considered several scenarios dealing with domains like academic institution information, e-commerce catalogs, travel information, and educational materials. We have worked with ontologies taken from standard repositories like the ontology libraries from the DAML project [8] . We are currently improving our tool to deal with more complex edition cases such as advanced widget manipulation, and dealing with new models like user and platform. Monitoring user actions is a common practice to provide so-called intelligent interaction between users and web applications. One example of this approach is AVANTI [19] , in which the system tracks the user interaction in a web browser designed for universal accessibility, as a front-end of the AVANTI information system. This tool uses Unified User Interfaces (U 2 Is) from Unified User Interface Development methodology (U 2 ID) to achieve selfadaptability, monitoring any kind of user during the navigation, including elderly and disabled people. To this respect, DESK monitors user's actions by using configurable and parametrical information from different knowledge models. An intelligent agent tracks user's steps during the interaction to infer atomic changes, this way our authoring tool supplies the user with an easy-to-use web browser and WYSIWYG edition tool to automate changes to web presentations.. Turquoise [16] is also an intelligent browser and editor for the web that allows users to create dynamic pages by example rather than by writing program code. With such authoring tool, users without programming experience can create scripts that combine data from several web pages, automate repetitive browsing or editing tasks, convert other data formats into HTML, and process submitted forms. Scripts are demonstrated by familiar browsing and editing actions, which Turquoise records and generalizes into a program. Like DESK, Turquoise is based on the Programming By Demonstration paradigm [4] , [6] , [7] , where the system infers procedural information from examples of what the user wants to achieve. Turquoise operates by inferring scripts from user's actions by copying HTML contents to a special window. Similarly, in Scrapbook [20] the users can demonstrate which portions of web pages they are interested in by creating a personal page, that is, selecting data on a web browser and copying it to the single personal page. Web data is copied directly from Netscape Navigator using APIs of those browsers. Once the personal page is created, the system automatically updates it by extracting the user-specified portions from the latest web pages. Thus, the user can browse only the necessary information on a single page and avoid repetitive access to multiple web pages. In contrast, DESK allows the user to freely edit the HTML document, inferring a great deal of changes to domain and presentation models.. In the TriIAs system [1] the user interacts with a web-based application like a travel planning agent that is intended to create a schedule for a trip satisfying all of the constraints entered by the User. TriIAs calls an information extraction trainer that is able to learn new wrappers for extracting relevant pieces of information from web documents. Wrappers [11] , [18] , provide a uniform access to the information stores in heterogeneous repositories like data bases, files and so forth. The extraction of structured information, such as context semantics used and generated by DESK from a semi-structured document (HTML and JSP code), is very similar to the way wrappers operate. In DESK, an agent tracks the user during the interaction for relieving him/her from having to complete repetitive tasks (e.g. copying items from a widget to another) being completed by the authoring tool. In that sense, our agent uses semantic enriched information present in the monitoring model, as well as semantics coming from our ontology and domain-knowledge, making easy the inference step, without the necessity of using extraction languages.. WebSheets [23] is also a WYSIWYG authoring tool based on the Programming By Demonstration paradigm for building dynamic web pages that access and modify databases. WebSheets uses QBE (Query By Example) by which queries are generated in the authoring environment from user actions, using an SQL-like language to access databases for requesting information. In DESK the domain information is stored using an ontology-based semantic model rather than a relational model. To this end, our authoring tool can afford complex changes made by the user, relating domain objects and widget structures, so that the variety of changes can be quite heterogeneous, and all of them can be applied to both domain and presentation model.. On the other hand, in [2] a model-based approach for web engineering is proposed. This work consists of an architecture for the reverse engineering of web pages, with a view to apply forward engineering subsequently. Reengineering methods are then applied to produce new user interfaces for multiple contexts of use, thus creating a capability to rapidly produce user interfaces for different computer platforms, access devices, etc. To this respect, DESK provides a reengineering mechanism by following the inverse of the path followed by PEGASUS, starting from the HTML code back to the constructor models [13] .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Interval arithmetic: From principles to implementation Abstract. We start with a mathematical definition of a real interval as a closed, connected set of reals. Interval arithmetic operations (addition, subtraction, multiplication, and division) are likewise defined mathematically and we provide algorithms for computing these operations assuming exact real arithmetic. Next, we define interval arithmetic operations on intervals with IEEE 754 floating point endpoints to be sound and optimal approximations of the real interval operations and we show that the IEEE standard's specification of operations involving the signed infinities, signed zeros, and the exact/inexact flag are such as to make a correct and optimal implementation more efficient. From the resulting theorems, we derive data that are sufficiently detailed to convert directly to a program for efficiently implementing the interval operations. Finally, we extend these results to the case of general intervals, which are defined as connected sets of reals that are not necessarily closed. One of the earliest lessons of digital numerical computation was that, although most programs give highly accurate results, it can happen that rounding errors build up in such a way that none of the many decimals in the result is meaningful. A good early summary of this is Forsythe's [1970] "Pitfalls of Computation." Numerical analysis emerged as the science of determining conditions under which algorithms give accurate results.. Instead of using a single floating-point number as approximation for the value of a real variable in the mathematical model under investigation, interval arithmetic acknowledges limited precision by associating with the variable a set of reals as possible values. For ease of storage and computation, these sets are restricted to intervals. The computation rules aim at maintaining the property of containing all possible values.. Initially, the attraction of interval arithmetic was that it would not be necessary to analyze whether the conventional pointwise floating-point computations are safe. As the interval results contain all possible values, a narrow interval indicates success. A wide interval does not prove that a conventionally computed result is wrong, but it does indicate a risk.. When evaluating expressions in interval arithmetic, the interval result tends to be disappointingly large. This is due partly to its conservative nature: the result has to contain all possible values, including those where rounding errors combine in an unfavourable way. But the main cause of wide intervals is that interval arithmetic does not identify different occurrences of the same variable. Because it treats these as if they were unique occurrences, the result is much wider than one could reasonably expect.. Interval arithmetic has the property of correctness: result intervals are guaranteed to contain the real number that is the value of the expression. Implementations can only realize this potential by rounding floating-point operations in the right direction. In the early days, such code was not portable, if rounding direction could be controlled at all. In addition, in the early days of interval arithmetic, the extra demands of interval computation on the processor and on memory (several operations instead of one; two numbers to be stored instead of one) posed serious problems.. The utility of interval algorithms has shifted over time from automatically performing rigorous error analysis to solving nonlinear problems, including: systems of nonlinear equations, global optimization (unconstrained and constrained), and integrating differential equations. One of the keys to overcoming overly conservative interval bounds, even in the presence of rounding, is an algorithm that is a contracting map. A contracting map produces a sequence of interval results that are successively narrower subintervals of each other. With infinite precision interval arithmetic, the final result width of a contracting map can approach zero. In practice, with finite-precision arithmetic and positive-width interval parameters, a point of diminishing returns is reached, a point at which finite precision prevents further contraction of the result interval.. Most expositions of interval arithmetic require intervals to be bounded from above and below. They disallow unbounded intervals and they disallow division of intervals when the denominator contains zero. In practice, however, these classical restrictions on interval operations must sometimes be subverted. The restriction of division to interval denominators not containing zero means that the interval division operator, like its real counterpart, is a partial function, and this lack of totality creates the same kinds of problems as arise for pointwise computation. Tests have to be inserted for the presence of zero in the divisor interval, which then needs to be split. On the other hand, a correct and total interval division operation allows the algorithm to proceed independently of whether the divisor interval contains zero or not.. In this paper, we present a system of interval arithmetic that has the following properties:. (1) Correctness (2) Totality (3) Closedness (4) Optimality (5) Efficiency. Contributions to these goals are scattered over a number of publications. Looking at any one of these, one may get the impression that the state of the art is far from achieving these goals. But, as we show in this paper, relatively little needs to be added to the combined literature. We now comment on each of the above five criteria.. Correctness. The criterion for correctness of a definition of interval arithmetic is that the "Fundamental Theorem of Interval Arithmetic" hold: when an expression is evaluated using intervals, it yields an interval containing all results of pointwise evaluations based on point values that are elements of the argument intervals. Existing proofs of this restrict the types of interval arithmetic definitions for which it can be used. To minimize such a restriction, we give a new and more general proof that is based on elementary facts of set theory. Reals are hard to represent and operate on. Paradoxically, sets of reals have tractable and sound approximations. Undefined results of real operations have given trouble in various forms, such as overflow and division by zero. We have shown that these are avoidable.. What most people know about interval arithmetic is that it is a safe alternative for expression evaluation. The practitioner rightly suspects that the usual examples, where expression evaluation goes wildly wrong, are contrived. What is not widely known is that interval arithmetic is a powerful method for extending numerical computation into areas such as nonlinear equation solving and nonconvex global optimization [Hansen 1992; Van Hentenryck et al. 1997] , where noninterval methods experience serious difficulties.. We should emphasize that we do not attempt to decree that the full details of interval division, as revealed in this paper, must be implemented in any interval arithmetic system. Indeed, in any implementation, efficiency and simplicity of code have to be weighed against minimizing departures from optimality. Our work can be interpreted as saying to implementers: "Here are all the cases in which topologically distinguishable results appear-there is no more detail. Preserving correctness, simplify as much as you need to." The formulas for interval arithmetic operations on closed, connected sets of reals have in fact been used successfully in several interval-arithmetic-based constraint solvers [Hickey 2000a [Hickey , 2000b Hickey et al. 2000] .. In our view, interval arithmetic should be simply and clearly defined in terms of the underlying mathematical model of real arithmetic. Much of scientific and numerical computation is based on a mathematical model in which the variables range over the reals. In conventional computation, floating-point operations are substituted for the real arithmetic operations, with all the well-documented dire consequences; see Forsythe [1970] for an early warning.. The current state of the art makes it possible to regard interval computations as computer-generated proofs that certain reals (real reals) belong to certain small sets of reals (intervals with floating-point endpoints that are not much wider than the limits imposed by the processor's precision). This breakthrough has taken place by piecemeal improvements through the long history of interval arithmetic.. In this article, we have demonstrated that one can formulate a theory of Interval Arithmetic based on extended notions of intervals that allow intervals to be unbounded and nonclosed, and that this results in an elegant theory that is correct, closed, total, optimal, and efficient. Contributions to division by an interval containing zero span a long period. The initial contribution by Kahan [1968] was important if only for pointing out that such an operation can be usefully defined. Kahan capitalized on the fact that, whether division results in a connected set or not, only a single pair of reals is needed to specify the result. Such a pair is, according to Kahan, an interior interval (a connected set) or an exterior interval (a union of two connected sets).. Various expressions for the endpoints of the result of interval division of two bounded real intervals can be found in Hammer et al. [1993] , Ratz [1996] , Novoa [1993] , Hansen [1992] , and Walster [1998] . The book [Hansen 1992 ] defines an interval division that returns results between functional and relational division. Except for Ratz [1996] , all of these expressions are presented as defininitions of division for an extended interval arithmetic and many give inconsistent answers for the case of [0, 0]/[0, 1]. As far as we know, Ratz [1996] , was the first to define interval division set theoretically in the case where the denominator contains zero and to prove that his rules are correct, at least for the case of closed and bounded intervals.. The work on BNR Prolog [Older 1989 ] was in many ways the most advanced version of interval arithmetic when its Unix version came out in 1994. Like Pascal-XSC, BNR Prolog optimally determines that, for example, −0.5, 0.5 ∩ ( 1, 1 / −1, 1 ) is empty. Unlike the Pascal-XSC code in Hammer et al. [1993] , BNR Prolog optimally determines −1, 0 ∩ ( 1, 1 / 1, ∞ ) to be empty because the quotient does not contain its greatest lower bound. Unfortunately, apart from Older [1989] and Benhamou and Older [1997] , nothing seems to have been published about the arithmetic of BNR Prolog.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Non-uniform deblurring for shaken images Blur from camera shake is mostly due to the 3D rotation of the camera, resulting in a blur kernel that can be significantly non-uniform across the image. However, most current deblurring methods model the observed image as a convolution of a sharp image with a uniform blur kernel. We propose a new parametrized geometric model of the blurring process in terms of the rotational velocity of the camera during exposure. We apply this model to two different algorithms for camera shake removal: the first one uses a single blurry image (blind deblurring), while the second one uses both a blurry image and a sharp but noisy image of the same scene. We show that our approach makes it possible to model and remove a wider class of blurs than previous approaches, including uniform blur as a special case, and demonstrate its effectiveness with experiments on real images. Everybody is familiar with camera shake, since the resulting blur spoils many photos taken in low-light conditions. While significant progress has been made recently towards removing this blur from images, almost all approaches model the blurred image as a convolution of a sharp image with a spatially uniform filter [5, 10, 24, 29] . However, real camera shake does not in general cause uniform blur [13] , as illustrated by Figure 1 .. In this paper we propose a geometrically consistent model of non-uniform image blur due to camera shake, arising from rotations of the camera. We develop a global representation of such parametrically non-uniform blur, using a single "blur kernel" analogous to (but different from) a convolution kernel, and demonstrate its ability to model a more general class of blurs than previous approaches. We demonstrate the effectiveness of our model by using it to replace the uniform blur model in two existing approaches to camera shake removal, and show quantitative and qualitative improvements in the results. In addition, we show that uniform blur is a special case of our model. Specifically, we consider the problems of "blind" deblurring, where only a single blurred image is available, and the case where an additional sharp but noisy image of the same scene is available. To approach these two problems, we apply our model within the frameworks proposed by Miskin & MacKay [16] and Fergus et al. [10] for the blind case, and Yuan et al. [29] for the case of a noisy / blurred image pair. We have proposed a new model for camera shake, derived from the geometric properties of cameras, and applied it to two deblurring problems within the frameworks of existing camera shake removal algorithms. The model assumes rotation of the camera about its optical center during exposure, and is temporally-agnostic on the distribution over camera orientations. Note, however, that camera rotations that are off the optical center can be modeled by camera rotations about the optical center together with translation; these translations should generally be small for rotation centers that are not far from the optical center. The model is not applicable for non-static scenes, or nearby scenes with large camera translations where parallax effects may become significant.. In the future, we plan to investigate the use of our general bilinear model to other non-uniform blurs. Also, since our model is valid over the whole image, it may be possible to estimate the sharp image and blur kernel simultaneously, as suggested by the result in Figure 4 . We compare in this section our results to those obtained with the code provided by Fergus et al. [10] on both synthetic and real data. Implementation details are discussed in Section 7. Figures 3 and 4 show blind deblurring results on images blurred by real camera shake. Our approach is able to capture and remove the blur, while the uniform algorithm of Fergus et al. fails to find meaningful kernels or good deblurred results. This is explained by both the short focal length (typical of compact cameras), and the fact that the kernels estimated using our algorithm exhibit significant inplane components. Figure 5 shows results for blind deblurring of synthetic images using the two methods, and demonstrates two im- The result of blind deblurring on a real camera shake image, captured with a shutter speed of 1 2 second, using the algorithm of Fergus et al. and our non-uniform approach. Our approach is able to recover a useful kernel and a good deblurred image, while the uniform algorithm of Fergus et al. fails to find a meaningful kernel. The rotational kernel visualized in the right-hand column shows the non-zero kernel elements plotted as points in the 3D rotational parameter space (θX , θY , θZ ). Each of the cuboid's faces shows the projection of the kernel onto that face. Note that our estimated rotational kernel has a significant in-plane component (non-zeros over many values of θZ ). portant points: first, small out-of-plane (e.g. Y -axis) components of a blur are sufficiently uniform that the two models both perform well, although the rotational model performs better. Second, only our approach is able to remove in-plane (Z-axis) blurs, which cannot be represented as convolutions. In this case, and also for the largest outof-plane blurs, we are able to recover a good sharp image, whereas the uniform approach breaks down due to the nonuniformity of the blur. camera about its Y and Z-axis, and the kernels and deblurred results for different cases. We compare the results of blind deblurring on a range of blur sizes and noise levels, and the reconstruction errors are summarized in the tables at the bottom. For each single-axis blur, the table contains the root-mean-square (RMS) errors between the deblurred results and the ground-truth sharp image for 10, 20, and 30 pixel blurs, using our model (R) and the uniform model (U). In each cell we also show, in parentheses, the ratio between the RMS error and the corresponding error for that blurry image deblurred with the ground-truth kernel.. for blur kernel estimation.. In order to estimate the blur kernel, we solve the following problem:. where, by analogy with Eqn. (9) ,ĝ(f N ,ŵ) = B Nŵ , where B N is the matrix whose columns contain all the projectively transformed copies of f N . This least-squares formulation with non-negative 1 constraints is an instance of the Lasso problem [26] , for which efficient optimization algorithms Ours, unconstrained Ground-truth Figure 6 . Blind deblurring of a real uniform blur. A real camera shake blur from the dataset of [13] , with the deblurred results and kernels for four cases; the original algorithm of Fergus et al., our approach with a large focal length and no in-plane rotation (θZ = 0), our approach with θZ unconstrained, and the groundtruth (uniform) kernel. Notice that this unconstrained kernel has the same diagonal shape as the true blur, and that the non-zeros are centered around a single value of θZ . exist [9, 15] , and we use these algorithms here.. For comparison, we have also implemented this algorithm for uniform blurs, using a matrix B N in Eqn. (12) whose columns contained translated versions of f N , rather than projectively transformed versions.. Image Reconstruction. Having estimated the blur kernel, Yuan et al. [29] propose several modifications to the Richardson-Lucy (RL) algorithm, which take advantage of the fact that it is possible to recover much of the lowfrequency content of f from a denoised version of f N . Images deblurred with the standard RL algorithm often exhibit "ringing" artifacts -low-frequency ripples spreading across the image, such as in Figure 4 -but using the denoised image it is possible to disambiguate the true low frequencies from these artifacts, and largely remove them from the result. We refer the reader to [29] for full details of the modified RL algorithm, omitted here for brevity. We have adapted the algorithm for our non-uniform blur model, along the same lines as for the standard RL algorithm in Section 5.. In this section, we present results with noisy / blurry image pairs, and refer the reader to Section 7 for implementation details. Figure 7 shows a comparison between the uniform model and our rotational one, using the algorithm described above to estimate the blur kernels. Having estimated the kernel, we deblur the blurred images using the modified RL algorithm of Yuan et al. [29] . As can be seen from the deblurred images obtained with the two models, our results exhibit more detail and fewer artifacts than those using the uniform blur model. Previous work on non-uniform deblurring has focused on piecewise-uniform blurs arising from multiple moving objects in the scene [7, 12] , spatially varying combinations of localized uniform blurs [17, 27] , or blur arising from rotations of planar objects in the scene [24] . However, apart from Sawchuk [22] , who assumes a known transformation, these approaches generally rely on the assumption that the blur is locally uniform, and do not consider global models for continuously varying blur, such as those arising from arbitrary rotations of the camera about its optical center during exposure, as modelled in our work.. If the blur kernel is known, standard deblurring techniques such as the Wiener filter (for uniform blur) or the Richardson-Lucy algorithm (for general blur) can be used to recover the original sharp image [3, 8, 14, 19, 21] . However, blind image deblurring with an unknown blur kernel is a difficult problem, which is typically addressed by first estimating the kernel, and then estimating the sharp image assuming the kernel is known. Blind camera shake removal from a single image using a uniform blur model and priors on image statistics has been addressed in [10, 13, 16, 23] . To simplify the deblurring problem, others have considered using additional information in the form of additional blurry images [6, 20] , or a sharp but noisy image of the same scene [29] . The implementation of the blind kernel estimation method presented in Section 5 is based on the code made available by Miskin & MacKay [16] and by Fergus et al. [10] . We have modified the algorithm to use a kernel defined over rotations of the camera rather than a convolution kernel, and replaced the parameter update equations with the corresponding versions derived for our bilinear blur model (see [1] ). The implementations of the Richardson-Lucy algorithm, and the modified RL algorithm of Yuan et al. [29] are our own, and we use these implementations for both blur models when comparing results.. Sampling the set of rotations. One important detail to consider is how finely to discretize the rotational parameter θ. Undersampling the set of rotations will affect our ability to accurately reconstruct the blurred image, but sampling it too finely will lead to unnecessary calculations. Since the kernel is defined over the 3 rotational parameters θ X , θ Y and θ Z , doubling the sampling resolution increases the number of kernel elements by a factor of 8, so the choice is an important one. In practice, we have found that a good choice of sample spacing is one which corresponds approximately to a displacement of 1 pixel at the edge of the image. Since we are fundamentally limited by the resolution of our images, setting the resolution higher leads to redundant rotations, that are indistinguishable from their neighbours. We set the size of our kernel along each dimension in terms of the size of the blur we need to model, typically a few degrees along each dimension of θ, e.g.. Multiscale implementation. Both of the kernel estimation algorithms presented here are applied within a multiscale framework, starting with a coarse representation of image and kernel, and repeatedly refining the estimated kernel at higher resolutions. In the case of blind deblurring, the reason for this is that the variational algorithm is susceptible to poor local minima, and performing the optimization at increasingly fine scales can help to avoid this. When deblurring with noisy / blurry image pairs, the problem is that the kernel at the original resolution may have thousands or tens of thousands of elements. However, very few of these should have non-zero values. To solve Eqn. (12) directly at full resolution would involve transforming f N for every possible rotation under consideration and storing all the copies simultaneously in B N . This represents a significant amount of redundant computation, since most of these copies will correspond to zeros in the kernel, and this may furthermore cause B N to become impractically large.. In both of the applications presented in this paper, we use the solutionŵ s at each scale s to constrain the solution at the next scaleŵ s+1 , by defining an "active region" [29] . Also shown for illustration are a selection of the local filters generated by the rotational kernel. As can be seen in the close-ups, our result contains more details and fewer artifacts from the deblurring than when using the uniform blur model. whereŵ s is non-zero, and constraining the non-zeros at the next scale to lie within this region. We first build Gaussian pyramids for the blurred image (and noisy image, if applicable), and at the finest scale s = 0, define the active region to cover the full kernel. At each scale s, we find the optimal kernelŵ s for that scale. We then upsampleŵ s to the next scale (s + 1) using bilinear interpolation, find the non-zero elements of this upsampled kernel, and dilate this region using a 3 × 3 × 3 cube. When finding the optimal kernel w s+1 , we fix all elements outside the active region to zero. We repeat this process at each scale, until we have found the optimal kernel at the finest scale.. Geometric and photometric registration. For the case of noisy / blurry image pairs, the two images are simply taken one after the other with a hand-held camera, so they may not be registered with each other. Thus, we estimate an approximate registration θ 0 between them at the coarsest scale, using an exhaustive search over a large set of rotations, for example ±10 • about all 3 axes, and remove this mis-registration from the noisy image. To compensate for the difference in exposure between the noisy and blurry images, at each scale s, after computingŵ s for that scale, we estimate a linear rescaling a by computing the linear least-squares fit between the pixels of g s and those ofĝ s (ŵ s , f N,s ), and apply this to the noisy image, i.e. f N ← af N .

O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O