the problem of degeneracy in structure and motion recovery from uncalibrated image sequences the aim of this work is the recovery of 3d structure and camera projection matrices for each frame of an uncalibrated image sequence in order to achieve this correspondences are required throughout the sequence a significant and successful mechanism for automatically establishing these correspondences is by the use of geometric constraints arising from scene rigidity however problems arise with such geometry guided matching if general viewpoint and general structure are assumed whilst frames in the sequence and or scene structure do not conform to these assumptions such cases are termed degenerate in this paper we describe two important cases of degeneracy and their effects on geometry guided matching the cases are a motion degeneracy where the camera does not translate between frames and a structure degeneracy where the viewed scene structure is planar the effects include the loss of correspondences due to under or over fitting of geometric models estimated from image data leading to the failure of the tracking method these degeneracies are not a theoretical curiosity but commonly occur in real sequences where models are statistically estimated from image points with measurement error we investigate two strategies for tackling such degeneracies the first uses a statistical model selection test to identify when degeneracies occur the second uses multiple motion models to overcome the degeneracies the strategies are evaluated on real sequences varying in motion scene type and length from 13 to 120 frames

recursive estimation of motion structure and focal length presents a formulation for recursive recovery of motion pointwise structure and focal length from feature correspondences tracked through an image sequence in addition to adding focal length to the state vector several representational improvements are made over earlier structure from motion formulations yielding a stable and accurate estimation framework which applies uniformly to both true perspective and orthographic projection results on synthetic and real imagery illustrate the performance of the estimator

tracking while zooming using affine transfer and multifocal tensors this paper presents algorithms for tracking unknown objects in the presence of zoom since prior models are unavailable point and line matches in affine views are used to characterize the structure and to transfer a fixation point into new images in a sequence because any affine projection matrix is permitted the intrinsic camera parameters such as focal length may change freely also since the techniques do not require long feature tracks a further desirable property is insensitivity to partial occlusion caused for instance by part of the object falling off the image plane while zooming in if only point matches are available a previous method based on factorization is applied when also incorporating lines the affine trifocal and quadrifocal tensors are used for tracking in monocular and stereo systems respectively methods for computing the tensors minimizing algebraic error are developed in comparison with their projective counterparts the affine tensors offer significant advantages in terms of computation time and convenience of parameterization and the relations between the different tensors are shown to be much simpler successful tracking is demonstrated on several real image sequences

structure and motion from line segments in multiple images this paper presents a new method for recovering the three dimensional structure of a scene composed of straight line segments using the image data obtained from a moving camera the recovery algorithm is formulated in terms of an objective function which measures the total squared distance in the image plane between the observed edge segments and the projections perspective of the reconstructed lines this objective function is minimized with respect to the line parameters and the camera positions to obtain an estimate for the structure of the scene the effectiveness of this approach is demonstrated quantitatively through extensive simulations and qualitatively with actual image sequences the implementation is being made publicly available

using points at infinity for parameter decoupling in camera calibration the majority of camera calibration methods including the gold standard algorithm use point based information and simultaneously estimate all calibration parameters in contrast we propose a novel calibration method that exploits line orientation information and decouples the problem into two simpler stages we formulate the problem as minimization of the lateral displacement between single projected image lines and their vanishing points unlike previous vanishing point methods parallel line pairs are not required additionally the invariance properties of vanishing points mean that multiple images related by pure translation can be used to increase the calibration data set size without increasing the number of estimated parameters we compare this method with vanishing point methods and the gold standard algorithm and demonstrate that it has comparable performance

fitting parameterized three dimensional models to images model based recognition and motion tracking depend upon the ability to solve for projection and model parameters that will best fit a 3 d model to matching 2 d image features the author extends current methods of parameter solving to handle objects with arbitrary curved surfaces and with any number of internal parameters representing articulation variable dimensions or surface deformations numerical stabilization methods are developed that take account of inherent inaccuracies in the image measurements and allow useful solutions to be determined even when there are fewer matches than unknown parameters the levenberg marquardt method is used to always ensure convergence of the solution these techniques allow model based vision to be used for a much wider class of problems than was possible with previous methods their application is demonstrated for tracking the motion of curved parameterized objects

modular elliptic curves and fermat s last theorem when andrew john wiles was 10 years old he read eric temple bell s the last problem and was so impressed by it that he decided that he would be the first person to prove fermat s last theorem this theorem states that there are no nonzero integers a b c n with n 2 such that an bn cn the object of this paper is to prove that all semistable elliptic curves over the set of rational numbers are modular fermat s last theorem follows as a corollary by virtue of previous work by frey serre and ribet

egomotion estimation using assorted features we propose a novel minimal solver for recovering camera motion across two views of a calibrated stereo rig the algorithm can handle any assorted combination of point and line features across the four images and facilitates a visual odometry pipeline that is enhanced by well localized and reliably tracked line features while retaining the well known advantages of point features the mathematical framework of our method is based on trifocal tensor geometry and a quaternion representation of rotation matrices a simple polynomial system is developed from which camera motion parameters may be extracted more robustly in the presence of severe noise as compared to the conventionally employed direct linear subspace solutions this is demonstrated with extensive experiments and comparisons against the 3 point and line sfm algorithms

dynamic 3d models with local and global deformations deformable superquadrics the authors present a physically based approach to fitting complex three dimensional shapes using a novel class of dynamic models that can deform both locally and globally they formulate the deformable superquadrics which incorporate the global shape parameters of a conventional superellipsoid with the local degrees of freedom of a spline the model s six global deformational degrees of freedom capture gross shape features from visual data and provide salient part descriptors for efficient indexing into a database of stored models the local deformation parameters reconstruct the details of complex shapes that the global abstraction misses the equations of motion which govern the behavior of deformable superquadrics make them responsive to externally applied forces the authors fit models to visual data by transforming the data into forces and simulating the equations of motion through time to adjust the translational rotational and deformational degrees of freedom of the models model fitting experiments involving 2d monocular image data and 3d range data are presented

on the modularity of elliptic curves over q wild 3 adic exercises in this paper building on work of wiles wi and of wiles and one of us r t tw we will prove the following two theorems see 2 2 theorem a if e q is an elliptic curve then e is modular theorem b if gal q q gl2 f5 is an irreducible continuous representation with cyclotomic determinant then is modular we will first remind the reader of the content of these results and then briefly outline the method of proof if n is a positive integer then we let 1 n denote the subgroup of sl2 z consisting of matrices that modulo n are of the form 1 0 1

constraints on deformable models recovering 3d shape and nongrid motion abstract inferring the 3d structures of nonrigidly moving objects from images is a difficult yet basic problem in computational vision our approach makes use of dynamic elastically deformable object models that offer the geometric flexibility to satisfy a diversity of real world visual constraints we specialize these models to include intrinsic forces inducing a preference for axisymmetry image based constraints are applied as extrinsic forces that mold the symmetry seeking model into shapes consistent with image data we describe an extrinsic force that applies constraints derived from profiles of monocularly viewed objects we generalize this constraint force to incorporate profile information from multiple views and use it to exploit binocular image data for time varying images the force becomes dynamic and the model is able to infer not only depth but nonrigid motion as well we demonstrate the recovery of 3d shape and nonrigid motion from natural imagery

closed form solutions for physically based shape modeling and recognition the authors present a closed form physically based solution for recovering a three dimensional 3 d solid model from collections of 3 d surface measurements given a sufficient number of independent measurements the solution is overconstrained and unique except for rotational symmetries the proposed approach is based on the finite element method fem and parametric solid modeling using implicit functions this approach provides both the convenience of parametric modeling and the expressiveness of the physically based mesh formulation and in addition can provide great accuracy at physical simulation a physically based object recognition method that allows simple closed form comparisons of recovered 3 d solid models is presented the performance of these methods is evaluated using both synthetic range data with various signal to noise ratios and using laser rangefinder data

structure from motion using line correspondences a theory is presented for the computation of three dimensional motion and structure from dynamic imagery using only line correspondences the traditional approach of corresponding microfeatures interesting points highlights corners high curvature points etc is reviewed and its shortcomings are discussed then a theory is presented that describes a closed form solution to the motion and structure determination problem from line correspondences in three views the theory is compared with previous ones that are based on nonlinear equations and iterative methods

on special zeros of p adic l functions of hilbert modular forms let e be a modular elliptic curve over a totally real number field f we prove the weak exceptional zero conjecture which links a higher derivative of the p adic l function attached to e to certain p adic periods attached to the corresponding hilbert modular form at the places above p where e has split multiplicative reduction under some mild restrictions on p and the conductor of e we deduce the exceptional zero conjecture in the strong form i e where the automorphic p adic periods are replaced by the mathcal l invariants of e defined in terms of tate periods from a special case proved earlier by mok crucial for our method is a new construction of the p adic l function of e in terms of local data

c ondensation conditional density propagation forvisual tracking the problem of tracking curves in dense visual clutter is challenging kalman filtering is inadequate because it is based on gaussian densities which being unimo dal cannot represent simultaneous alternative hypotheses the condensation algorithm uses factored sampling previously applied to the interpretation of static images in which the probability distribution of possible interpretations is represented by a randomly generated set condensation uses learned dynamical models together with visual observations to propagate the random set over time the result is highly robust tracking of agile motion notwithstanding the use of stochastic methods the algorithm runs in near real time

the 3d line motion matrix and alignment of line reconstructions we study the problem of aligning two 3d line reconstructions in projective affine metric or euclidean space we introduce the 6 6 3d line motion matrix that acts on plucker coordinates we characterize its algebraic properties and its relation to the usual 4 4 point motion matrix and propose various methods for estimating 3d motion from line correspondences based on cost functions defined in images or 3d space we assess the quality of the different estimation methods using simulated data and real images

projective reconstruction and invariants from multiple images this correspondence investigates projective reconstruction of geometric configurations seen in two or more perspective views and the computation of projective invariants of these configurations from their images a basic tool in this investigation is the fundamental matrix that describes the epipolar correspondence between image pairs it is proven that once the epipolar geometry is known the configurations of many geometric structures for instance sets of points or lines are determined up to a collineation of projective 3 space spl pscrsup 3 by their projection in two independent images this theorem is the key to a method for the computation of invariants of the geometry invariants of six points in spl pscrsup 3 and of four lines in spl pscrsup 3 are defined and discussed an example with real images shows that they are effective in distinguishing different geometrical configurations since the fundamental matrix is a basic tool in the computation of these invariants new methods of computing the fundamental matrix from seven point correspondences in two images or six point correspondences in three images are given

sequential updating of projective and affine structure from motion a structure from motion algorithm is described which recovers structure and camera position modulo a projective ambiguity camera calibration is not required and camera parameters such as focal length can be altered freely during motion the structure is updated sequentially over an image sequence in contrast to schemes which employ a batch process a specialisation of the algorithm to recover structure and camera position modulo an affine transformation is described together with a method to periodically update the affine coordinate frame to prevent drift over time we describe the constraint used to obtain this specialisation structure is recovered from image corners detected and matched automatically and reliably in real image sequences results are shown for reference objects and indoor environments and accuracy of recovered structure is fully evaluated and compared for a number of reconstruction schemes a specific application of the work is demonstrated affine structure is used to compute free space maps enabling navigation through unstructured environments and avoidance of obstacles the path planning involves only affine constructions

lines and points in three views and the trifocal tensor this paper discusses the basic role of the trifocal tensor in scene reconstruction from three views this 3 times 3 times 3 tensor plays a role in the analysis of scenes from three views analogous to the role played by the fundamental matrix in the two view case in particular the trifocal tensor may be computed by a linear algorithm from a set of 13 line correspondences in three views it is further shown in this paper that the trifocal tensor is essentially identical to a set of coefficients introduced by shashua to effect point transfer in the three view case this observation means that the 13 line algorithm may be extended to allow for the computation of the trifocal tensor given any mixture of sufficiently many line and point correspondences from the trifocal tensor the camera matrices of the images may be computed and the scene may be reconstructed for unrelated uncalibrated cameras this reconstruction will be unique up to projectivity thus projective reconstruction of a set of lines and points may be carried out linearly from three views

efficient agnostic learning of neural networks with bounded fan in we show that the class of two layer neural networks with bounded fan in is efficiently learnable in a realistic extension to the probably approximately correct pac learning model in this model a joint probability distribution is assumed to exist on the observations and the learner is required to approximate the neural network which minimizes the expected quadratic error as special cases the model allows learning real valued functions with bounded noise learning probabilistic concepts and learning the best approximation to a target function that cannot be well approximated by the neural network the networks we consider have real valued inputs and outputs an unlimited number of threshold hidden units with bounded fan in and a bound on the sum of the absolute values of the output weights the number of computation steps of the learning algorithm is bounded by a polynomial in 1 spl epsiv 1 spl delta n and b where spl epsiv is the desired accuracy spl delta is the probability that the algorithm fails n is the input dimension and b is the bound on both the absolute value of the target which may be a random variable and the sum of the absolute values of the output weights in obtaining the result we also extended some results on iterative approximation of functions in the closure of the convex hull of a function class and on the sample complexity of agnostic learning with the quadratic loss function

motion and structure from line correspondences closed form solution uniqueness and optimization this work discusses estimating motion and structure parameters from line correspondences of a rigid scene the authors present a closed form solution to motion and structure parameters from line correspondences through three monocular perspective views the algorithm makes use of redundancy in the data to improve the accuracy of the solutions the uniqueness of the solution is established and necessary and sufficient conditions for degenerate spatial line configurations are given optimization has been employed to further improve the accuracy of the estimates in the presence of noise simulations have shown that the errors of the optimized estimates are close to the theoretical lower error bound

on degeneracy of linear reconstruction from three views linear line complex and applications this paper investigates the linear degeneracies of projective structure estimation from line features across three views we show that the rank of the linear system of equations for recovering the trilinear tensor of three views reduces to 23 instead of 26 when the scene is a linear line complex llc a set of lines in space intersecting at a common line the llc situation is only linearly degenerate and one can obtain a unique solution when the admissibility constraints of the tensor are accounted for the line configuration described by an llc rather than being some obscure case is in fact quite typical it includes as a particular example the case of a camera moving down a hallway in an office environment or down an urban street furthermore an llc situation may occur as an artifact such as in direct estimation from spatio temporal derivatives of image brightness therefore an investigation into degeneracies and their remedy is important also in practice

shintani cocycles and the order of vanishing of p adic hecke l series at s 0 let chi be a hecke character of finite order of a totally real number field f by using hill s shintani cocycle we provide a cohomological construction of the p adic l series l p chi s associated to chi this is used to show that l p chi s has a trivial zero at s 0 of order at least equal to the number of places of f above p where the local component of chi is trivial

canonical representation and multi view geometry of cylinders this paper first introduces a canonical representation for cylinders the canonical representation introduced here is closely related to the plucker line representation in this paper we show that this representation is an appropriate one for computer vision applications in particular it allows us to easily develop a series of mathematical methods for pose estimation 3d reconstruction and motion estimation one of the major novelties in this paper is the introduction of the main equations dominating the three view geometry of cylinders we show the relationship between cylinders three view geometry and that of lines spetsakis and aloimonos 1990 weng et al 1993 and points shashua 1995 defined by the trilinear tensor hartley 1997 and propose a linear method which uses the correspondences between six cylinders across three views in order to recover the motion and structure cylindrical pipes and containers are the main components in the majority of chemical water treatment and power plants oil platforms refineries and many other industrial installations we have developed a professional software called cylicon which allows efficient as built reconstruction of such installations from a series of pre calibrated images markers are used for this pre calibration process the theoretical and practical results in this paper represent the first steps towards marker less calibration and reconstruction of such industrial sites here the experimental results take advantage of the two view and three view geometry of cylinders introduced in this paper to provide initial camera calibration results

application of lie algebras to visual servoing a novel approach to visual servoing is presented which takes advantage of the structure of the lie algebra of affine transformations the aim of this project is to use feedback from a visual sensor to guide a robot arm to a target position the target position is learned using the principle of teaching by showing in which the supervisor places the robot in the correct target position and the system captures the necessary information to be able to return to that position the sensor is placed in the end effector of the robot the camera in hand approach and thus provides direct feedback of the robot motion relative to the target scene via observed transformations of the scene these scene transformations are obtained by measuring the affine deformations of a target planar contour under the weak perspective assumption captured by use of an active contour or snake deformations of the snake are constrained using the lie groups of affine and projective transformations properties of the lie algebra of affine transformations are exploited to provide a novel method for integrating observed deformations of the target contour these can be compensated with appropriate robot motion using a non linear control structure the local differential representation of contour deformations is extended to allow accurate integration of an extended series of small perturbations this differs from existing approaches by virtue of the properties of the lie algebra representation which implicitly embeds knowledge of the three dimensional world within a two dimensional image based system these techniques have been implemented using a video camera to control a 5 dof robot arm experiments with this implementation are presented together with a discussion of the results

the geometry and matching of lines and curves over multiple views this paper describes the geometry of imaged curves in two and three views multi view relationships are developed for lines conics and non algebraic curves the new relationships focus on determining the plane of the curve in a projective reconstruction and in particular using the homography induced by this plane for transfer from one image to another it is shown that given the fundamental matrix between two views and images of the curve in each view then the plane of a conic may be determined up to a two fold ambiguity but local curvature of a curve uniquely determines the plane it is then shown that given the trifocal tensor between three views this plane defines a homography map which may be used to transfer a conic or the curvature from two views to a third simple expressions are developed for the plane and homography in each case a set of algorithms are then described for automatically matching individual line segments and curves between images the algorithms use both photometric information and the multiple view geometric relationships for image pairs the homography facilitates the computation of a neighbourhood cross correlation based matching score for putative line curve correspondences for image triplets cross correlation matching scores are used in conjunction with line curve transfer based on the trifocal geometry to disambiguate matches algorithms are developed for both short and wide baselines the algorithms are robust to deficiencies in the segment extraction and partial occlusion experimental results are given for image pairs and triplets for varying motions between views and for different scene types the methods are applicable to line curve matching in stereo and trinocular rigs and as a starting point for line curve matching through monocular image sequences

trinocular geometry revisited when do the visual rays associated with triplets of point correspondences converge that is intersect in a common point classical models of trinocular geometry based on the fundamental matrices and trifocal tensor associated with the corresponding cameras only provide partial answers to this fundamental question in large part because of underlying but seldom explicit general configuration assumptions this paper uses elementary tools from projective line geometry to provide necessary and sufficient geometric and analytical conditions for convergence in terms of transversals to triplets of visual rays without any such assumptions in turn this yields a novel and simple minimal parameterization of trinocular geometry for cameras with non collinear or collinear pinholes which can be used to construct a practical and efficient method for trinocular geometry parameter estimation we present numerical experiments using synthetic and real data

rank classification of linear line structures from images by trifocal tensor determinability the problem we address is given line correspondences over three views what is the condition of the line correspondences for the spatial relation of the three associated camera positions to be uniquely recoverable the observed set of lines in space is called critical if there are multiple projectively nonequivalent configurations of the camera positions that can picture the same image triplet of the lines we tackle the problem from the perspective of trifocal tensor a quantity that captures the relative pose of the cameras in relation to the captured views we show that the rank of a matrix that leads to the estimation of the tensor is reduced to 7 11 15 if the observed lines come from a line pencil a line bundle and a line field respectively which are line families belonging to linear line space and 12 19 23 if the lines come from a general linear ruled surface a general linear line congruence and a general linear line complex which are subclasses of linear line structures we show that the above line structures with the exception of linear line congruence and linear line complex ought to be critical line structures all of these structures are quite typical in reality and thus the findings are important to the validity and stability of practically all algorithms related to structure from motion and projective reconstruction using line correspondences

projective reconstruction of ellipses from multiple images this paper presents a new approach for reconstructing 3d ellipses including circles from a sequence of 2d images taken by uncalibrated cameras our strategy is to estimate an ellipse in 3d space by reconstructing n 5 3d points called representative points on it where the representative points are reconstructed by minimizing the distances from their projections to the measured 2d ellipses on different images i e 2d reprojection error this minimization problem is transformed into a sequence of minimization sub problems that can be readily solved by an algorithm which is guaranteed to converge to a local minimum of the 2d reprojection error our method can reconstruct multiple 3d ellipses simultaneously from multiple images and it readily handles images with missing and or partially occluded ellipses the proposed method is evaluated using both synthetic and real data

blended deformable models this paper develops a new class of parameterized models based on the linear interpolation of two parameterized shapes along their main axes using a blending function this blending function specifies the relative contribution of each component shape on the resulting blended shape the resulting blended shape can have aspects of each of the component shapes using a small number of additional parameters blending extends the coverage of shape primitives while also providing abstraction of shape in particular it offers the ability to construct shapes whose genus can change blended models are incorporated into a physics based shape estimation framework which uses dynamic deformable models finally we present experiments involving the extraction of complex shapes from range data including examples of dynamic genus change

3 d shape recovery using distributed aspect matching an approach to the recovery of 3 d volumetric primitives from a single 2 d image is presented the approach first takes a set of 3 d volumetric modeling primitives and generates a hierarchical aspect representation based on the projected surfaces of the primitives conditional probabilities capture the ambiguity of mappings between levels of the hierarchy from a region segmentation of the input image the authors present a formulation of the recovery problem based on the grouping of the regions into aspects no domain independent heuristics are used only the probabilities inherent in the aspect hierarchy are exploited once the aspects are recovered the aspect hierarchy is used to infer a set of volumetric primitives and their connectivity as a front end to an object recognition system the approach provides the indexing power of complex 3 d object centered primitives while exploiting the convenience of 2 d viewer centered aspect matching aspects are used to represent a finite vocabulary of 3 d parts from which objects can be constructed

gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks we analyze algorithms for approximating a function f x x mapping d to d using deep linear neural networks that is that learn a function h parameterized by matrices 1 l and defined by h x

rigidity checking of 3d point correspondences under perspective projection an algorithm is described which rapidly verifies the potential rigidity of three dimensional point correspondences from a pair of two dimensional views under perspective projection the output of the algorithm is a simple yes or no answer to the question could these corresponding points from two views be the projection of a rigid configuration potential applications include 3d object recognition from a single previous view and correspondence matching for stereo or motion over widely separated views the rigidity checking problem is different from the structure from motion problem because it is often the case that two views cannot provide an accurate structure from motion estimate due to ambiguity and ill conditioning whereas it is still possible to give an accurate yes no answer to the rigidity question rigidity checking verifies point correspondences using 3d recovery equations as a matching condition the proposed algorithm improves upon other methods that fall under this approach because it works with as few as six corresponding points under full perspective projection handles correspondences from widely separated views makes full use of the disparity of the correspondences and is integrated with a linear algorithm for 3d recovery due to kontsevich 1993 results are given for experiments with synthetic and real image data a complete implementation of this algorithm is being made publicly available

feature transfer and matching in disparate stereo views through the use of plane homographies many vision tasks rely upon the identification of sets of corresponding features among different images this paper presents a method that given some corresponding features in two stereo images matches them with features extracted from a second stereo pair captured from a distant viewpoint the proposed method is based on the assumption that the viewed scene contains two planar surfaces and exploits geometric constraints that are imposed by the existence of these planes to first transfer and then match image features between the two stereo pairs the resulting scheme handles point and line features in a unified manner and is capable of successfully matching features extracted from stereo pairs that are acquired from considerably different viewpoints experimental results are presented which demonstrate that the performance of the proposed method compares favorably to that of epipolar and tensor based approaches

recovery of nonrigid motion and structure the authors introduce a physically correct model of elastic nonrigid motion this model is based on the finite element method but decouples the degrees of freedom by breaking down object motion into rigid and nonrigid vibration or deformation modes the result is an accurate representation for both rigid and nonrigid motion that has greatly reduced dimensionality capturing the intuition that nonrigid motion is normally coherent and not chaotic because of the small number of parameters involved this representation is used to obtain accurate overstrained estimates of both rigid and nonrigid global motion it is also shown that these estimates can be integrated over time by use of an extended kalman filter resulting in stable and accurate estimates of both three dimensional shape and three dimensional velocity the formulation is then extended to include constrained nonrigid motion examples of tracking single nonrigid objects and multiple constrained objects are presented

continuous stereo self calibration by camera parameter tracking this paper presents a consistent framework for continuous stereo self calibration based on a practical analysis of the sensitivity of stereo reconstruction to camera calibration uncertainties we identify important parameters for self calibration we evaluate different geometric constraints for estimation and tracking of these parameters bundle adjustment with reduced structure representation relating corresponding points in image sequences the epipolar constraint between stereo image pairs and trilinear constraints between image triplets continuous recursive calibration refinement is obtained with a robust adapted iterated extended kalman filter to achieve high accuracy physically relevant geometric optimization criteria are formulated in a gauss helmert type model the self calibration framework is tested on an active stereo system experiments with synthetic data as well as on natural indoor and outdoor imagery indicate that the different constraints are complementing each other and thus a method combining two of the above constraints is proposed while reduced order bundle adjustment gives by far the most accurate results and might suffice on its own in some environments the epipolar constraint yields instantaneous calibration that is not affected by independently moving objects in the scene hence it expedites and stabilizes the calibration process

recovering multiple view geometry from mutual projections of multiple cameras in this paper we analyze the computation of epipolar geometry in some special cases where multiple cameras are projected each other in their images in such cases epipoles can be obtained directly from images as the projection of cameras as the result the epipolar geometry can be computed from less image correspondences with higher stability in this paper we analyze the number of corresponding points required for computing bifocal trifocal and quadrifocal tensors linearly in the case where cameras are projected mutually we next show a practical linear method for computing multifocal tensors by using the mutual projection of cameras the degenerate configurations of points and cameras is also studied and it is shown that some degenerate configurations in general cases are no longer degenerate under the mutual projection of cameras

building roadmaps of minima and transitions in visual models becoming trapped in suboptimal local minima is a perennial problem when optimizing visual models particularly in applications like monocular human body tracking where complicated parametric models are repeatedly fitted to ambiguous image measurements we show that trapping can be significantly reduced by building roadmaps of nearby minima linked by transition pathways paths leading over low mountain passes in the cost surface found by locating the transition state codimension 1 saddle point at the top of the pass and then sliding downhill to the next minimum we present two families of transition state finding algorithms based on local optimization in eigenvector tracking unconstrained newton minimization is modified to climb uphill towards a transition state while in hypersurface sweeping a moving hypersurface is swept through the space and moving local minima within it are tracked using a constrained newton method these widely applicable numerical methods which appear not to be known in vision and optimization generalize methods from computational chemistry where finding transition states is critical for predicting reaction parameters experiments on the challenging problem of estimating 3d human pose from monocular images show that our algorithms find nearby transition states and minima very efficiently but also underline the disturbingly large numbers of minima that can exist in this and similar model based vision problems

algebraic functions for recognition in the general case a trilinear relationship between three perspective views is shown to exist the trilinearity result is shown to be of much practical use in visual recognition by alignment yielding a direct reprojection method that cuts through the computations of camera transformation scene structure and epipolar geometry moreover the direct method is linear and sets a new lower theoretical bound on the minimal number of points that are required for a linear solution for the task of reprojection the proof of the central result may be of further interest as it demonstrates certain regularities across homographics of the plane and introduces new view invariants experiments on simulated and real image data were conducted including a comparative analysis with epipolar intersection and the linear combination methods with results indicating a greater degree of robustness in practice and a higher level of performance in reprojection tasks

motion of points and lines in the uncalibrated case in the present paper we address the problem of computing structure and motion given a set point and or line correspondences in a monocular image sequence when the camera is not calibrated

shape and motion from image streams under orthography a factorization method inferring scene geometry and camera motion from a stream of images is possible in principle but is an ill conditioned problem when the objects are distant with respect to their size we have developed a factorization method that can overcome this difficulty by recovering shape and motion under orthography without computing depth as an intermediate step an image stream can be represented by the 2fxp measurement matrix of the image coordinates of p points tracked through f frames we show that under orthographic projection this matrix is of rank 3 based on this observation the factorization method uses the singular value decomposition technique to factor the measurement matrix into two matrices which represent object shape and camera rotation respectively two of the three translation components are computed in a preprocessing stage the method can also handle and obtain a full solution from a partially filled in measurement matrix that may result from occlusions or tracking failures the method gives accurate results and does not introduce smoothing in either shape or motion we demonstrate this with a series of experiments on laboratory and outdoor image streams with and without occlusions

recovery of parametric models from range images the case for superquadrics with global deformations a method for recovery of compact volumetric models for shape representation of single part objects in computer vision is introduced the models are superquadrics with parametric deformations bending tapering and cavity deformation the input for the model recovery is three dimensional range points model recovery is formulated as a least squares minimization of a cost function for all range points belonging to a single part during an iterative gradient descent minimization process all model parameters are adjusted simultaneously recovery position orientation size and shape of the model such that most of the given range points lie close to the model s surface a specific solution among several acceptable solutions where are all minima in the parameter space can be reached by constraining the search to a part of the parameter space the many shallow local minima in the parameter space are avoided as a solution by using a stochastic technique during minimization results using real range data show that the recovered models are stable and that the recovery procedure is fast

a multi frame approach to visual motion perception one of the main issues in the area of motion estimation given the correspondences of some features in a sequence of images is sensitivity to error in the input the main way to attack the problem as with several other problems in science and engineering is redundancy in the data up to now all the algorithms developed either used two frames or depended on assumptions about the motion or the shape of the scene we present in this paper an algorithm based on multiple frames that employs only the rigidity assumption is simple and mathematically elegant and experimentally proves to be a major improvement over the two frame algorithms the algorithm does minimization of the squared error which we prove equivalent to an eigenvalue minimization problem one of the side effects of this mean square method is that the algorithm can have a very descriptive physical interpretation in terms of the loaded spring model

a theory of self calibration of a moving camera there is a close connection between the calibration of a single camera and the epipolar transformation obtained when the camera undergoes a displacement the epipolar transformation imposes two algebraic constraints on the camera calibration if two epipolar transformations arising from different camera displacements are available then the compatible camera calibrations are parameterized by an algebraic curve of genus four the curve can be represented either by a space curve of degree seven contained in the intersection of two cubic surfaces or by a curve of degree six in the dual of the image plane the curve in the dual plane has one singular point of order three and three singular points of order two if three epipolar transformations are available then two curves of degree six can be obtained in the dual plane such that one of the real intersections of the two yields the correct camera calibration the two curves have a common singular point of order three experimental results are given to demonstrate the feasibility of camera calibration based on the epipolar transformation the real intersections of the two dual curves are found by locating the zeros of a function defined on the interval 0 2 the intersection yielding the correct camera calibration is picked out by referring back to the three epipolar transformations

incremental model based estimation using geometric constraints we present a model based framework for incremental adaptive object shape estimation and tracking in monocular image sequences parametric structure and motion estimation methods usually assume a fixed class of shape representation splines deformable superquadrics etc that is initialized prior to tracking since the model shape coverage is fixed a priori the incremental recovery of structure is decoupled from tracking thereby limiting both processes in their scope and robustness in this work we describe a model based framework that supports the automatic detection and integration of low level geometric primitives lines incrementally such primitives are not explicitly captured in the initial model but are moving consistently with its image motion the consistency tests used to identify new structure are based on trinocular constraints between geometric primitives the method allows not only an increase in the model scope but also improves tracking accuracy by including the newly recovered features in its state estimation the formulation is a step toward automatic model building since it allows both weaker assumptions on the availability of a prior shape representation and on the number of features that would otherwise be necessary for entirely bottom up reconstruction we demonstrate the proposed approach on two separate image based tracking domains each involving complex 3d object structure and motion

ring theoretic properties of certain hecke algebras the purpose of this article is to provide a key ingredient of w2 by establishing that certain minimal hecke algebras considered there are complete intersections as is recorded in w2 a method going back to mazur m allows one to show that these algebras are gorenstein but for the complete intersection property a new approach is required the methods of this paper are related to those of chapter 3 of w2 the methods of section 3 of this paper are based on a previous approach of one of us a w we would like to thank henri darmon fred diamond and gerd faltings for carefully reading the first version of this article gerd faltings has also suggested a simplification of our argument as well as of the argument of chapter 3 of w2 and we would like to thank him for allowing us to reproduce these in the appendix to this paper r t would like to thank a w for his invitation to collaborate and for sharing his many insights into the questions considered r t would also like to thank princeton university universite de paris 7 and harvard university for their hospitality during this collaboration a w was supported by an nsf grant

structure from motion with line segments under relaxed endpoint constraints we present a novel structure from motion pipeline which estimates motion and wiry 3d structure from imaged line segments across multiple views although the position and orientation of line segments can be determined more accurately than point features the instability of their endpoints and the fact that lines are not constrained by epipolar geometry diverted most research focus away to point based methods in our approach we tackle the problem of instable endpoints by utilizing relaxed constraints on their positions both during matching and as well in the following bundle adjustment stage furthermore we gain efficiency in estimating trifocal image relations by decoupling rotation and translation to this end a novel linear solver for relative translation estimation given rotations from five line correspondences in three views is introduced extensive experiments on long image sequences show that our line based structure from motion pipeline advantageously complements point based methods giving more meaningful 3d representation for indoor scenarios

hessian schatten norm regularization for linear inverse problems we introduce a novel family of invariant convex and non quadratic functionals that we employ to derive regularized solutions of ill posed linear inverse imaging problems the proposed regularizers involve the schatten norms of the hessian matrix which are computed at every pixel of the image they can be viewed as second order extensions of the popular total variation tv semi norm since they satisfy the same invariance properties meanwhile by taking advantage of second order derivatives they avoid the staircase effect a common artifact of tv based reconstructions and perform well for a wide range of applications to solve the corresponding optimization problems we propose an algorithm that is based on a primal dual formulation a fundamental ingredient of this algorithm is the projection of matrices onto schatten norm balls of arbitrary radius this operation is performed efficiently based on a direct link we provide between vector projections onto norm balls and matrix projections onto schatten norm balls finally we demonstrate the effectiveness of the proposed methods through experimental results on several inverse imaging problems with real and simulated data