Patent Description:
This disclosure relates to systems and methods that facilitate positioning points and objects in a work space or at a worksite, such as for example at a construction site. When an interior of a building is being finished, connectors, anchors, and the like are attached to the floors, ceilings and other structures in the building, and cuts are made and holes drilled using power saws and drills. Many tasks are accomplished using special power tools at predetermined locations, such that the tools are operated at numerous, precisely-defined positions in the building. For example, nail guns, power saws, power-anchor tools, and the like are used to nail, cut, install fasteners, and perform other operations at predetermined points within the building. In a building, a large number of electrical, plumbing, and HVAC components are properly sited and installed, usually with power tools. Additionally, finishing a building interior also uses a number of different tools that are not powered, yet are to be operated at precisely defined positions, such as for example reinforcement bar scanners. Positioning both power tools and non-power tools is to be accomplished quickly and with some precision with respect to the surrounding walls, ceilings, and floors as they are roughed in. Typically, it has used a significant amount of labor to layout various construction points at a construction site. Teams of workers have been used to measure and mark predetermined locations. It will be appreciated that this process has been subject to errors, resulting from measurement mistakes and from accumulated errors. Further, the cost of this layout process and the time needed to perform the layout process have both been significant.

Various location determining systems, including systems that incorporate one or more robotic total stations, have been used for building layout. The total station in such a system, positioned at a fixed, known location, directs a beam of laser light at a retro-reflective target. As the target moves, robotics in the total station cause the beam of light to track the target. Since the dimensional coordinates of the total station are known, the dimensional coordinates of the retro-reflective target can be determined. Based on the measured position of the retro-reflective target, and the desired position of some building feature, such as a drilled hole and/ or a fastener, the operator can move the reflector to the desired position, and mark the position.

Although position determination systems, such as ranging radio systems and robotic total station systems, can facilitate and speed the layout process, the layout process has continued to be lengthy, tedious, and expensive. Previous applications have been filed addressing systems and/or methods to reduce construction expense and/or labor. For example, <CIT>, provides systems and methods for positioning a tool in a work space.

<CIT> discloses a method to geo-localize a camera acquiring an image of a city environment, based on a 3D model of the city and the analysis of features extracted from the acquired image, where the features include lines and points. A reprojection of the features is used to determine the camera location.

<CIT> discloses a method of accurately estimating a pose of a camera within a scene using a three dimensional model of the scene. The method begins by generating an initial estimate of the camera pose. Next, a set of relevant features of the three-dimensional model based on the estimate of the pose is selected. A virtual projection of this set of relevant features as seen from the estimated pose is then created. The virtual projection of the set of relevant features is matched to features of the received image and matching errors between the features of the image and the features of the projection are measured. The estimate of the pose is then updated to reduce the matching errors. <CIT> discloses a method for determining a camera location capturing images in a city environment. The method begins with receiving geographic orientation data associated with the image. A subsequent step is performed of accessing a bin of model features from a database based on the geographic orientation data, wherein the model features are from a pre-generated three-dimensional model of the environment and are binned in the database based on geographic orientation of the model features. Finally features are extracted from the image, and are compared to model features in the bin of model features to produce a set of corresponding features, which are then used to determine a position and an orientation of the camera.

In some embodiments, a system, not forming part of the claimed invention, for tool positioning, as-built documentation, and/or personnel monitoring in construction site using a camera network is described. Camera units are placed at multiple, unknown locations in a construction site to visually cover a working volume. A camera unit is self-positioned by comparing an image to a model of the construction site. Camera units, in combination with a main processing computer, can detect and calculate positions of objects in the working volume.

A method for using a camera system to determine a location of a camera unit in relation to a three-dimensional model according to claim <NUM> is disclosed.

A camera system according to claim <NUM> is also disclosed.

A memory device according to claim <NUM> is also disclosed.

Further areas of applicability of the present disclosure will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description and specific examples, while indicating various embodiments, are intended for purposes of illustration only and are not intended to necessarily limit the scope of the disclosure.

The present disclosure is described in conjunction with the appended figures.

In the appended figures, similar components and/or features may have the same reference label.

The ensuing description provides preferred exemplary embodiment(s) only, and is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the ensuing description of the preferred exemplary embodiment(s) will provide those skilled in the art with an enabling description for implementing a preferred exemplary embodiment.

While an interior of a building is being finished, connectors, anchors, and the like are attached to the floors, ceilings, and other structures in the building. Further, cuts are made and holes drilled using power saws and drills. Tools, such as special power tools, are used at predetermined locations, such that the tools are operated at numerous, precisely-defined positions in the building. For example, nail guns, power saws, power anchor tools, and the like are used to nail, cut, install fasteners, and/or perform other operations at predetermined points within the building. In some buildings, a large number of electrical, plumbing, and HVAC components are sited and installed, usually with power tools. Additionally, finishing a building interior also uses a number of different tools that are not powered, which are also operated at precisely-defined positions, such as for example reinforcement bar scanners. Positioning both power tools and non-power tools quickly and with precision (e.g., with respect to surrounding walls, ceilings, and floors) can save time and reduce construction costs. In some embodiments, systems and/or methods are described to speed up, reduce manual labor, and/or reduce expense of construction.

Referring first to <FIG>, a simplified drawing of an embodiment of a camera system is shown. The camera system comprises camera units <NUM> and a main processing computer. The camera system comprises two or more camera units <NUM>. In some embodiments, the camera system comprises two or more measurement sticks <NUM>, and/or two or more main processing computers.

A camera unit <NUM> comprises a processor unit, a communication device, and/or one or more imaging devices <NUM> (e.g., two and/or three imaging devices <NUM>). The imaging devices <NUM> have a position and/or orientation that is known relative to other imaging device(s) <NUM> and/or camera unit(s) <NUM>. The imaging devices <NUM> acquire image sequences. The processor unit in the camera unit <NUM> processes the image sequences. The communication device transmits and receives data with the main processing computer.

In <FIG>, a simplified drawing of an embodiment of a configuration of camera units <NUM> at a worksite is shown. An operator attaches camera units <NUM> to a column, wall, etc. The operator can add and/or subtract camera units <NUM> depending on a size of the worksite and/or a position of the camera unit <NUM>.

<FIG> depicts a simplified perspective drawing of an embodiment of camera units <NUM> placed in a worksite. The operator attaches the camera units <NUM> to a column <NUM>, wall <NUM>, ceiling <NUM>, floor <NUM>, truss <NUM>, etc. Camera units <NUM>, which are modular, allow for quick setup.

Camera units <NUM> can be used to determine a three-dimensional position of an object at the worksite. In some embodiments, a collinearity condition, sometimes referred to as collinearity equations, is used in calculating the three-dimensional line equation. <FIG> depicts a simplified drawing of the collinearity condition. The collinearity condition represents a geometric relationship wherein a point (A) in an object space, an image of the point (a) on an image plane <NUM>, and a perspective center (L) of the image plane <NUM> are on a straight line in three-dimensional space.

The collinearity condition can be expressed by the following equations:
<MAT>
<MAT>.

In the above equations, xa and ya are coordinates of the image of the point (a) (sometimes referred to as photo coordinates); xo and yo are coordinates of the principal point (the principal point is the point on the image plane <NUM> onto which the perspective center (L) is projected);f is a focal length of a camera (e.g., of an imaging device <NUM>); XL, YL, and ZL are object space coordinates of the perspective center L; XA, YA, and ZA are object space coordinates of point A; and the m's are functions of the three rotation angles (ω, φ, and κ) such that:
<MAT>
<MAT>
<MAT>
<MAT>
<MAT>
<MAT>
<MAT>
<MAT>
<MAT>.

Since the collinearity equations are nonlinear, the linearized forms of the equations are used to iteratively solve many photogrammetric solutions. The collinearity equations can be linearized by using Taylor's theorem. In this manner, the collinearity equations can be written as follows (simplified):
<MAT>
<MAT>.

Space Intersection. Space intersection is the photogrammetric process that estimates 3D coordinates of a point in the object space from photo coordinates observations in a stereo pair. <FIG> illustrates the concept of space intersection. Known values are the exterior orientation parameters of two photographs (L<NUM>, L<NUM>), photo coordinates of a point (a<NUM>, a<NUM>) in each photograph, and initial value for the point in object space (A). In some embodiments, the initial value for the point in object space (A) is determined by calculating a first vector starting at the perspective center of the first camera and to photo coordinates of a point on an image plane of the first camera; calculate a second vector for the second camera (e.g., starting at a perspective center of the second camera to a point on an image plane of the second camera); calculating an intersection of the first vector and the second vector, and if an intersection does not exist, finding a point near (e.g., closest point between) the first vector and the second vector.

The following equations shows the linearized form of the collinearity equations for space intersection. Since the exterior orientation parameters of each photograph are known, the object coordinate terms (dXA, dYA, dZA) are estimated iteratively in the linearized form of the collinearity condition equation. <MAT>
<MAT>.

The following equation shows the matrix form of the previous equations. Subscripts C0 and C1 represent cameras <NUM> and <NUM> (or photographs <NUM> and <NUM>), respectively. The solution of the equation can be estimated iteratively using Least-square solution.

The following table shows the iterative space intersection algorithm.

Space intersection with multiple cameras. A bundle adjustment technique can be used to solve for the space intersection solution in a multiple camera case. <FIG> illustrates the concept of space intersection for the multiple camera case. Known values are the exterior orientation parameters of the cameras or photographs (L<NUM>, L<NUM>, L<NUM>, and L<NUM>), the photo coordinates of a point (a<NUM>, a<NUM>, a<NUM>, and a<NUM>), and the initial value for the point in object space (A) is a center of the object space.

The following equation shows a matrix form of space intersection equation for the multiple camera case. In this equation, subscripts C<NUM>, C<NUM>, C<NUM> and C<NUM> represent cameras <NUM>, <NUM>, <NUM>, and <NUM>, respectively. Solution of equation can be estimated iteratively by using least-squares solution. The sizes of the design matrix and the observation matrix are 2n×<NUM> and 2n×<NUM>, respectively; where n is the number of cameras (or photographs).

Referring next to <FIG>, a flowchart of an embodiment of a process <NUM> for determining a position and/or orientation of a camera at a worksite is illustrated. Process <NUM> begins in step <NUM> with retrieving a model of the worksite. The model is a three-dimensional model. In some embodiments, the model is a Building Information Modeling (BIM) file.

In step <NUM>, an image of the worksite is retrieved. In some embodiments, the image is stored as an image file, such as a Joint Photographic Experts Group (JPEG), a Tagged Image File Format (TIFF), a Graphics Interchange Format (GIF), and/or a Windows bitmap (BMP) file. The image is acquired by the camera (e.g., using an image sensor in a camera unit <NUM>) at the worksite.

In step <NUM>, coarse input for a camera position is received. The coarse input for the camera is in relation to the model. In some embodiments, the coarse input for the camera position is received by a user marking an estimated camera position in a computer application (e.g., on the model). The model is projected to a two-dimensional projection based on the coarse input, step <NUM>.

Features in the image are compared to features in the two-dimensional projection, <NUM>. To compare features in the image to features in the two-dimensional projection, features in the image are identified, which correspond to features in the two-dimensional projection. A feature is a line. For example, an outline of a wall (e.g., four lines) in the two-dimensional projection is overlaid on the image. An outline of the wall in the image is detected. And the position and/or orientation of the camera is calculated so that the outline of the wall in the two-dimensional projection overlaps the outline of the wall in the image. In some embodiments, to determine an overlap, a least-squares fit is used.

In optional step <NUM>, an iterative process is used. In step <NUM>, a determination is made to refine the calculation of the position and/or orientation of the camera. The calculated position and/or orientation of the camera of step <NUM> is fed back into step <NUM> as coarse input and/or into step <NUM> for comparing features in the image to features of the two-dimensional projection. In some embodiments, different features are used (e.g., a second wall, a third wall, a ceiling, and/or a floor).

In an example approach, an estimation of a camera position and orientation is estimated in relation to a building information model (BIM). Since some BIMs contain a large amount of data, the example approach is semi-automated. In the example approach, the BIM is divided into a plurality of BIM files. In the example approach, only the BIM file(s), of the plurality of BIM files, that corresponds to a locality of the camera is retrieved. The BIM file is retrieved based on user input for the coarse position of the camera.

After retrieval of the relevant model, the user provides the coarse position of the camera. Algorithms, starting from this coarse position, estimate position and orientation of the camera using both the BIM and the image. The coarsely aligned BIM provides 3D information of the worksite (e.g., dimensions of elements and relative placements of each element at the worksite). To match a BIM to an image, the following equation is used: <MAT>
where, <MAT> <MAT> <MAT> and <MAT>
such that, <MAT>.

B corresponds to known two-dimensional points of the image. C corresponds to known internal parameters (e.g., focal length of a lens of the camera). D corresponds to camera orientation and position (unknown). E corresponds to three-dimensional known points (e.g., from the BIM file).

Initial estimation. An initial projection matrix can be determined using either points or lines. The initial estimation of the projection matrix starts with selecting point correspondences from the BIM and the image. In some embodiments, a system selects lines in both the BIM and the image (e.g., selecting lines from an outline of a wall, ceiling, or floor). In some embodiments, as few as five to nine points are used. In some embodiments, selected points in the image do not have to exactly match BIM points, wherein others' approaches algorithms highly depend on points a user selects. Although a poor initial point selection does not have a significantly negative effect on the accuracy of the estimated position and orientation, it may be more consuming for processing compared to a more accurate initial point selection. Applicant has found that line-based projection is generally more stable than point-based projection. Moreover, line-based estimation has a better performance than point-based estimation in terms of noise resistance from a computational point of view. Although estimation based on line correspondences is considered as a dual of one based on point correspondences, or is regarded projectively equivalent, we find from a numerical point of view, an estimation based on line correspondences is more robust.

<FIG> depicts an embodiment of a point projection. C1 is a position of a first camera. The first camera has a first image plan <NUM>-<NUM>. C2 is a position of a second camera. The second camera has a second image plan <NUM>-<NUM>. Points p and q are imaged in the first image plane <NUM>-<NUM> and in the second image plane <NUM>-<NUM>.

<FIG> depicts an embodiment of a line projection. C1 is a position of a first camera. The first camera has a first image plane <NUM>-<NUM>. C2 is a position of a second camera. The second camera has a second image plane <NUM>-<NUM>. Lines l and n are imaged in the first image plane <NUM>-<NUM> and in the second image plane <NUM>-<NUM>.

An initial alignment comprises two steps: (<NUM>) a projection and (<NUM>) a homography. The projection maps the 3D model to the image. The homography uses the lines from the 3D model (e.g., lines outlining a wall). In some embodiments, the lines are selected by a user previously. In some embodiments, the system selects lines automatically (e.g., the system is configured to identify a wall, a floor, and/or a ceiling in the 3D model). In some embodiments, the homography uses a perspective from a coarse position of the camera entered by the user.

<FIG> depicts an embodiment of an image <NUM> with a projection <NUM> of an outline of a wall from a 3D model. <FIG> demonstrates the projection and the homography steps of the initial estimation. Although the initial estimation is close to aligning the projection <NUM> with a wall <NUM> in the image, the projection <NUM> can be aligned more closely to the wall <NUM>.

The projection <NUM> is fitted to edges in the image <NUM>. For example, the image <NUM> is converted from color to gray tone. A derivative filter is then applied to the gray-tone image to produce a derivative image. In some embodiments, the derivative filter is a Laplacian filter. In the derivative image, edges are easier to identify. Edges are areas where pixel values change rapidly (sometimes referred to as "corners"). A plane (e.g., projection <NUM>) is fitted to intensity of edges of the image <NUM>. After a parametric plane for each pixel intensity based on neighbor edges is defined, gradient of each pixel of the image edges can be determined from a plane normal vector. In some embodiments, the parametric plane is a virtual plane (not physical) fitted to grayscale values in space (x,y, I(x,y)) where I(x,y) are intensity values. Plane fitting uses gradient of intensities; a normal of a virtual plane defines an edge direction.

<FIG> shows an intensity-based fitting plane algorithm successfully aligning a BIM to an edge. Extrinsic camera parameters (rotation and position) are updated. <FIG> shows normal directions for fitted virtual planes np. A first normal direction np1, a second normal direction np2, a third normal direction np3, and a fourth normal direction np4 are shown. <FIG> shows an embodiment of the projection <NUM> better aligned to the image <NUM>. Extrinsic camera parameters (rotation and position) are updated in each iteration and eventually final camera parameters are generated that align the projection <NUM> and the image <NUM>, <FIG>. Final camera parameters generate a homography that maps the initial estimation from <FIG>.

Extrinsic camera parameters (rotation and position) are calculated based on constraints that fulfill characteristics of rotation and translation matrices. In some embodiments, an accuracy of estimated camera orientation and position for a relatively large room (23x16x10 feet<NUM>) is as low as <NUM> inches. A similar procedure can be done for other walls, ceiling, and/or floor. Each wall provides a different set of camera parameters. Since walls other than the wall <NUM> used to estimate extrinsic parameters were not involved, camera parameters might be biased. To reduce biasing, camera parameters from walls, the ceiling, and/or the floor are used. Using camera parameters from more than one surface (e.g., wall) will reduce errors and render more accurate results.

Referring next to <FIG>, an embodiment for a single camera matrix is shown. The single camera matrix reduces re-projection error using bundle adjustment. Bundle adjustment refines 3D coordinates describing a scene geometry, parameters of the relative motion, and/or optical characteristics of the camera. However, a typical bundle adjustment compensates reprojection errors using several cameras. Yet bias caused by considering walls separately with a modified bundle adjustment(BA) algorithm can reduce errors. Thus only one camera with parameters for different walls is used (e.g., first parameters for a first wall and second parameters for a second wall). <FIG> displays how bundle adjustment reduces reprojection error. Reprojection error <NUM> is calculated by: <MAT> The objective function is: <MAT> The variable, wij is <NUM> if point j is visible, <NUM> otherwise.

Referring next to <FIG>, an exemplary environment with which embodiments may be implemented is shown with a computer system <NUM> that can be used by a designer <NUM> to design, for example, electronic designs. The computer system <NUM> can include a computer <NUM>, keyboard <NUM>, a network router <NUM>, a printer <NUM>, and a monitor <NUM>. The monitor <NUM>, processor <NUM> and keyboard <NUM> are part of a computer system, which can be a laptop computer, desktop computer, handheld computer, mainframe computer, etc. The monitor <NUM> can be a CRT, flat screen, etc..

A designer <NUM> can input commands into the computer <NUM> using various input devices, such as a mouse, keyboard <NUM>, track ball, touch screen, etc. If the computer system <NUM> comprises a mainframe, a designer <NUM> can access the computer <NUM> using, for example, a terminal or terminal interface. Additionally, the computer <NUM> may be connected to a printer <NUM> and a server <NUM> using a network router <NUM>, which may connect to the Internet <NUM> or a WAN.

The server <NUM> may, for example, be used to store additional software programs and data. In one embodiment, software implementing the systems and methods described herein can be stored on a storage medium in the server <NUM>. Thus, the software can be run from the storage medium in the server <NUM>. In another embodiment, software implementing the systems and methods described herein can be stored on a storage medium in the computer <NUM>. Thus, the software can be run from the storage medium in the computer <NUM>. Therefore, in this embodiment, the software can be used whether or not computer <NUM> is connected to network router <NUM>. Printer <NUM> may be connected directly to computer <NUM>, in which case, the computer system <NUM> can print whether or not it is connected to network router <NUM>.

With reference to <FIG>, an embodiment of a special-purpose computer system <NUM> is shown. The above methods may be implemented by computer-program products that direct a computer system to perform the actions of the above-described methods and components. Each such computer-program product may comprise sets of instructions (codes) embodied on a computer-readable medium that directs the processor of a computer system to perform corresponding actions. The instructions may be configured to run in sequential order, or in parallel (such as under different processing threads), or in a combination thereof. After loading the computer-program products on a general purpose computer system, it is transformed into the special-purpose computer system <NUM>.

Special-purpose computer system <NUM> comprises a computer <NUM>, a monitor <NUM> coupled to computer <NUM>, one or more additional user output devices <NUM> (optional) coupled to computer <NUM>, one or more user input devices <NUM> (e.g., keyboard, mouse, track ball, touch screen) coupled to computer <NUM>, an optional communications interface <NUM> coupled to computer <NUM>, a computer-program product <NUM> stored in a tangible computer-readable memory in computer <NUM>. Computer-program product <NUM> directs system <NUM> to perform the above-described methods. Computer <NUM> may include one or more processors <NUM> that communicate with a number of peripheral devices via a bus subsystem <NUM>. These peripheral devices may include user output device(s) <NUM>, user input device(s) <NUM>, communications interface <NUM>, and a storage subsystem, such as random access memory (RAM) <NUM> and nonvolatile storage drive <NUM> (e.g., disk drive, optical drive, solid state drive), which are forms of tangible computer-readable memory.

Computer-program product <NUM> may be stored in non-volatile storage drive <NUM> or another computer-readable medium accessible to computer <NUM> and loaded into memory <NUM>. Each processor <NUM> may comprise a microprocessor, such as a microprocessor from Intel® or Advanced Micro Devices, Inc. ®, or the like. To support computer-program product <NUM>, the computer <NUM> runs an operating system that handles the communications of product <NUM> with the above-noted components, as well as the communications between the above-noted components in support of the computer-program product <NUM>. Exemplary operating systems include Windows® or the like from Microsoft Corporation, Solaris® from Sun Microsystems, LINUX, UNIX, and the like.

User input devices <NUM> include all possible types of devices and mechanisms to input information to computer <NUM>. These may include a keyboard, a keypad, a mouse, a scanner, a digital drawing pad, a touch screen incorporated into the display, audio input devices such as voice recognition systems, microphones, and other types of input devices. In various embodiments, user input devices <NUM> are typically embodied as a computer mouse, a trackball, a track pad, a joystick, wireless remote, a drawing tablet, a voice command system. User input devices <NUM> typically allow a user to select objects, icons, text and the like that appear on the monitor <NUM> via a command such as a click of a button or the like. User output devices <NUM> include all possible types of devices and mechanisms to output information from computer <NUM>. These may include a display (e.g., monitor <NUM>), printers, non-visual displays such as audio output devices, etc..

Communications interface <NUM> provides an interface to other communication networks and devices and may serve as an interface to receive data from and transmit data to other systems, WANs and/or the Internet <NUM>. Embodiments of communications interface <NUM> typically include an Ethernet card, a modem (telephone, satellite, cable, ISDN), a (asynchronous) digital subscriber line (DSL) unit, a FireWire® interface, a USB® interface, a wireless network adapter, and the like. For example, communications interface <NUM> may be coupled to a computer network, to a FireWire® bus, or the like. In other embodiments, communications interface <NUM> may be physically integrated on the motherboard of computer <NUM>, and/or may be a software program, or the like.

RAM <NUM> and non-volatile storage drive <NUM> are examples of tangible computer-readable media configured to store data such as computer-program product embodiments of the present invention, including executable computer code, human-readable code, or the like. Other types of tangible computer-readable media include floppy disks, removable hard disks, optical storage media such as CD-ROMs, DVDs, bar codes, semiconductor memories such as flash memories, read-only-memories (ROMs), battery-backed volatile memories, networked storage devices, and the like. RAM <NUM> and non-volatile storage drive <NUM> may be configured to store the basic programming and data constructs that provide the functionality of various embodiments of the present invention, as described above.

Software instruction sets that provide the functionality of the present invention may be stored in RAM <NUM> and non-volatile storage drive <NUM>. These instruction sets or code may be executed by the processor(s) <NUM>. RAM <NUM> and non-volatile storage drive <NUM> may also provide a repository to store data and data structures used in accordance with the present invention. RAM <NUM> and non-volatile storage drive <NUM> may include a number of memories including a main random access memory (RAM) to store of instructions and data during program execution and a read-only memory (ROM) in which fixed instructions are stored. RAM <NUM> and non-volatile storage drive <NUM> may include a file storage subsystem providing persistent (nonvolatile) storage of program and/or data files. RAM <NUM> and non-volatile storage drive <NUM> may also include removable storage systems, such as removable flash memory.

Bus subsystem <NUM> provides a mechanism to allow the various components and subsystems of computer <NUM> communicate with each other as intended. Although bus subsystem <NUM> is shown schematically as a single bus, alternative embodiments of the bus subsystem <NUM> may utilize multiple busses or communication paths within the computer <NUM>.

Implementation of the techniques, blocks, steps and means described above may be done in various ways. For example, these techniques, blocks, steps and means may be implemented in hardware, software, or a combination thereof. For a hardware implementation, the processing units may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described above, and/or a combination thereof.

Also, it is noted that the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. A process is terminated when its operations are completed, but could have additional steps not included in the figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function.

Furthermore, embodiments may be implemented by hardware, software, scripting languages, firmware, middleware, microcode, hardware description languages, and/or any combination thereof. When implemented in software, firmware, middleware, scripting language, and/or microcode, the program code or code segments to perform the necessary tasks may be stored in a machine readable medium such as a storage medium. A code segment or machineexecutable instruction may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a script, a class, or any combination of instructions, data structures, and/or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, and/or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc..

For a firmware and/or software implementation, the methodologies may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. Any machine-readable medium tangibly embodying instructions may be used in implementing the methodologies described herein. For example, software codes may be stored in a memory. Memory may be implemented within the processor or external to the processor. As used herein the term "memory" refers to any type of long term, short term, volatile, nonvolatile, or other storage medium and is not to be limited to any particular type of memory or number of memories, or type of media upon which memory is stored.

Moreover, as disclosed herein, the term "storage medium" may represent one or more memories for storing data, including read only memory (ROM), random access memory (RAM), magnetic RAM, core memory, magnetic disk storage mediums, optical storage mediums, flash memory devices and/or other machine readable mediums for storing information. The term "machine-readable medium" includes, but is not limited to portable or fixed storage devices, optical storage devices, wireless channels, and/or various other storage mediums capable of storing that include or carry instruction(s) and/or data.

While the principles of the disclosure have been described above in connection with specific apparatuses and methods, it is to be clearly understood that this description is made only by way of example and not as limitation on the scope of the disclosure.

The above description of exemplary embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form described, and many modifications and variations are possible in light of the teaching above.

The embodiments were chosen and described in order to explain the principles of the invention and practical applications to thereby enable others skilled in the art to best utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated.

Also, it is noted that the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. A process is terminated when its operations are completed, but could have additional steps not included in the figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc..

Claim 1:
A computer-implemented method (<NUM>) for using a camera system to determine a location of a camera unit in relation to a three-dimensional model, the method comprising:
retrieving (<NUM>) a model of a worksite, wherein the model is a three-dimensional model;
retrieving (<NUM>) an image of the worksite, wherein:
the image is acquired using a camera at the worksite;
the worksite has a first surface and a second surface, wherein the first surface is a wall, floor or ceiling of the worksite, and wherein the second surface is a wall, floor or ceiling of the worksite;
the image comprises at least a portion of the first surface and at least a portion of the second surface;
receiving (<NUM>) coarse input for a position and/or an orientation of the camera in relation to the model;
projecting (<NUM>) a first feature of the model onto the image;
comparing (<NUM>) the first surface in the image to the first feature of the model projected onto the image, wherein the first feature comprises lines, wherein each line corresponds to an edge or outline of the first surface;
projecting (<NUM>) a second feature of the model onto the image;
comparing (<NUM>) the second surface in the image to the second feature of the model projected onto the image, wherein the second feature comprises lines, wherein each line corresponds to an edge or outline of the second surface;
calculating a first set of parameters of the camera based on aligning the first feature to the first surface in the image;
calculating a second set of parameters of the camera based on aligning the second feature with the second surface in the image, wherein calculating the second set of parameters is performed independently from calculating the first set of parameters; and
calculating (<NUM>) the position and/or the orientation of the camera in relation to the model based on the first set of parameters and the second set of parameters.