Patent Publication Number: US-11024050-B2

Title: System and method of scanning an environment

Description:
BACKGROUND 
     The present application is directed to a system that optically scans an environment, such as a building, and in particular to a portable system that generates two-dimensional floorplans of the scanned environment and links information regarding points of interest of the scanned environment to coordinates within an image of the scanned environment. 
     The automated creation of digital two-dimensional floorplans for existing structures is desirable as it allows the size and shape of the environment to be used in many processes. For example, a floorplan may be desirable to allow construction drawings to be prepared during a renovation. Such floorplans may find other uses such as in documenting a building for a fire department or to document a crime scene. 
     Existing measurement systems typically use a scanning device that determines coordinates of surfaces in the environment by both emitting a light and capturing a reflection to determine a distance or by triangulation using cameras. These scanning device are mounted to a movable structure, such as a cart, and moved through the building to generate a digital representation of the building. These systems tend to be more complex and require specialized personnel to perform the scan. Further, the scanning equipment including the movable structure may be bulky, which could further delay the scanning process in time sensitive situations, such as a crime or accident scene investigation. 
     Additionally, in some situations such as a crime scene, accident scene, or other emergency situation, it may be helpful to link information about points of interest in the environment to coordinates within an image generated by the scanning system. Such a system could quickly capture important details and link these details to a specific point in the image, thereby allowing personnel to more quickly gather an organize data in time-sensitive situations. 
     Accordingly, while existing scanners are suitable for their intended purposes, what is needed is a system for having certain features of embodiments of the present invention. 
     BRIEF DESCRIPTION 
     An embodiment of a system for linking information of a point of interest to a position within an image of the location may include a portable device structured to determine a position of the point of interest in the image when the portable device is present within the location depicted by the image; an accessory operably coupled to the portable device and comprising a tool structured to provide information related to the point of interest; a processor operably coupled to the portable device and configured to create a data structure linking the information with the position of the point of interest; and a storage structured to store the data structure. 
     An embodiment of a method for linking information of a point of interest to a position within a location may include acquiring an image depicting the location; determining, with a portable device while the portable device is present within the location depicted by the image, a position of the point of interest in the image; acquiring information related to the point of interest with a tool, the tool being provided in an accessory operably coupled to the portable device; creating a data structure linking the information with the position of the point of interest; and storing the data structure in a storage. 
     At least an embodiment of a non-transitory computer-readable medium may store therein computer-executable instructions that, when executed by a computer, cause the computer to perform acquiring an image depicting a location; determining, with a portable device while the portable device is present within the location depicted by the image, a position of a point of interest in the image; acquiring information related to the point of interest with a tool, the tool being provided in an accessory operably coupled to the portable device; creating a data structure linking the information with the position of the point of interest; and storing the data structure in a storage. 
     These and other advantages and features will become more apparent from the following description taken in conjunction with the drawings. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The subject matter, which is regarded as the invention, is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other features, and advantages of the invention are apparent from the following detailed description taken in conjunction with the accompanying drawings in which: 
         FIGS. 1-3  are perspective views of a scanning and mapping system in accordance with an embodiment; 
         FIG. 4  is a first end view of the system of  FIG. 1 ; 
         FIG. 5  is a side sectional view of the system of  FIG. 1 ; 
         FIG. 6  is a side sectional view of the system of a scanning and mapping system in accordance with another embodiment; 
         FIG. 7  is a first end view of the system of  FIG. 6 ; 
         FIG. 8  is a top sectional view of the system of  FIG. 6 ; 
         FIG. 9  is an enlarged view of a portion of the second end of  FIG. 7 ; 
         FIG. 10  is a block diagram of the system of  FIG. 1  and  FIG. 6 ; 
         FIG. 11-13  are schematic illustrations of the operation of system of  FIG. 9  in accordance with an embodiment; 
         FIG. 14  is a flow diagram of a method of generating a two-dimensional map of an environment; 
         FIGS. 15-16  are plan views of stages of a two-dimensional map generated with the method of  FIG. 14  in accordance with an embodiment; 
         FIG. 17-18  are schematic views of the operation of the system of  FIG. 9  in accordance with an embodiment; 
         FIG. 19  is a flow diagram of a method of generating a two-dimensional map using the system of  FIG. 9  in accordance with an embodiment. 
         FIG. 20  is a perspective view of a system in accordance with an embodiment. 
         FIG. 21  is an exploded perspective view of a system in accordance with an embodiment. 
         FIG. 22  is a front view of a system in accordance with an embodiment. 
         FIG. 23  is a cross section view of a system in accordance with an embodiment. 
         FIG. 24  is a plan view of a two-dimensional map and points of interest in accordance with at least an embodiment. 
         FIG. 25  shows a table relating coordinates of a point of interest with information about the point of interest in accordance with at least an embodiment. 
         FIG. 26  is a flowchart showing a method of linking coordinates of a point of interest to information related to the point of interest according to at least an embodiment. 
         FIG. 27  is a flowchart showing a method of linking coordinates of a point of interest to information related to the point of interest according to at least an embodiment. 
         FIG. 28  is a schematic illustration showing the connections of the accessory, controller, and mobile device according to at least an embodiment. 
     
    
    
     The detailed description explains embodiments of the invention, together with advantages and features, by way of example with reference to the drawings. 
     DETAILED DESCRIPTION 
     The present invention relates to a device that includes a system having a 2D scanner that works cooperatively with an inertial measurement unit to generate an annotated two-dimensional map of an environment. 
     Referring now to  FIGS. 1-5 , an embodiment of a system  30  may have a housing  32  that includes a body  34  and a handle  36 . In an embodiment, the handle  36  may include an actuator  38  that allows the operator to interact with the system  30 . In the exemplary embodiment, the body  34  includes a generally rectangular center portion  35  with a slot  40  formed in an end  42 . The slot  40  is at least partially defined by a pair walls  44  that are angled towards a second end  48 . As will be discussed in more detail herein, a portion of a two-dimensional scanner  50  is arranged between the walls  44 . The walls  44  are angled to allow the scanner  50  to operate by emitting a light over a large angular area without interference from the walls  44 . As will be discussed in more detail herein, the end  42  may further include a three-dimensional camera or RGBD camera  60 . 
     Extending from the center portion  35  is a mobile device holder  41 . The mobile device holder  41  is configured to securely couple a mobile device  43  to the housing  32 . The holder  41  may include one or more fastening elements, such as a magnetic or mechanical latching element for example, that couples the mobile device  43  to the housing  32 . In an embodiment, the mobile device  43  is coupled to communicate with a controller  68  ( FIG. 10 ). The communication between the controller  68  and the mobile device  43  may be via any suitable communications medium, such as wired, wireless or optical communication mediums for example. 
     In the illustrated embodiment, the holder  41  is pivotally coupled to the housing  32 , such that it may be selectively rotated into a closed position within a recess  46 . In an embodiment, the recess  46  is sized and shaped to receive the holder  41  with the mobile device  43  disposed therein. 
     In the exemplary embodiment, the second end  48  includes a plurality of exhaust vent openings  56 . In an embodiment, shown in  FIGS. 6-9 , the exhaust vent openings  56  are fluidly coupled to intake vent openings  58  arranged on a bottom surface  62  of center portion  35 . The intake vent openings  58  allow external air to enter a conduit  64  having an opposite opening  66  ( FIG. 6 ) in fluid communication with the hollow interior  67  of the body  34 . In an embodiment, the opening  66  is arranged adjacent to a controller  68  which has one or more processors that is operable to perform the methods described herein. In an embodiment, the external air flows from the opening  66  over or around the controller  68  and out the exhaust vent openings  56 . 
     The controller  68  is coupled to a wall  70  of body  34 . In an embodiment, the wall  70  is coupled to or integral with the handle  36 . The controller  68  is electrically coupled to the 2D scanner  50 , the 3D camera  60 , a power source  72 , an inertial measurement unit (IMU)  74 , a laser line projector  76 , and a haptic feedback device  77 . 
     Referring now to  FIG. 10  with continuing reference to  FIGS. 1-9 , elements are shown of the system  30  with the mobile device  43  installed or coupled to the housing  32 . Controller  68  is a suitable electronic device capable of accepting data and instructions, executing the instructions to process the data, and presenting the results. The controller  68  includes one or more processing elements  78 . The processors may be microprocessors, field programmable gate arrays (FPGAs), digital signal processors (DSPs), and generally any device capable of performing computing functions. The one or more processors  78  have access to memory  80  for storing information. 
     Controller  68  is capable of converting the analog voltage or current level provided by 2D scanner  50 , camera  60  and IMU  74  into a digital signal to determine a distance from the system  30  to an object in the environment. In an embodiment, the camera  60  is a 3D or RGBD type camera. Controller  68  uses the digital signals that act as input to various processes for controlling the system  30 . The digital signals represent one or more system  30  data including but not limited to distance to an object, images of the environment, acceleration, pitch orientation, yaw orientation and roll orientation. As will be discussed in more detail, the digital signals may be from components internal to the housing  32  or from sensors and devices located in the mobile device  43 . 
     In general, when the mobile device  43  is not installed, controller  68  accepts data from 2D scanner  50  and IMU  74  and is given certain instructions for the purpose of generating a two-dimensional map of a scanned environment. Controller  68  provides operating signals to the 2D scanner  50 , the camera  60 , laser line projector  76  and haptic feedback device  77 . Controller  68  also accepts data from IMU  74 , indicating, for example, whether the operator is operating in the system in the desired orientation. The controller  68  compares the operational parameters to predetermined variances (e.g. yaw, pitch or roll thresholds) and if the predetermined variance is exceeded, generates a signal that activates the haptic feedback device  77 . The data received by the controller  68  may be displayed on a user interface coupled to controller  68 . The user interface may be one or more LEDs (light-emitting diodes)  82 , an LCD (liquid-crystal diode) display, a CRT (cathode ray tube) display, or the like. A keypad may also be coupled to the user interface for providing data input to controller  68 . In one embodiment, the user interface is arranged or executed on the mobile device  43 . 
     The controller  68  may also be coupled to external computer networks such as a local area network (LAN) and the Internet. A LAN interconnects one or more remote computers, which are configured to communicate with controller  68  using a well-known computer communications protocol such as TCP/IP (Transmission Control Protocol/Internet({circumflex over ( )}) Protocol), RS-232, ModBus, and the like. Additional systems  30  may also be connected to LAN with the controllers  68  in each of these systems  30  being configured to send and receive data to and from remote computers and other systems  30 . The LAN may be connected to the Internet. This connection allows controller  68  to communicate with one or more remote computers connected to the Internet. 
     The processors  78  are coupled to memory  80 . The memory  80  may include random access memory (RAM) device  84 , a non-volatile memory (NVM) device  86 , a read-only memory (ROM) device  88 . In addition, the processors  78  may be connected to one or more input/output (I/O) controllers  90  and a communications circuit  92 . In an embodiment, the communications circuit  92  provides an interface that allows wireless or wired communication with one or more external devices or networks, such as the LAN discussed above. 
     Controller  68  includes operation control methods embodied in application code shown or described with reference to  FIGS. 11-14  and  FIG. 19 . These methods are embodied in computer instructions written to be executed by processors  78 , typically in the form of software. The software can be encoded in any language, including, but not limited to, assembly language, VHDL (Verilog Hardware Description Language), VHSIC HDL (Very High Speed IC Hardware Description Language), Fortran (formula translation), C, C++, C #, Objective-C, Visual C++, Java, ALGOL (algorithmic language), BASIC (beginners all-purpose symbolic instruction code), visual BASIC, ActiveX, HTML (HyperText Markup Language), Python, Ruby and any combination or derivative of at least one of the foregoing. 
     Coupled to the controller  68  is the 2D scanner  50 . The 2D scanner  50  measures 2D coordinates in a plane. In the exemplary embodiment, the scanning is performed by steering light within a plane to illuminate object points in the environment. The 2D scanner  50  collects the reflected (scattered) light from the object points to determine 2D coordinates of the object points in the 2D plane. In an embodiment, the 2D scanner  50  scans a spot of light over an angle while at the same time measuring an angle value and corresponding distance value to each of the illuminated object points. 
     Examples of 2D scanners  50  include, but are not limited to Model LMS100 scanners manufactured by Sick, Inc of Minneapolis, Minn. and scanner Models URG-04LX-UG01 and UTM-30LX manufactured by Hokuyo Automatic Co., Ltd of Osaka, Japan. The scanners in the Sick LMS100 family measure angles over a 270 degree range and over distances up to 20 meters. The Hoyuko model URG-04LX-UG01 is a low-cost 2D scanner that measures angles over a 240 degree range and distances up to 4 meters. The Hoyuko model UTM-30LX is a 2D scanner that measures angles over a 270 degree range and to distances up to 30 meters. It should be appreciated that the above 2D scanners are exemplary and other types of 2D scanners are also available. 
     In an embodiment, the 2D scanner  50  is oriented so as to scan a beam of light over a range of angles in a generally horizontal plane (relative to the floor of the environment being scanned). At instants in time the 2D scanner  50  returns an angle reading and a corresponding distance reading to provide 2D coordinates of object points in the horizontal plane. In completing one scan over the full range of angles, the 2D scanner returns a collection of paired angle and distance readings. As the system  30  is moved from place to place, the 2D scanner  50  continues to return 2D coordinate values. These 2D coordinate values are used to locate the position of the system  30  thereby enabling the generation of a two-dimensional map or floorplan of the environment. 
     Also coupled to the controller  86  is the IMU  74 . The IMU  74  is a position/orientation sensor that may include accelerometers  94  (inclinometers), gyroscopes  96 , a magnetometers or compass  98 , and altimeters. In the exemplary embodiment, the IMU  74  includes multiple accelerometers  94  and gyroscopes  96 . The compass  98  indicates a heading based on changes in magnetic field direction relative to the earth&#39;s magnetic north. The IMU  74  may further have an altimeter that indicates altitude (height). An example of a widely used altimeter is a pressure sensor. By combining readings from a combination of position/orientation sensors with a fusion algorithm that may include a Kalman filter, relatively accurate position and orientation measurements can be obtained using relatively low-cost sensor devices. In the exemplary embodiment, the IMU  74  determines the pose or orientation of the system  30  about three-axis to allow a determination of a yaw, roll and pitch parameter. 
     In the embodiment shown in  FIGS. 6-9 , the system  30  further includes a camera  60  that is a 3D or RGB-D camera. As used herein, the term 3D camera refers to a device that produces a two-dimensional image that includes distances to a point in the environment from the location of system  30 . The 3D camera  30  may be a range camera or a stereo camera. In an embodiment, the 3D camera  30  includes an RGB-D sensor that combines color information with a per-pixel depth information. In an embodiment, the 3D camera  30  may include an infrared laser projector  31  ( FIG. 9 ), a left infrared camera  33 , a right infrared camera  39 , and a color camera  37 . In an embodiment, the 3D camera  60  is a RealSense™ camera model R200 manufactured by Intel Corporation. 
     In an embodiment, when the mobile device  43  is coupled to the housing  32 , the mobile device  43  becomes an integral part of the system  30 . In an embodiment, the mobile device  43  is a cellular phone, a tablet computer or a personal digital assistant (PDA). The mobile device  43  may be coupled for communication via a wired connection, such as ports  100 ,  102 . The port  100  is coupled for communication to the processor  78 , such as via I/O controller  90  for example. The ports  100 ,  102  may be any suitable port, such as but not limited to USB, USB-A, USB-B, USB-C, IEEE 1394 (Firewire), or Lightning™ connectors. 
     The mobile device  43  is a suitable electronic device capable of accepting data and instructions, executing the instructions to process the data, and presenting the results. The mobile device  43  includes one or more processing elements  104 . The processors may be microprocessors, field programmable gate arrays (FPGAs), digital signal processors (DSPs), and generally any device capable of performing computing functions. The one or more processors  104  have access to memory  106  for storing information. 
     The mobile device  43  is capable of converting the analog voltage or current level provided by sensors  108  and processor  78 . Mobile device  43  uses the digital signals that act as input to various processes for controlling the system  30 . The digital signals represent one or more system  30  data including but not limited to distance to an object, images of the environment, acceleration, pitch orientation, yaw orientation, roll orientation, global position, ambient light levels, and altitude for example. 
     In general, mobile device  43  accepts data from sensors  108  and is given certain instructions for the purpose of generating or assisting the processor  78  in the generation of a two-dimensional map or three-dimensional map of a scanned environment. Mobile device  43  provides operating signals to the processor  78 , the sensors  108  and a display  110 . Mobile device  43  also accepts data from sensors  108 , indicating, for example, to track the position of the mobile device  43  in the environment or measure coordinates of points on surfaces in the environment. The mobile device  43  compares the operational parameters to predetermined variances (e.g. yaw, pitch or roll thresholds) and if the predetermined variance is exceeded, may generate a signal. The data received by the mobile device  43  may be displayed on display  110 . In an embodiment, the display  110  is a touch screen device that allows the operator to input data or control the operation of the system  30 . 
     The controller  68  may also be coupled to external networks such as a local area network (LAN), a cellular network and the Internet. A LAN interconnects one or more remote computers, which are configured to communicate with controller  68  using a well-known computer communications protocol such as TCP/IP (Transmission Control Protocol/Internet({circumflex over ( )}) Protocol), RS-232, ModBus, and the like. Additional systems  30  may also be connected to LAN with the controllers  68  in each of these systems  30  being configured to send and receive data to and from remote computers and other systems  30 . The LAN may be connected to the Internet. This connection allows controller  68  to communicate with one or more remote computers connected to the Internet. 
     The processors  104  are coupled to memory  106 . The memory  106  may include random access memory (RAM) device, a non-volatile memory (NVM) device, and a read-only memory (ROM) device. In addition, the processors  104  may be connected to one or more input/output (I/O) controllers  112  and a communications circuit  114 . In an embodiment, the communications circuit  114  provides an interface that allows wireless or wired communication with one or more external devices or networks, such as the LAN or the cellular network discussed above. 
     Controller  68  includes operation control methods embodied in application code shown or described with reference to  FIGS. 11-4  and  FIG. 19 . These methods are embodied in computer instructions written to be executed by processors  78 ,  104 , typically in the form of software. The software can be encoded in any language, including, but not limited to, assembly language, VHDL (Verilog Hardware Description Language), VHSIC HDL (Very High Speed IC Hardware Description Language), Fortran (formula translation), C, C++, C #, Objective-C, Visual C++, Java, ALGOL (algorithmic language), BASIC (beginners all-purpose symbolic instruction code), visual BASIC, ActiveX, HTML (HyperText Markup Language), Python, Ruby and any combination or derivative of at least one of the foregoing. 
     Also coupled to the processor  104  are the sensors  108 . The sensors  108  may include but are not limited to: a microphone  116 ; a speaker  118 ; a front or rear facing camera  120 ; accelerometers  122  (inclinometers), gyroscopes  124 , a magnetometers or compass  126 ; a global positioning satellite (GPS) module  128 ; a barometer  130 ; a proximity sensor  132 ; and an ambient light sensor  134 . By combining readings from a combination of sensors  108  with a fusion algorithm that may include a Kalman filter, relatively accurate position and orientation measurements can be obtained. 
     It should be appreciated that the sensors  60 ,  74  integrated into the scanner  30  may have different characteristics than the sensors  108  of mobile device  43 . For example, the resolution of the cameras  60 ,  120  may be different, or the accelerometers  94 ,  122  may have different dynamic ranges, frequency response, sensitivity (mV/g) or temperature parameters (sensitivity or range). Similarly, the gyroscopes  96 ,  124  or compass/magnetometer may have different characteristics. It is anticipated that in some embodiments, one or more sensors  108  in the mobile device  43  may be of higher accuracy than the corresponding sensors  74  in the system  30 . As described in more detail herein, in some embodiments the processor  78  determines the characteristics of each of the sensors  108  and compares them with the corresponding sensors in the system  30  when the mobile device. The processor  78  then selects which sensors  74 ,  108  are used during operation. In some embodiments, the mobile device  43  may have additional sensors (e.g. microphone  116 , camera  120 ) that may be used to enhance operation compared to operation of the system  30  without the mobile device  43 . In still further embodiments, the system  30  does not include the IMU  74  and the processor  78  uses the sensors  108  for tracking the position and orientation/pose of the system  30 . In still further embodiments, the addition of the mobile device  43  allows the system  30  to utilize the camera  120  to perform three-dimensional (3D) measurements either directly (using an RGB-D camera) or using photogrammetry techniques to generate 3D maps. In an embodiment, the processor  78  uses the communications circuit (e.g. a cellular 4G internet connection) to transmit and receive data from remote computers or devices. 
     In the exemplary embodiment, the system  30  is a handheld portable device that is sized and weighted to be carried by a single person during operation. Therefore, the plane  136  ( FIG. 18 ) in which the 2D scanner  50  projects a light beam may not be horizontal relative to the floor or may continuously change as the computer moves during the scanning process. Thus, the signals generated by the accelerometers  94 , gyroscopes  96  and compass  98  (or the corresponding sensors  108 ) may be used to determine the pose (yaw, roll, tilt) of the system  30  and determine the orientation of the plane  51 . 
     In an embodiment, it may be desired to maintain the pose of the system  30  (and thus the plane  136 ) within predetermined thresholds relative to the yaw, roll and pitch orientations of the system  30 . In an embodiment, a haptic feedback device  77  is disposed within the housing  32 , such as in the handle  36 . The haptic feedback device  77  is a device that creates a force, vibration or motion that is felt or heard by the operator. The haptic feedback device  77  may be, but is not limited to: an eccentric rotating mass vibration motor or a linear resonant actuator for example. The haptic feedback device is used to alert the operator that the orientation of the light beam from 2D scanner  50  is equal to or beyond a predetermined threshold. In operation, when the IMU  74  measures an angle (yaw, roll, pitch or a combination thereof), the controller  68  transmits a signal to a motor controller  138  that activates a vibration motor  140 . Since the vibration originates in the handle  36 , the operator will be notified of the deviation in the orientation of the system  30 . The vibration continues until the system  30  is oriented within the predetermined threshold or the operator releases the actuator  38 . In an embodiment, it is desired for the plane  136  to be within 10-15 degrees of horizontal (relative to the ground) about the yaw, roll and pitch axes. 
     In an embodiment, the 2D scanner  50  makes measurements as the system  30  is moved about an environment, such from a first position  142  to a second registration position  144  as shown in  FIG. 11 . In an embodiment, 2D scan data is collected and processed as the system  30  passes through a plurality of 2D measuring positions  146 . At each measuring position  146 , the 2D scanner  50  collects 2D coordinate data over an effective FOV  148 . Using methods described in more detail below, the controller  68  uses 2D scan data from the plurality of 2D scans at positions  146  to determine a position and orientation of the system  30  as it is moved about the environment. In an embodiment, the common coordinate system is represented by 2D Cartesian coordinates x, y and by an angle of rotation  9  relative to the x or y axis. In an embodiment, the x and y axes lie in the plane of the 2D scanner and may be further based on a direction of a “front” of the 2D scanner  50 . 
       FIG. 12  shows the 2D system  30  collecting 2D scan data at selected positions  108  over an effective FOV  110 . At different positions  146 , the 2D scanner  50  captures a portion of the object  150  marked A, B, C, D, and E.  FIG. 121  shows 2D scanner  50  moving in time relative to a fixed frame of reference of the object  150 . 
       FIG. 13  includes the same information as  FIG. 12  but shows it from the frame of reference of the system  30  rather than the frame of reference of the object  150 .  FIG. 13  illustrates that in the system  30  frame of reference, the position of features on the object change over time. Therefore, the distance traveled by the system  30  can be determined from the 2D scan data sent from the 2D scanner  50  to the controller  68 . 
     As the 2D scanner  50  takes successive 2D readings and performs best-fit calculations, the controller  68  keeps track of the translation and rotation of the 2D scanner  50 , which is the same as the translation and rotation of the system  30 . In this way, the controller  68  is able to accurately determine the change in the values of x, y,  9  as the system  30  moves from the first position  142  to the second position  144 . 
     In an embodiment, the controller  68  is configured to determine a first translation value, a second translation value, along with first and second rotation values (yaw, roll, pitch) that, when applied to a combination of the first 2D scan data and second 2D scan data, results in transformed first 2D data that closely matches transformed second 2D data according to an objective mathematical criterion. In general, the translation and rotation may be applied to the first scan data, the second scan data, or to a combination of the two. For example, a translation applied to the first data set is equivalent to a negative of the translation applied to the second data set in the sense that both actions produce the same match in the transformed data sets. An example of an “objective mathematical criterion” is that of minimizing the sum of squared residual errors for those portions of the scan data determined to overlap. Another type of objective mathematical criterion may involve a matching of multiple features identified on the object. For example, such features might be the edge transitions  152 ,  154 , and  156  shown in  FIG. 11 . The mathematical criterion may involve processing of the raw data provided by the 2D scanner  50  to the controller  68 , or it may involve a first intermediate level of processing in which features are represented as a collection of line segments using methods that are known in the art, for example, methods based on the Iterative Closest Point (ICP). Such a method based on ICP is described in Censi, A., “An ICP variant using a point-to-line metric,” IEEE International Conference on Robotics and Automation (ICRA) 2008, which is incorporated by reference herein. 
     In an embodiment, assuming that the plane  136  of the light beam from 2D scanner  50  remains horizontal relative to the ground plane, the first translation value is dx, the second translation value is dy, and the first rotation value dθ. If the first scan data is collected with the 2D scanner  50  having translational and rotational coordinates (in a reference coordinate system) of (x 1 , y 1 , θ 1 ), then when the second 2D scan data is collected at a second location the coordinates are given by (x 2 , y 2 , θ 2 )=(x 1 +dx, y 1 +dy, θ 1 +dθ). In an embodiment, the controller  68  is further configured to determine a third translation value (for example, dz) and a second and third rotation values (for example, pitch and roll). The third translation value, second rotation value, and third rotation value may be determined based at least in part on readings from the IMU  74 . 
     The 2D scanner  50  collects 2D scan data starting at the first position  142  and more 2D scan data at the second position  144 . In some cases, these scans may suffice to determine the position and orientation of the system  30  at the second position  144  relative to the first position  142 . In other cases, the two sets of 2D scan data are not sufficient to enable the controller  68  to accurately determine the first translation value, the second translation value, and the first rotation value. This problem may be avoided by collecting 2D scan data at intermediate scan positions  146 . In an embodiment, the 2D scan data is collected and processed at regular intervals, for example, once per second. In this way, features in the environment are identified in successive 2D scans at positions  146 . In an embodiment, when more than two 2D scans are obtained, the controller  68  may use the information from all the successive 2D scans in determining the translation and rotation values in moving from the first position  142  to the second position  144 . In another embodiment, only the first and last scans in the final calculation, simply using the intermediate 2D scans to ensure proper correspondence of matching features. In most cases, accuracy of matching is improved by incorporating information from multiple successive 2D scans. 
     It should be appreciated that as the system  30  is moved beyond the second position  144 , a two-dimensional image or map of the environment being scanned may be generated. 
     Referring now to  FIG. 14 , a method  160  is shown for generating a two-dimensional map with annotations. The method  160  starts in block  162  where the facility or area is scanned to acquire scan data  170 , such as that shown in  FIG. 15 . The scanning is performed by carrying the system  30  through the area to be scanned. The system  30  measures distances from the system  30  to an object, such as a wall for example, and also a pose of the system  30  in an embodiment the user interacts with the system  30  via actuator  38 . In the illustrated embodiments, the mobile device  43  provides a user interface that allows the operator to initiate the functions and control methods described herein. Using the registration process desired herein, the two dimensional locations of the measured points on the scanned objects (e.g. walls, doors, windows, cubicles, file cabinets etc.) may be determined. It is noted that the initial scan data may include artifacts, such as data that extends through a window  172  or an open door  174  for example. Therefore, the scan data  170  may include additional information that is not desired in a 2D map or layout of the scanned area. 
     The method  120  then proceeds to block  164  where a 2D map  176  is generated of the scanned area as shown in  FIG. 16 . The generated 2D map  176  represents a scan of the area, such as in the form of a floor plan without the artifacts of the initial scan data. It should be appreciated that the 2D map  176  may be utilized directly by an architect, interior designer or construction contractor as it represents a dimensionally accurate representation of the scanned area. In the embodiment of  FIG. 14 , the method  160  then proceeds to block  166  where optional user-defined annotations are made to the 2D maps  176  to define an annotated 2D map that includes information, such as dimensions of features, the location of doors, the relative positions of objects (e.g. liquid oxygen tanks, entrances/exits or egresses or other notable features such as but not limited to the location of automated sprinkler systems, knox or key boxes, or fire department connection points (“FDC”). In some geographic regions, public safety services such as fire departments may keep records of building or facility layouts for use in case of an emergency as an aid to the public safety personnel in responding to an event. It should be appreciated that these annotations may be advantageous in alerting the public safety personnel to potential issues they may encounter when entering the facility, and also allow them to quickly locate egress locations. 
     Once the annotations of the 2D annotated map are completed, the method  160  then proceeds to block  168  where the 2D map is stored in memory, such as nonvolatile memory  86  for example. The 2D map may also be stored in a network accessible storage device or server so that it may be accessed by the desired personnel. 
     Referring now to  FIG. 17  and  FIG. 18  an embodiment is illustrated with the mobile device  43  coupled to the system  20 . As described herein, the 2D scanner  50  emits a beam of light in the plane  136 . The 2D scanner  50  has a field of view (FOV) that extends over an angle that is less than 360 degrees. In the exemplary embodiment, the FOV of the 2D scanner is about 270 degrees. In this embodiment, the mobile device  43  is coupled to the housing  32  adjacent the end where the 2D scanner  50  is arranged. The mobile device  43  includes a forward facing camera  120 . The camera  120  is positioned adjacent a top side of the mobile device and has a predetermined field of view  180 . In the illustrated embodiment, the holder  41  couples the mobile device  43  on an obtuse angle  182 . This arrangement allows the mobile device  43  to acquire images of the floor and the area directly in front of the system  20  (e.g. the direction the operator is moving the system  20 ). 
     In embodiments where the camera  120  is a RGB-D type camera, three-dimensional coordinates of surfaces in the environment may be directly determined in a mobile device coordinate frame of reference. In an embodiment, the holder  41  allows for the mounting of the mobile device  43  in a stable position (e.g. no relative movement) relative to the 2D scanner  50 . When the mobile device  43  is coupled to the housing  32 , the processor  78  performs a calibration of the mobile device  43  allowing for a fusion of the data from sensors  108  with the sensors of system  20 . As a result, the coordinates of the 2D scanner may be transformed into the mobile device coordinate frame of reference or the 3D coordinates acquired by camera  120  may be transformed into the 2D scanner coordinate frame of reference. 
     In an embodiment, the mobile device is calibrated to the 2D scanner  50  by assuming the position of the mobile device based on the geometry and position of the holder  41  relative to 2D scanner  50 . In this embodiment, it is assumed that the holder that causes the mobile device to be positioned in the same manner. It should be appreciated that this type of calibration may not have a desired level of accuracy due to manufacturing tolerance variations and variations in the positioning of the mobile device  43  in the holder  41 . In another embodiment, a calibration is performed each time a different mobile device  43  is used. In this embodiment, the user is guided (such as via the user interface  110 ) to direct the system  30  to scan a specific object, such as a door, that can be readily identified in the laser readings of the system  30  and in the camera-sensor  120  using an object recognition method. 
     Referring now to  FIG. 19 , a method  200  is provided for generating a 2D map of an environment. The method  200  begins in block  202  where the operator couples the mobile device  43  to the holder  41 . In an embodiment, the coupling includes forming a communication connection between the processor  78  and the processor  104 . This communication connection allows the processors  78 ,  104  to exchange data, including sensor data, therebetween. The method  200  then proceeds to block  204  where information regarding the sensors  108  is transmitted to the processor  78 . The information transmitted includes the type of sensors (e.g. accelerometer) and performance characteristics or parameters of the sensor (e.g. dynamic range, frequency response, sensitivity (mV/g) temperature sensitivity, or temperature range). 
     The method  200  then proceeds to block  206  where the processor  78  compares the sensors  108  with the corresponding sensors in the system  20 . In an embodiment, this comparison includes comparing performance characteristics or parameters and determining which sensor would provide a desired accuracy of the scan. It should be appreciated that this comparison is performed on a sensor by sensor basis. In some embodiments, the data used for tracking and pose may be a combination of the sensors from the mobile device  43  and the system  20 . For example, the accelerometer  122  may be used in combination with the gyroscope  96  and compass  98  for determining tracking and pose. 
     In an embodiment, once the sensors are selected the method  200  a calibration step is performed in block  208 . As discussed herein, the calibration step allows the transforming of data between the mobile device coordinate frame of reference and the 2D scanner coordinate frame of reference. 
     The method  200  then proceeds to block  210  where the scan is performed by moving the system  20  (with mobile device  43  attached) about the environment. As the scan is being performed (e.g. the 2D scanner is emitting and receiving reflected light and determining distances), the method  200  is transforming data in block  212  into a common frame of reference, such as the 2D scanner frame of reference for example, so that coordinates of the points of surfaces in the environment may be determined. As the scan is being performed, the position and pose of the system  20  is determined on a periodic, aperiodic or continuous basis as described herein. 
     Once the scan is completed, the method  200  proceeds to block  214  where the 2D map is generated of the scanned area. It should be appreciated that in embodiments where the camera  120  is a 3D camera or RGB-D type camera, a 3D map of the environment may be generated. 
     It should be appreciated that while embodiments herein describe the performance of operational control methods by the processor  78 , this is for exemplary purposes and the claims should not be so limited. In other embodiments, the operation control methods may be performed by the processor  104  or a combination of the processor  78  and the processor  104 . 
       FIGS. 20-23  illustrate another exemplary embodiment of a system  30  that may be used to link information about a point of interest to a position of the point of interest within an image of a location where the point of interest is located, such as 2D map  176 . The image may alternately be a 3D map as described above. 
       FIG. 20  shows a system  30  having a body  34 . As described above, system  30  may also include scanner  50 , which can determine a distance from the system  30  to another position, such as a wall for example. In other words, a position of the system  30  within the image, or relative to other objected within the image may be determined. 
     System  30  may further include an accessory  300  operably coupled to body  34 . Accessory  300  may be detachable from body  34 . Accessory  300  may include tools  302 ,  304  structured to provide information related to the point of interest. While  FIGS. 20-23  show tools  302 ,  304 , it will be understood that accessory  300  is not limited to embodiments having two tools. For example, accessory  300  may include only a single tool, or it may include three or more tools. 
     Further, it should be noted that while  FIGS. 20-23  illustrate the system  30  with the mobile device holder  41  in a closed position, this is for exemplary purposes and the claims should not be so limited. In other embodiments, the system  30  may be operated with the holder  41  in the open or closed position. 
     As seen in  FIG. 10 , the system  30  may include a processor  78  as part of controller  68 , which may be operably coupled to the tool  302  and/or tool  304 .  FIG. 28  shows a schematic drawing illustrating one possibility of connecting the tools  302 ,  304  to the processor  78 . As described above, tools  302 ,  304  may be connected to accessory electronic circuit  308 , which may in turn connect to accessory external terminal  312 . Accessory external terminal  312  may connect to body external terminal  316 , which is connected to port  101  of the controller  68 . Similarly, mobile device  43  connects to controller  68  via port  100  as described above, through which communication between mobile device  43  and accessory  300  can be achieved. The accessory external connector  312  and body external terminal  316  may be any suitable port, such as but not limited to USB, USB-A, USB-B, USB-C, IEEE 1394 (Firewire), or Lightning™ connectors. Additionally, the connection between body external terminal  316  and controller  68  may be any suitable connection, such as but not limited to USB, USB-A, USB-B, USB-C, IEEE 1394 (Firewire), or Lightning™ connectors. It will be understood that  FIG. 27  is showing a connection of accessory  300  and controller  68  schematically, and is not intended to indicate any kind of required specific arrangement of the components. 
     However, it will also be understood that in some embodiments, coupling between processor  78  and tool  302  or tool  304  and the processor may not be necessary. For example, a user may directly input data observed while using tool  302  or tool  304 . Processor  78  may be configured to create a data structure linking the information with the position of the point of interest.  FIG. 25  discussed below illustrates at least one possible embodiment of the data structure. System  30  further may further include a storage such as memory  80  in which the image and the data structure may be stored. 
       FIG. 21  shows an exploded view of system  30  to further illustrate the components of accessory  300 . For example,  FIG. 300  may include a housing  306 . Housing  306  is structured to accommodate tools  302 ,  304  an accessory electronic circuit  308 . Accessory electronic circuit  308  may be a printed-circuit board (PCB) and is structured to couple with tools  302 ,  304  within the housing  306 . Accessory  300  may further include accessory external terminal  312  provided on an exterior of housing  306  and structured to electronically connect the electronic circuit to processor  78 . In at least an embodiment, accessory external terminal  312  may be a 10-pin spring connector, but it will be understood that other types of connectors could be substituted. Accessory external terminal  312  is structured to couple with a body external terminal  316  provided on body  34 . Accessory  300  may also include connectors  310  provided on an exterior of housing  306 . Connectors  310  are structured to couple with complimentary connectors  314  provided on body  34 . Connectors  310  may couple mechanically with connectors  314 , or connectors  310 ,  314  may be complimentary magnets structured to couple via magnetic force. 
       FIGS. 22-23  further show how accessory  300  connects to body  34 .  FIG. 23  is a cross section of  FIG. 22  along line A-A. 
     System  30  may further include a display structured to display the image, the point of interest, and the information related to the point of interest. As noted above, the display may be an LCD (liquid-crystal diode) display, a CRT (cathode ray tube) display, or the like. Additionally, system  30  may include buttons, switches, a keypad or other input for providing data input to controller  68 . 
     System  30  may further include a portable computer terminal, such as mobile device  43 . Mobile device  43  may be operably connected to controller  68  and accessory  300 . Additionally, mobile device  43  may be configured to control tools  302 ,  304  and to receive and analyze data from accessory  300 . Mobile device  43  may include software programs or applications to allow interface between the user and the accessory  300 .  FIG. 10  shows that mobile device  43  may include a processor  104 , memory  106 , and display  110 . Processor  104  of mobile device  43  may be used in place of processor  78  to create a data structure linking the information with the position of the point of interest. Memory  106  of mobile device may be used in place of storage  80  for storing the image and data structure. Display  110  of mobile device  43  may be used in place of displays on the portable device to display the image, the point of interest, and the information related to the point of interest. 
     While  FIGS. 20-23  show two tools  302 ,  304 , the description below will only refer to tool  302  for simplicity. Tool  302  may be a device such as a flashlight. If the flashlight reveals information of note to the user, the user can use move scanner  50  to a position of the information of note, determine the position of the scanner, and create a data structure linking the position of the scanner (and the position of the information) to the information. 
     One possible example can be explained using  FIG. 24 , which shows 2D map  176 . 2D map  176  may be previously stored, or it may be generated by the user simultaneously while information about points of interest are being gathered. For example, if the system  30  is being used by law enforcement to investigate a crime scene at a building depicted by 2D map  176 , tool  302  configured as a flashlight may indicate evidence of interest to the investigator. The investigator, carrying the system  30 , may use tool  302  configured as a flashlight to notice broken glass at point of interest A in  FIG. 24 . The investigator could carry system  30  to point A, determine a position of point A within the 2D image using scanner  50 , and then inputting information about the broken glass at point A. This position and information could be linked in a data structure and stored as described above. Next, the investigator could notice a muddy footprint at point B using the flashlight (i.e., tool  302 ), and link it to a position in the 2D map  176  similar to the broken glass. As a second tool  304 , accessory  300  may include a camera that can take a photograph of the point of interest and link it to the point of interest in the data structure. 
     Tool  302  is not limited to a flashlight and can be any time of tool for helping to determine information about a location. For example, tool  302  may be a UV light that could help investigators determine the past presence of bodily fluids. If tool  302  configured as a UV light indicated bodily fluids at point C in 2D map  176 , the investigator could determine a position of point C and link information regarding the detected bodily fluids to coordinates of point C in a data structure, similar to the process described above. 
     It will also be understood that the tools on accessory  300  may be sensors that detect information instead of just passive lights that reveal information. For example, tool  302  may be a fingerprint detector. If an investigator discovered a crime victim at point D, he could carry the system  30  to point D, determine a position of point D, use tool  302  to record a fingerprint of the crime victim, and then create a data structure linking the fingerprint data to the coordinates of point D in 2D map  176 . 
     It will also be understood that accessory  300  is not limited to law enforcement applications. For example, if tool  302  is configured as a surface temperature detector, a firefighter could use system  30  to systematically scan a building for hot spots after a fire is extinguished, and link them to positions on the image so that they can be rechecked later. Alternatively, accessory  300  may be equipped with a Geiger counter or other radiation detector to be used by emergency personnel to map radiation exposure during a radiation event. Overall, it will be understood that any type of detector or sensor that can be sized to use in accessory  300  may be used as part of the system  30 . 
       FIG. 25  shows one possible embodiment of a data structure used to store the information linked to positions of the points of interest. For example, a point of interest may have coordinates X1, Y1 based on the coordinate system of an image such as 2D map  176 . These coordinates may be stored in a table that relates coordinates X1, Y1 with certain notes about the location, such as “broken glass.” A photograph or other information may also be linked to the coordinates. Alternatively, if the tool was a fingerprint scanner, the notes may indicate a description of the fingerprint, and a file with an image of the fingerprint may be attached. If the tool is a detector or sensor, the table may include values detected at each of the listed coordinates. Other types of data structures may be used to store the data, as long as an information regarding a point of interest is linked to the coordinates of the point of interest. 
     Next, a method for linking information of a point of interest to a position within a location will be described with respect to  FIG. 26 . In block  400 , an image depicting the location is acquired. The image may be a previously-recorded image that is retrieved from storage, or it may be an image that is being actively acquired and generated while information is going gathered. 
     In block  402 , a portable device present within the location depicted by the image is used to determine a position of the point of interest. The portable device may be system  30  as described above. In determining a position of the point of interest, the portable device may be brought to the point of interest. As the portable device has the capability to determine its position relative to other objects within the environment being scanned, a position of the point of interest may be determined and indicated on the image (e.g. a map). Alternatively, it may be possible to remotely determine a position of the point of interest by detecting a distance an angle from the portable device and calculating the position of the point of interest based on the measured distance and angle. The position may be expressed in the coordinate system of the image. 
     In block  404 , information related to the point of interest is acquired using a tool provided in an accessory operably coupled to the portable device. For example, information may be acquired using tool  302  provided in accessory  300 . The tool may be a passive tool such as a flashlight or UV light that reveals information to a user, which is then entered by the user into system  30 . Alternatively, the tool may be a sensor or detector that can actively detect a property of the point of interest, such as a camera, a fingerprint scanner, a temperature detector, radiation detector, etc. 
     In block  406 , a data structure linking the information to the position of the point of interest is created. This data structure may take the form of the table shown in  FIG. 25 . In block  408 , the data structure is stored in a storage, such as memory  80  for example. 
     The methods illustrated in  FIG. 26  may also be implemented as a non-transitory computer-readable medium storing therein computer-executable instructions that, when executed by a computer, cause the computer to perform the operations depicted by blocks  400 ,  402 ,  404 ,  406 , and  408 . 
       FIG. 27  shows another embodiment of a method for linking information of a point of interest to a position within a location. The method shown in  FIG. 27  is similar to the method shown in  FIG. 28 , with the addition of optional blocks  410 ,  412 . In block  410 , a user selects an accessory that has the desired tools. For example, if the user is investigating a crime scene, the user may select a tool that includes a UV light and a fingerprint scanner. Alternatively, if the user is investigating the remains of a structure fire, the user may select a tool that includes a flashlight and a surface temperature scanner. In block  412 , the selected accessory is coupled to the system. 
     The term “about” is intended to include the degree of error associated with measurement of the particular quantity based upon the equipment available at the time of filing the application. 
     The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, element components, and/or groups thereof. 
     While the invention has been described in detail in connection with only a limited number of embodiments, it should be readily understood that the invention is not limited to such disclosed embodiments. Rather, the invention can be modified to incorporate any number of variations, alterations, substitutions or equivalent arrangements not heretofore described, but which are commensurate with the spirit and scope of the invention. Additionally, while various embodiments of the invention have been described, it is to be understood that aspects of the invention may include only some of the described embodiments. Accordingly, the invention is not to be seen as limited by the foregoing description, but is only limited by the scope of the appended claims.