Patent Document

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT 
       [0001]    The invention described herein was made by an employee of the United States Government and may be manufactured and used by the Government of the United States of America for governmental purposes without the payment of any royalties thereon or therefore. 
     
    
     BACKGROUND OF THE INVENTION 
       [0002]    1. Field of Invention 
         [0003]    This invention relates to the field of image analysis and more specifically to image enhancement and restoration. 
         [0004]    2. Description of Related Art 
         [0005]    Soil density is a critical factor when designing or building structures, or excavating for construction. If a structure stands on soil with a low or extremely variable density, it can collapse, sink, deform or otherwise become unsafe. Architects and engineers must either redesign a structure to fit a particular soil density or alter the soil density to ensure stable structural footing. Either scenario requires first testing the soil to determine density. 
         [0006]    Nuclear density gauges (NDG) are the fastest density testing systems, requiring less than 5 minutes to conduct an individual test. An NDG uses nuclear sources to determine bulk soil density and water mass within the soil. These radioactive sources present a health risk to users and a danger if stolen for illegal purposes, requiring constant monitoring of users&#39; radiation exposure levels and storage under high security. Transport of an NDG requires significant paperwork and can be difficult in both the US and around the world. 
         [0007]    Volume replacement systems require the user to excavate a hole and fill it with calibrated materials. The technique is laborious and time intensive, and limited in the hole size useable. Users must transport calibrated materials to the site. To obtain an accurate volume, the calibrated materials must fill all the gaps in the hole, which may prove difficult for coarser soils. Adding the calibrated materials to the hole may enlarge the hole, making any resulting data inaccurate. 
         [0008]    Electronic devices require pre-calibration to the soil of interest before returning a useable soil density to the operator. Device calibration requires use of a secondary density device such as a volume replacement system or NDG, or laboratory work requiring several days of testing the soil of interest. Calibration increases expense, labor and time, limiting the usefulness of these devices to continuous testing on a single soil. 
         [0009]    There is an unmet need in the art for an accurate, easily used soil density measurement system that does not require extensive calibration, or significant health, safety, or security measures. 
       BRIEF SUMMARY OF THE INVENTION 
       [0010]    The present invention is an apparatus for analyzing soil density utilizing a user-selected ground plane. The apparatus is a data processor configured with software to perform a photogrammetry method. First, the photogrammetry method receives a soil mass value M of excavated soil. Next, the photogrammetry method receives a variant image set of a calibration object and excavation site. The photogrammetry method then creates a point cloud from the variant image set. Next, the photogrammetry method instantiates a point cloud data object with point cloud data values to display a visual representation of the excavation site and the calibration object on a graphic user interface (GUI). The photogrammetry method then updates the point cloud data values using an autorotation method to orient the visual representation on the GUI. Next, the photogrammetry method updates the point cloud data values using a scaling method to scale the visual representation on the GUI. The photogrammetry method then displays a visual representation of the point cloud data object on the GUI. Next, the photogrammetry method receives plane coordinate values for a user-selected ground plane. The photogrammetry method then calculates an excavated volume V h  using a cubic volumetric method, wherein the plane coordinate values are utilized to define at least one boundary of a volumetric cube utilized in the cubic volumetric method. Next, the photogrammetry method calculates a soil density value D using the excavated volume V h  and the soil mass value M. The photogrammetry method then displays the soil density value D on the GUI. 
     
    
     
       BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWING(S) 
         [0011]      FIG. 1  illustrates an exemplary embodiment of a photogrammetry system. 
           [0012]      FIGS. 2 a  and 2 b    illustrate a flowchart of an exemplary embodiment of a photogrammetry method. 
           [0013]      FIG. 3  illustrates a flowchart of an exemplary embodiment of an autorotation method. 
           [0014]      FIG. 4  illustrates a flowchart of an exemplary embodiment of a scaling method. 
           [0015]      FIG. 5  illustrates a flowchart of an exemplary embodiment of a cubic volumetric method. 
       
    
    
     TERMS OF ART 
       [0016]    As used herein, the term “autorotation method” means a method for adjusting the relative angulation of a visual representation. 
         [0017]    As used herein, the term “calibration object” means an object of known dimensions. 
         [0018]    As used herein, the term “cubic volumetric method” means a method for determining the volume of an excavation site using at least one volumetric cube. 
         [0019]    As used herein, the term “excavated volume” means a volume of soil removed from an excavation site. 
         [0020]    As used herein, the term “excavation site” means a location from which soil is removed. 
         [0021]    As used herein, the term “plane coordinate values” means the values of coordinates of a ground plane. 
         [0022]    As used herein, the term “point cloud” means a data set containing three dimensional information extracted from two or more images of the same object. This information may include, but is not limited to, a quasi-unique set of coordinate values along the X-, Y- and Z-axis and/or a quasi-unique set of color levels using red, green and blue levels, denoted as R, G and B, respectively. 
         [0023]    As used herein, the term “scaling method” means a method for adjusting the relative location of points in a visual representation. 
         [0024]    As used herein, the term “soil density” means the bulk density of soil. 
         [0025]    As used herein, the term “soil mass value” means the mass of an excavated volume of soil. 
         [0026]    As used herein, the term “variant image set” means more than one image of the same subject matter wherein each image is adjusted for a particular variable including but not limited to magnification, elevation and angle. 
         [0027]    As used herein, the term “user-selected ground plane” means a horizontal plane selected by a user and extending parallel to a ground surface, intersecting a z-axis at a z-axis intersection point z gp . 
         [0028]    As used herein, the term “visual representation” means data displayed visually. 
         [0029]    As used herein, the term “volumetric cube” means a cube which contains the boundaries of a visual representation of an excavation site. 
       DETAILED DESCRIPTION OF THE INVENTION 
       [0030]      FIG. 1  illustrates an exemplary embodiment of a photogrammetry system  100 . System  100  includes a calibration object  10 , a container  20 , a mass scale  30 , an imaging apparatus  40  and a data processor  50 . 
         [0031]    Calibration object  10  is any object having a known size. In the exemplary embodiment, calibration object is a flat, annular ring having known inner and outer diameters. The respective dimensions of calibration object  10  vary based on the application, but are large enough to accommodate removal of soil to create an excavation site. In the exemplary embodiment, calibration object  10  is brightly colored to increase contrast and visibility against soil. In certain embodiments, calibration object  10  has a low-reflective coating. 
         [0032]    Container  20  is a container having known mass, of sufficient volume to hold all soil removed to create the excavation site. In various embodiments, container  20  is a bowl, box or bag. Placing the soil in container  20  allows measurement of soil mass using scale  30 . 
         [0033]    Imaging apparatus  40  is a digital imaging apparatus capable of capturing multiple images of the excavation site which form a variant image set. In various embodiments, imaging apparatus  40  is a cell phone camera, still camera, video camera or scanner. Imaging apparatus  40  is connected to data processor  50  or a removable data storage unit, allowing transfer of the images. The connection may be wired or wireless. Optionally, imaging apparatus  40  may include a light source to illuminate the excavation site in low-light conditions. 
         [0034]    Data processor  50  is configured with software allowing it to process the images received and determine soil density using a photogrammetry method  200 . In the exemplary embodiment, data processor  50  is a laptop computer with a user interface  51 . 
         [0035]      FIGS. 2 a  and 2 b    illustrate a flowchart of an exemplary embodiment of photogrammetry method  200 . 
         [0036]    In step  202 , method  200  places calibration object  10  on the upper surface of a soil. 
         [0037]    In step  204 , method  200  excavates soil to form an excavation site. In the exemplary embodiment, the excavation site is a hole having a convex, substantially conical shape. 
         [0038]    In step  206 , method  200  places all excavated soil within container  20 . In the exemplary embodiment, steps  204  and  206  occur simultaneously. 
         [0039]    In step  208 , method  200  obtains a soil mass value M of the excavated soil using scale  30 . 
         [0040]    In step  210 , method  200  creates a variant image set from digital images of the excavation site from multiple angles, magnifications and/or elevations. These digital images show both the excavation site and calibration object  10 . In the exemplary embodiment, step  210  creates at least 16 images: eight images at a first magnification every 45 degrees and eight images at a second, increased magnification every 45 degrees. 
         [0041]    In step  212 , method  200  opens a graphic user interface (GUI). 
         [0042]    In step  214 , method  200  receives at least one camera data value. Camera data values are any metadata describing the camera configuration when the digital images were created. Camera data values may include, but are not limited to, camera make and model, lens aperture, focal length, camera shutter speed, exposure program, focal ratio, lens type, metering mode, flash configuration and ISO sensitivity. These may be entered by a user or automatically retrieved from the digital images. 
         [0043]    In step  216 , method  200  creates a point cloud from information extracted from each digital image, as well as the camera data values. The point cloud is a plurality of pixels extracted from each digital image. Each pixel has a quasi-unique set of coordinate values along the X-, Y- and Z-axis. Each pixel also has a quasi-unique set of color levels using red, green and blue levels, denoted as R, G and B, respectively. 
         [0044]    In step  218 , method  200  instantiates a point cloud data object. 
         [0045]    In step  220 , method  200  updates the point cloud data object with point cloud information extracted from the point cloud. This information includes data values for the pixel identifier, pixel x-coordinate, pixel y-coordinate, pixel z-coordinate, pixel R-level, pixel G-level and pixel B-level. 
         [0046]    In step  222 , method  200  updates the pixel x-coordinate data values, pixel y-coordinate data values and pixel z-coordinate data values of the point cloud data object using autorotation method  300 . 
         [0047]    In step  224 , method  200  displays a visual representation of the excavation site and said calibration object on the GUI using the point cloud data object. Due to the use of autorotation method  300  in step  220 , the visual representation of the surface of the point cloud will appear to be perpendicular to the screen. 
         [0048]    In optional step  226 , method  200  updates the pixel x-coordinate data values, pixel y-coordinate data values and pixel z-coordinate data values of the point cloud data object using input values for θ x  and/or θ y  for steps  306  and/or  312  of autorotation method  300 . The input values for θ x  and/or θ y  may be predetermined or entered manually. 
         [0049]    In step  228 , method  200  updates the pixel x-coordinate data values, pixel y-coordinate data values and pixel z-coordinate data values of the point cloud data object using scaling method  400 . 
         [0050]    In step  230 , method  200  displays an updated visual representation on the GUI. 
         [0051]    In step  232 , method  200  receives plane coordinate values for a user-selected ground plane and calculates an excavated volume V h  using cubic volumetric method  500 . 
         [0052]    In step  234 , method  200  receives an input soil mass value M for the mass of the excavated soil. 
         [0053]    In optional step  236 , method  200  receives an input value for moisture content ω of the excavated soil. 
         [0054]    In optional step  238 , method  200  adjusts the value for mass M of the excavated soil based on the value for moisture content ω of the excavated soil using the following equation: 
         [0000]    
       
      
       M=ω*M 
       i  
      
     
         [0000]    where M i  is the original mass. 
         [0055]    In step  240 , method  200  calculates a soil density value D using the excavated volume V h  of the excavation site and soil mass value M. 
         [0056]    In step  242 , method  200  outputs the soil density value D. 
         [0057]    In certain embodiments, method  200  repeats steps  232 - 242  to obtain a new soil density value D. These steps may be iteratively repeated to provide multiple potential soil density values D. In certain embodiments, method  200  repeats steps  202 - 242  to obtain a comparative soil density value D for a different excavation site. 
         [0058]      FIG. 3  illustrates a flowchart of an exemplary embodiment of autorotation method  300 . 
         [0059]    In step  302 , method  300  extracts the largest pixel y-coordinate data value y max  and the smallest pixel y-coordinate data value y min  from the point cloud data object, along with the corresponding pixel z-coordinates, z ymax  and z ymin , respectively. 
         [0060]    In step  304 , method  300  calculates an x-axis angle of adjustment θ x  using the following equation: 
         [0000]    
       
         
           
             
               θ 
               x 
             
             = 
             
               
                 tan 
                 
                   - 
                   1 
                 
               
                
               
                 ( 
                 
                   
                     
                       z 
                       ymax 
                     
                     - 
                     
                       z 
                       ymin 
                     
                   
                   
                     
                       y 
                       max 
                     
                     - 
                     
                       y 
                       min 
                     
                   
                 
                 ) 
               
             
           
         
       
     
         [0061]    In step  306 , method  300  updates each pixel x-coordinate, y-coordinate and z-coordinate data value in the point cloud data object with an updated pixel x-coordinate data value x′ n , updated pixel y-coordinate data value y′ n  and updated pixel z-coordinate data value z′ n , respectively, using the equation: 
         [0000]    
       
         
           
             
               [ 
               
                 
                   
                     
                       
                         
                           
                             x 
                             n 
                             ′ 
                           
                         
                       
                       
                         
                           
                             y 
                             n 
                             ′ 
                           
                         
                       
                     
                   
                 
                 
                   
                     
                       z 
                       n 
                       ′ 
                     
                   
                 
               
               ] 
             
             = 
             
               
                 [ 
                 
                   
                     
                       
                         x 
                         n 
                       
                     
                   
                   
                     
                       
                         y 
                         n 
                       
                     
                   
                   
                     
                       
                         z 
                         n 
                       
                     
                   
                 
                 ] 
               
               * 
               
                 [ 
                 
                   
                     
                       1 
                     
                     
                       0 
                     
                     
                       0 
                     
                   
                   
                     
                       0 
                     
                     
                       
                         cos 
                          
                         
                             
                         
                          
                         
                           θ 
                           x 
                         
                       
                     
                     
                       
                         
                           - 
                           sin 
                         
                          
                         
                             
                         
                          
                         
                           θ 
                           x 
                         
                       
                     
                   
                   
                     
                       0 
                     
                     
                       
                         sin 
                          
                         
                             
                         
                          
                         
                           θ 
                           x 
                         
                       
                     
                     
                       
                         cos 
                          
                         
                             
                         
                          
                         
                           θ 
                           x 
                         
                       
                     
                   
                 
                 ] 
               
             
           
         
       
     
         [0062]    where x n  is the current pixel x-coordinate data value in the point cloud data object, y n  is the current pixel y-coordinate data value in the point cloud data object, z n  is the current pixel z-coordinate data value in the point cloud data object and n equals the number of pixels. 
         [0063]    In step  308 , method  300  extracts the largest pixel x-coordinate data value x max  and the smallest pixel x-coordinate data value x min  from the point cloud data object, along with the corresponding pixel z-coordinates, z xmax  and z xmin , respectively. 
         [0064]    In step  310 , method  300  calculates a y-axis angle of adjustment θ y  using the following equation: 
         [0000]    
       
         
           
             
               θ 
               y 
             
             = 
             
               
                 tan 
                 
                   - 
                   1 
                 
               
                
               
                 ( 
                 
                   
                     
                       z 
                       xmax 
                     
                     - 
                     
                       z 
                       xmin 
                     
                   
                   
                     
                       x 
                       max 
                     
                     - 
                     
                       x 
                       min 
                     
                   
                 
                 ) 
               
             
           
         
       
     
         [0065]    In step  312 , method  300  updates each pixel x-coordinate, y-coordinate and z-coordinate data value in the point cloud data object with an updated pixel x-coordinate data value x′ n , updated pixel y-coordinate data value y′ n  and updated pixel z-coordinate data value z′ n , respectively, using the equation: 
         [0000]    
       
         
           
             
               [ 
               
                 
                   
                     
                       
                         
                           
                             x 
                             n 
                             ′ 
                           
                         
                       
                       
                         
                           
                             y 
                             n 
                             ′ 
                           
                         
                       
                     
                   
                 
                 
                   
                     
                       z 
                       n 
                       ′ 
                     
                   
                 
               
               ] 
             
             = 
             
               
                 [ 
                 
                   
                     
                       
                         x 
                         n 
                       
                     
                   
                   
                     
                       
                         y 
                         n 
                       
                     
                   
                   
                     
                       
                         z 
                         n 
                       
                     
                   
                 
                 ] 
               
               * 
               
                 [ 
                 
                   
                     
                       
                         cos 
                          
                         
                             
                         
                          
                         
                           θ 
                           y 
                         
                       
                     
                     
                       0 
                     
                     
                       
                         
                           - 
                           sin 
                         
                          
                         
                             
                         
                          
                         
                           θ 
                           y 
                         
                       
                     
                   
                   
                     
                       0 
                     
                     
                       1 
                     
                     
                       0 
                     
                   
                   
                     
                       
                         sin 
                          
                         
                             
                         
                          
                         
                           θ 
                           y 
                         
                       
                     
                     
                       0 
                     
                     
                       
                         cos 
                          
                         
                             
                         
                          
                         
                           θ 
                           y 
                         
                       
                     
                   
                 
                 ] 
               
             
           
         
       
     
         [0066]      FIG. 4  illustrates a flowchart of an exemplary embodiment of scaling method  400 . 
         [0067]    In step  402 , method  400  instantiates a scaling data object. 
         [0068]    In step  404 , method  400  updates the scaling data object with data values for outer point x-coordinate x o , outer point y-coordinate y o , inner point x-coordinate x i , inner point y-coordinate y i  and a scale value S. Scale value S is a known quantity. In the exemplary embodiment, scale value S is the distance between two opposing sides of calibration object  10 . In other embodiments, scale value S may be a distance along a linear object, or a distance between two defined points on an object. 
         [0069]    In one embodiment, a user manually enters the data values for outer point x-coordinate x o , outer point y-coordinate y o , inner point x-coordinate x i , inner point y-coordinate y i  and a scale value S. In another embodiment, at least one of the data values for outer point x-coordinate x o , outer point y-coordinate y o , inner point x-coordinate x i , inner point y-coordinate y i  and a scale value S is entered manually and at least one of the data values for outer point x-coordinate x o , outer point y-coordinate y o , inner point x-coordinate x i , inner point y-coordinate y i  and a scale value S is entered by clicking on a point on the visual representation of the point cloud data object on the GUI. 
         [0070]    In step  406 , method  400  calculates a coordinate distance C between an inner point and an outer point using the equation: 
         [0000]        C =√{square root over (( x   o   −x   i ) 2 +( y   o   −y   i ) 2 )}
 
         [0071]    In step  408 , method  400  calculates a scaling factor F s  using the equation: 
         [0000]    
       
      
       F 
       s 
       =S/C  
      
     
         [0072]    In step  410 , method  400  updates data values for the pixel x-coordinate, pixel y-coordinate, pixel z-coordinate by multiplying each pixel x-coordinate data value, pixel y-coordinate data value and pixel z-coordinate data value in the point cloud data object by scaling factor F s . 
         [0073]      FIG. 5  illustrates a flowchart of an exemplary embodiment of cubic volumetric method  500 . 
         [0074]    In step  502 , method  500  receives plane coordinate values for a user-selected ground plane. In the exemplary embodiment, a user utilizes a slider bar on a GUI to move a visual plane representation along the Z-axis through the visual representation of the excavation site and calibration object. The user moves the slider bar until the ground plane intersects the perimeter of the excavation site. 
         [0075]    In step  504 , method  500  instantiates a perimeter data object. 
         [0076]    In step  506 , method  500  updates the perimeter data object with information extracted from the point cloud data object. This information includes data values for the pixel identifier, pixel x-coordinate and pixel y-coordinate of pixels that intersect the user-selected ground plane. 
         [0077]    In step  508 , method  500  extracts the largest perimeter x-coordinate data value x pmax , the smallest perimeter x-coordinate data value x pmin , the largest perimeter y-coordinate data value y pmax , the smallest perimeter y-coordinate data value y pmin  from the perimeter data object. 
         [0078]    In step  510 , method  500  creates a volumetric cube having outer boundaries defined by the largest perimeter x-coordinate data value x pmax , the smallest perimeter x-coordinate data value x pmin , the largest perimeter y-coordinate data value y pmax , the smallest perimeter y-coordinate data value y pmin , z-axis intersection point z gp , and the smallest z-coordinate data value z min  from the point cloud data object. 
         [0079]    In step  512 , method  500  divides the volumetric cube symmetrically into a cube grid comprising a plurality of sub-cubes having identical volume. The number of sub-cubes may be entered manually, selected from a menu or preprogrammed. Each sub-cube is located between the largest perimeter x-coordinate data value x pmax , the smallest perimeter x-coordinate data value x pmin , the largest perimeter y-coordinate data value y pmax , the smallest perimeter y-coordinate data value y pmin , z-axis intersection point z gp , and the smallest z-coordinate data value z min . The boundaries of each sub-cube are defined by a largest perimeter x-coordinate data value x cmax , a smallest perimeter x-coordinate data value x cmin , a largest perimeter y-coordinate data value y cmax , a smallest perimeter y-coordinate data value y cmin , a largest perimeter z-coordinate data value z cmax  and a smallest perimeter z-coordinate data value z cmin . 
         [0080]    In step  514 , method  500  instantiates a cube grid data object. 
         [0081]    In step  516 , method  500  updates the cube grid data object with cube grid information extracted from the cube grid. This information includes data values for the sub-cube identifier, the sub-cube volume, the largest sub-cube perimeter x-coordinate data value x cmax , the smallest sub-cube perimeter x-coordinate data value x cmin , the largest sub-cube perimeter y-coordinate data value y cmax , the smallest sub-cube perimeter y-coordinate data value y cmin , the largest sub-cube perimeter z-coordinate data value z cmax  and the smallest sub-cube perimeter z-coordinate data value z cmin . 
         [0082]    In step  518 , method  500  discards all data values for any sub-cubes located directly between the point cloud and the volumetric cube, as determined by the largest sub-cube perimeter x-coordinate data value x cmax , the smallest sub-cube perimeter x-coordinate data value x cmin , the largest sub-cube perimeter y-coordinate data value y cmax , the smallest sub-cube perimeter y-coordinate data value y cmin , the largest sub-cube perimeter z-coordinate data value z cmax  and the smallest sub-cube perimeter z-coordinate data value z cmin . Method  500  removes all data values for discarded sub-cubes from the cube grid data object. 
         [0083]    In optional step  520 , method  500  updates the sub-cube volume data value of any sub-cubes that pass through pixels in the point cloud to exclude any volume located directly between the point cloud and the volumetric cube. 
         [0084]    In step  522 , method  500  sums the remaining sub-cube volume data values to calculate the excavated volume V h . 
         [0085]    It will be understood that many additional changes in the details, materials, procedures and arrangement of parts, which have been herein described and illustrated to explain the nature of the invention, may be made by those skilled in the art within the principle and scope of the invention as expressed in the appended claims. 
         [0086]    It should be further understood that the drawings are not necessarily to scale; instead, emphasis has been placed upon illustrating the principles of the invention. Moreover, the term “substantially” as used herein may be applied to modify any quantitative representation that could permissibly vary without resulting in a change in the basic function to which it is related.

Technology Category: 3