Patent Publication Number: US-2023160183-A1

Title: System for Tracking Earthmoving Progress

Description:
BACKGROUND OF THE INVENTION 
     At least one specification heading is required. Please delete this heading section if it is not applicable to your application. For more information regarding the headings of the specification, please see MPEP 608.01(a). Earthmoving and construction industries often suffer from inefficiencies in operation, and difficulties in accurately tracking the progress of machinery operators at work sites. For example, it is difficult for operators to determine the amount of digging soil removed by equipment at worksites, which can lead to inefficient utilization of hauling equipment, leading to increased cost and time required to complete an excavation. 
     In addition, there are numerous safety hazards present at work sites in which earth moving activity occurs, and the dynamic environment of the site means that hazards must be continually monitored to ensure compliance with workplace safety regulations. For instance, a crew at a work site is required to maintain certain trench geometries to reduce the risk of trench collapse and injury to persons at the work site. These geometries must be maintained while a trench is excavated. 
     Therefore, there is a need for systems and improved methods that can track the performance and progress of earthmoving activities to improve efficiency and safety of these operations. 
     SUMMARY OF THE INVENTION 
     In general terms, the present disclosure is directed toward an integrated sensing device that can be adapted to be used in conjunction with earth-moving equipment at a work site where excavation activity may be taking place. Such a sensing device uses a variety of sensor inputs in conjunction with processing and memory circuits to determine and store information about the work site towards the goal of improving the productivity and safety of the work site. 
     In an aspect of the present disclosure, an integrated sensing device is disclosed that includes an imaging sensor, a networking interface, a digital variable range detection system, a processing circuit, and a memory circuit. 
     In some examples the device uses a stereoscopic camera as an imaging device, a light ranging and detection system (LIDAR) as the range detection system, and a cellular modem as a networking interface. 
     In some examples, the sensing device can also include a positioning device, such as a Global Navigation Satellite System (GNSS) receiver. 
     In some examples, the sensing device includes a housing with which all components are installed, which enables placement on an earthmoving machine. 
     A method for operating a sensing device includes directing the sensing device, which includes imaging and digital range detection toward an excavator bucket. A processing circuit and memory circuit receive data outputs from the imaging and range detection sensors, then determines the position of excavator arms and the bucket and creates a surface model of the bucket. The processing circuit and memory circuit can then determine the contents of the bucket by comparing the surface model against previous data. 
     In some examples the method can include determination of the material composition of the bucket contents. 
     In some examples the method includes transmitting data from the processing circuit and memory circuit through the networking interface to a cloud database through an event handler. 
     Another method includes using the integrated sensing device Including an imaging sensor, variable range detection system, processing circuit, and networking interface, to create a surface model of a work sit 
     In some examples, the method includes determining the critical dimensions of a trench present within the model and determining whether these dimensions comply with previously established limits. 
     In some examples, the method includes determining whether electrical power transmission lines or suspended loads are present within the surface model and determines their location relative to the sensing device. 
     In some examples, the method includes determining if a pipe segment is present with the surface model, and includes determination of the composition, length, and width of the pipe segment. 
     In some examples, the method includes transmitting data from the processing circuit and memory circuit to a cloud database using a networking interface and a data handler. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG.  1    is a schematic diagram of a representative sensor system 
         FIG.  2    is a schematic diagram of an illustrative network with a representative sensor system. 
         FIG.  3    is an illustration of a method for using the disclosed system with an excavator for soil measurement. 
         FIG.  3 A  illustrates a method for system calculation of a loaded excavator bucket 
         FIG.  3 B  depicts a method for the creating a model of an empty excavator bucket 
         FIG.  3 C  depicts a method for updating a model of an empty bucket to account for changes in the apparent capacity 
         FIG.  4    is an illustration of a method for using the disclosed system for detecting objects placed in the ground. 
         FIG.  5    is an illustration of a method for using the disclosed system for assessing worksite safety risks. 
         FIG.  6    is an illustration of a method for using the disclosed system to create and update a survey model. 
         FIG.  7    is a flow chart of some of the important steps performed by the disclosed system in example methods of using said system 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     Note that the specific embodiments given in the drawings and following description do not limit the disclosure. On the contrary, they provide the foundation for one of ordinary skill to discern the alternative forms, equivalents, and modifications that are contemplated by the inventors and encompassed in the claim scope. 
       FIG.  1    discloses one embodiment of an integrated sensing device  1  comprising image sensors  2 , a processor  3 , a networking interface  4 , a local storage database  5 , a GPS receiver  35 , and LiDAR sensors  36 , contained in a housing  28 . The sensors  2  may comprise an optical stereo imaging camera. 
       FIG.  2    is a schematic of a network configuration for the sensor system  1  in context. If the sensor system  1  detects an unsafe configuration at the worksite, the system can trigger an audible alarm  6 . The sensor system collects data and syncs data and events with a cloud database  8  through an event handler  7 . The cloud database  8  is referenced via an application programming interface  9  to a client portal or user interface  10 . 
       FIG.  3    depicts the disclosed sensor system  1  mounted on top of the cab portion  28  of an excavator. The sensor system  1  scans the area in front of its field of view  11 . To measure the contents  12  of an excavator bucket  30 , the sensor system first detects the positions of the excavator arms  29  and bucket  30 . Once said positioning is recognized, the sensor system scans the contents  12  of the bucket  30  and determines whether the bucket is in a loaded or empty state. If the empty state is detected, the system creates surface models  31  of the empty bucket interior  14  using data from the image sensors  2  and the LiDAR sensors  36 . The surface models  31  are then updated in the local database  5 . When the system determines the bucket is empty or unloaded, system creates a new model of the bucket interior  14  and overlays the new model to looks for outliers  15  such as material that is stuck to the back of the bucket as shown in  FIG.  3 C . A living model of the bucket interior is thus updated with each scan of the unloaded bucket. 
     If the sensor system detects the bucket  30  is loaded with contents  12 , the system creates a volumetric model of the loaded bucket  13  ( FIG.  3 A ) from image sensor  2  and LiDAR sensor  36  data then and compares the loaded bucket model  13  against the living model of the empty bucket interior  31  to isolate and determine the volume of the bucket contents  12 . The system can use the data from the image sensors  2  to identify the composition of the bucket contents  12  by comparing the apparent texture and color of the contents against database information. The volume calculation and composition of the contents are read and stored to the local database  5 . 
       FIG.  4    depicts the sensor system  1  scanning a pipe installation  16 . The system uses object detection-based data from the image sensors  2  and LiDAR sensors  36  including reflectivity, color, measured width to length ratio, and dynamic apparent stiffness of the pipe to recognize pipe segments. Once the pipe  16  is placed at its installation location  32  the system records the length of the pipe, type of pipe, and depth of the pipe installation  33  to the local database  5 . 
       FIG.  5    depicts how the sensor system  1  detects various safety hazards while operating at a worksite  27 . The system can measure worksite trench  34  dimensions such as trench width  17 , slope angle  19 , trench depth  18 , benching width  22 , benching height  21 , trench lower portion depth  20  and lower portion width  21 . The system can detect the distance  23  excavated spoils  24  are located relative to the edge of a worksite trench  34 . The system also detects “struck-by” hazards including suspended loads  25  and identifies the location of power lines  26 . In the event an unsafe worksite condition is detected, the system writes an event to the local database  5  and can issue an audible alarm  6 . 
       FIG.  6    depicts how the sensor system  1  creates and updates a survey model of the worksite  27 . The system  1  uses its onboard image sensors  2 , LiDAR  36  and GPS  35 , combined based on image object detection to create measurements of terrain elevations  37  and positions  38  relative to system. This measurement data is stored and uploaded to the cloud database for further processing into living survey models, allowing for site progress to be realized in real time. 
     Numerous alternative forms, equivalents, and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the claims be interpreted to embrace all such alternative forms, equivalents, and modifications where applicable.