Patent Publication Number: US-2023160689-A1

Title: Oil rig drill pipe and tubing tally system

Description:
CROSS-REFERENCES TO RELATED APPLICATIONS 
     This application is a continuation of U.S. patent application Ser. No. 17/192,735, filed on Mar. 4, 2021 and entitled OIL RIG DRILL PIPE AND TUBING TALLY SYSTEM, which is a continuation of U.S. patent application Ser. No. 16/502,689, filed Jul. 3, 2019 and entitled OIL RIG DRILL PIPE AND TUBING TALLY SYSTEM, now U.S. Pat. No. 10,982,950, which is a continuation of U.S. patent application Ser. No. 14/939,089, filed Nov. 12, 2015 and entitled SYSTEM AND METHOD FOR LOCATING, MEASURING, COUNTING, AND AIDING IN THE HANDLING OF DRILL PIPES, now U.S. Pat. No. 11,378,387, which claims benefit of U.S. Provisional Application No. 62/078,577, filed Nov. 12, 2014 and entitled SYSTEM AND METHOD FOR LOCATING, MEASURING, COUNTING, AND AIDING IN THE HANDLING OF DRILL PIPES, all of which are hereby incorporated in their entireties by this reference. 
    
    
     FIELD OF THE INVENTION 
     Embodiments described herein relate to systems and methods for locating, measuring, counting, and aiding in the handling of drill pipes. 
     BACKGROUND AND SUMMARY 
     Modern drilling involves scores of people and multiple inter-connecting activities. Obtaining real-time information about ongoing operations is of paramount importance for safe, efficient drilling. As a result, modern rigs often have thousands of sensors actively measuring numerous parameters related to vessel operation, in addition to information about the down-hole drilling environment. 
     Despite the multitude of sensors on today&#39;s rigs, a significant portion of rig activities and sensing problems remain difficult to measure with classical instrumentation and person-in-the-loop sensing is often utilized in place of automated sensing. 
     By applying automated, computer-based video interpretation, continuous, robust, and accurate assessment of many different phenomena can be achieved through pre-existing video data without requiring a person-in-the-loop. Automated interpretation of video data is known as computer vision, and recent advances in computer vision technologies have led to significantly improved performance across a wide range of video-based sensing tasks. Computer vision can be used to improve safety, reduce costs and improve efficiency. 
     Handling and counting of drill pipes on a rig is typically accomplished using primarily human-in-the-loop techniques. For example, a person is responsible for maintaining an accurate log of the types, diameters and lengths of pipes entered into the well-bore as drilling progresses and responsible for counting pipes as they are removed from the well-bore. Although a relatively simple human endeavor, errors in pipe tallying can and do occur, and these errors can cause significant disruptions to drilling activities. 
     Classical instrumentation for pipe tallying is either time-consuming (e.g., manual measurement of each pipe) or not suitable for harsh down-well conditions (e.g., RFID tagging). In contrast, computer vision technologies can be utilized to perform many of the activities currently undertaken manually, providing significant savings in drilling time and cost and reducing the risk from pipe tally errors. These techniques provide a more accurate technique for generating pipe tallies and can significantly reduce rig down-time due to pipe tally errors; potentially saving millions of dollars per year. Therefore, there is a need for an automated computer vision based technique for measuring pipe lengths and diameters, and counting pipe segments as they enter into or are removed from the well-bore. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG.  1    depicts one of many embodiments of a system involving multiple cameras and CPUs for monitoring drilling pipe and assisting in drilling pipe handling. 
         FIG.  2    depicts a potential series of steps involved in a system for monitoring drilling pipe and assisting in drilling pipe handling. 
         FIG.  3    depicts a potential series of steps involved in visually analyzing pipe detection. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     The “Pipe Tally System” system, PTS, consists of several parts. In one preferred embodiment, one or more video cameras  102  positioned so as to be able to see the drilling pipe  106  as it is attached to, or removed from the drill-string  107 . Depending on requirements, one camera  102  at sufficient distance from the bore-hole  108  to view the entire pipe  106  segment at once may be sufficient, otherwise two or more cameras  102  may be used, each of which may only see part of the pipe  106  as it is entered into the drill-string  107 , but information or data  150  can be aggregated across the different cameras using the known camera positions and poses. 
     Each camera  102  may contain or be connected to a computer  110  which detects and localizes pipes  106 , the iron roughneck  116 , the elevator  118 , or other relevant components. Different regions of interest for each object type can be defined using the known camera  102  geometry, or using user-inputs. Since the cameras  102  are at known distances from the bore-hole  108 , camera transform information can be used to calculate the pipe lengths and diameters as they are tracked into and out of the well-bore  108 . Information about the well-state, including the number of pipe stands and pipe segments  106  in the well may be accumulated on a central computing resource  110 . In an alternative embodiment involving multiple cameras  102 , pipe length, diameter, location and tracking information may be calculated by accumulating information  150  about the pipe  106  across multiple camera feeds. 
     Pipes  106  on a rig may be marked with paint or other marking system to make them easier to detect (e.g., a colorful stripe of paint near either end of the pipe  106  can help in detection, localization and length estimation). 
     In certain embodiments, the resulting information  150  about pipes  106  may be amalgamated into an automatically generated well-state report which may include a pipe tally (information about the pipe lengths and diameters, time the pipe  106  was added to or removed from the drill-string  107 , or any other pipe  106  specific information). Automatic alarms  120  may be raised to the attention of the drill team (1) if at any time the automatic pipe tally does not match a manually generated pipe tally, (2) if a new piece of pipe  106  being added to the drill-string  107  is not commensurate with the current drill-string  107  (e.g., wrong pipe diameter), or (3) any other condition arises in which an alarm  120  is desired. In  FIG.  1   , cameras  102  are mounted around the drill-string  107 , oriented to be able to see new segments of pipe  106  as they are added to the drill-string  107 , or as they are removed from the drill-string  107 . If the rig design allows it, one camera  102  may be sufficient. Alternatively, multiple cameras  102 , each of which may only be able to see part of the pipe  106  can also be utilized. In some embodiments, cameras  102  may also be able to see well-bore  108 , rough neck  116  and elevator  118 . Processor  110  is connected to cameras  102  and capable of analyzing the visual data  150  gathered by the cameras  102 . 
     In  FIG.  2   , a potential configuration of the disclosed system is shown. Cameras  102  are connected to processor  110 . Processor  110  may be connected to a logging system  122 , an alarm  120  and/or a display device  124 . It will be appreciated that many embodiments may contain greater or fewer cameras  102 , processors  110  or other components than specifically shown in  FIG.  2   . 
       FIG.  3    shows the steps involved in a potential method for locating, measuring, counting, and/or aiding in the handling of drill pipes. The method includes acquiring visual data  202 , analyzing visual data  204 , recording data  206 , displaying data  208 , alerting staff  210  and interrupting operations  212 . 
     Specific regions of the scene (region of interest) may be identified during installation to specify the location of the vertical region above the well-bore  108 , the location of the iron roughneck  116 , or other relevant locations in each camera&#39;s  102  field of view. 
     During installation, the locations and poses of each camera  102  may be recorded. Camera locations can be finely estimated using standard camera calibration techniques (e.g., fiducial objects of known size and location in each camera&#39;s  102  field of view) immediately after installation, or whenever the cameras  102  have moved enough to require re-calibration. 
     In the case of multiple cameras  102 , at least one camera  102  should be able to see the top and another camera  102  see the bottom of the pipe  106  at the same time when the pipe  106  is directly above the drill-string  107 . 
     Pipe  106 , roughneck  116  and/or elevator  118  detection may be accomplished using a combination of techniques. In an alternative embodiment, adaptive background estimation and subtraction models (e.g., adaptive Gaussian mixture models) may be applied to perform foreground and/or background segmentation. Since the background should be relatively stable over the time-frames during which each object is in-frame, adaptive background updating can be halted when a specific object is detected. This prevents the background estimation from “learning” the pipe  106  as part of the background. Furthermore, shape and size constraints can be applied to reduce false-alarms due to other non-pipe related changes in the scene. Pipes  106  used in drilling are long and narrow, and the diameters of the pipes  106  under consideration are tightly constrained. As a result, object aspect ratio and object size (given known camera  102  location relative to the drill-string  107 ) can be used to reduce non-pipe false alarms. 
     Changes in the background that are approximately the correct size and shape are then sent to a confirmation step, which takes into account features extracted from the detected regions. These features include pixel values, color histograms, and texture features, since each of these is indicative of the material composition of the object under consideration. A support vector machine trained to recognize pipe  106 , roughneck  116 , and/or elevator  118  like regions is then applied to the features extracted from each foreground region. The detections may be input into a finite state machine ( FIG.  4   ). 
     Finite state machine logic systems may be used to ensure that the pipe tally is accurate by ensuring that the computer vision system  100  only increments the pipe tally when a suitable series of events has transpired.  FIG.  4    shows a finite state machine which may be used for incrementing the pipe tally during tripping out of the hole. 
     In each state, state specific variables may be calculated and recorded. For example, the pipe tracker uses a combination of point-matching (using Harris features and SIFT and BRIEF descriptors) within the pipe region, as well as Lucas-Kanade optical-flow techniques to estimate the per frame velocity of the pipe  106 . If the aggregate motion of the pipe  106  is “in well” (down), the pipe  106  is considered added to the drill-string  107 , and this is marked in the pipe tally. If the aggregate motion of the pipe  106  is “out of well” (up), the pipe  106  is considered removed from the drill-string  107 , and this is marked in the pipe tally. 
     Once a pipe  106  is tracked, its length and diameter may be constantly estimated and updated over time as long as the pipe  106  is in-frame. Estimation of the pipe length and diameter are possible since the aggregate change detection and pipe detection steps described above result in a bounding-box in image space containing the projection of the pipe  106  into the frame. Given the pixels comprising the pipe  106 , and the camera  102  location and pose information, it is possible to measure the pipe diameter and length. These measurements are refined over time to reduce uncertainty and noise due to inter-pixel variance. 
     When the pipe  106  exits the scene (whether into or out of the well), the average measured pipe length and diameter may be provided to the pipe tally. For pipes  106  exiting the well, if these values do not agree with the same values measured when the pipe  106  entered the well, an alarm  120  may be raised. For pipes  106  entering the well, if these values are outside the normal bounds, or are not commensurate with the previous pipe  106  to enter the well, an alarm  120  may be raised. 
     Embodiments disclosed herein may relate to a system for locating, measuring, counting or aiding in the handling of drill pipes  106 . The system may include at least one camera  102  which is operably connected to at least one processor  110 . The camera  102  may be capable of gathering visual data  150  regarding detecting and/or localizing components of a drilling rig which may include pipes  106 , drill pipes, roughnecks  116 , elevators  118 , drill-string components and combinations thereof. The processor  110  may be configured to analyze the visual data and may also be operably connected to the pipe elevator  118 . The processor may be configured to halt elevator  118  operations when the visual data is outside of a pre-determined set of conditions. The system may also include at least one logging system  124  connected to said processor  110  for recording said visual data  150  and any analyzed data. 
     Certain embodiments may also include a display system  122  for displaying the collected and/or analyzed data. Embodiments may include a camera  102  which also comprises the processor  110 . Embodiments of the system may also include an alarm  120  for alerting staff to the occurrence of a predetermined condition. 
     Disclosed embodiments may also relate to a method for locating, measuring, counting or aiding in the handling of drill pipes. The method includes acquiring visual data from at least one camera  102 , analyzing said visual data  150 , recording said analyzed data and disrupting the operations of a pipe elevator in response to a pre-determined condition. 
     Certain embodiments may also include displaying the acquired, analyzed or recorded data on a display device  122 . Embodiments may also include alerting staff to any occurrence of a pre-determined condition or any measurement that falls outside of a pre-determined range using an alarm  120 . 
     Additional embodiments relate to a system for assisting in the handling of drill pipe segments. The system may include a well-bore  108  which is being worked by a drill-string  107 . The drill-string  107  may comprise a plurality of drill pipe  106  segments. The system may also contain at least one camera  102  configured to observe the addition or subtraction of drill pipe  106  segments to the drill-string  107  and gathering visual data  150 . The camera  102  may be operably connected to a processor  110 . The process  110  may be capable of analyzing the visual data  150 . 
     Certain embodiments may also include a logging system  124  connected to the processor  110 . Embodiments may also include a display system  122  for displaying the collected and/or analyzed data. Some embodiments may include a camera  102  which includes a processor  110 . Embodiments may also contain an alarm  120  for alerting staff of the occurrence of a pre-determined condition.