Augmented reality system for use in horizontal directional drilling operations

A system for tracking and steering a downhole tool using an augmented reality device. A tracker tracks the location of a downhole tool as it moves underground and transmits data to the device, while one or more sensors measure a position and orientation of the device. The device analyzes the data received from the tracker and the sensors and generates a virtual image of the downhole tool. The virtual image is displayed on the device at its detected location relative to the ground surface and relative to the position of the device. The position of the displayed virtual image is modified in response to updated information from the tracker or the sensors. Virtual images representing various parameters of the drilling operation are also displayed on the device in juxtaposition with the virtual image of the downhole tool.

SUMMARY

The present disclosure is directed to a system comprising a drill rig supported on a ground surface, a downhole tool positioned beneath the ground surface, and a drill string having a first end and a second end, in which the first end of the drill string is attached to the downhole tool and the second end of the drill string is attached to the drill rig. The system further comprises a portable, above-ground tracker having an antenna configured to detect a magnetic dipole field emitted from the downhole tool, and an augmented reality device having a field of view and a screen, in which the screen depicts one or more images within the field of view, and one or more sensors supported on the device and configured to determine a position of the device relative to the downhole tool. The system further comprises one or more controllers in communication with the device, the tracker, and the one or more sensors. The one or more controllers are configured to determine a position and orientation of the downhole tool, generate a virtual image of the downhole tool relative to the ground surface based on the information received from the tracker and the one or more sensors, and display the virtual image on the screen.

The present disclosure is also directed to a method. The method comprises the steps of driving a downhole tool attached to a drill string along an underground borepath, and tracking a location of the downhole tool using a portable, above-ground tracker. The method further comprises the steps of transmitting the location of the downhole tool to an augmented reality device, and generating a virtual image of a position of the downhole tool relative to the ground surface.

DETAILED DESCRIPTION

With reference toFIG.1, a horizontal directional drilling system10is shown. The system10is used to create a borehole12under an above-ground obstacle, such as a roadway. The system10uses a drill string14having a first end16and a second end18. The drill string14is attached to a drill rig22at its first end16and a drill bit24at its second end18. The drill rig22is supported on a ground surface26and is operated by a rig operator28positioned at an operator station30, as shown inFIG.10. The drill string14transmits thrust and rotation force from the drill rig22to the drill bit24.

The drill string14is made up of a plurality of hollow pipe sections32arranged in an end-to-end relationship. In some embodiments, each pipe section is made of a single pipe section. In other embodiments, each pipe section comprises an inner pipe section disposed within an outer pipe section. Such pipe sections, when joined together, make up an inner and outer drive train.

Continuing withFIG.1, a downhole tool34is attached to the second end18of the drill string14. The downhole tool34carries the drill bit24and houses a beacon. The beacon is configured to emit a magnetic dipole signal36. An above-ground tracker38, operated by a tracker operator40, is configured to detect and analyze the beacon signal36in order to determine the downhole position of the beacon. The beacon signal36includes information about the beacon as well as the downhole conditions, such as temperature and fluid pressure. One embodiment of a tracker and its methods of use are described in U.S. Pat. No. 7,786,731 issued to Cole et al., the contents of which are incorporated herein by reference.

The drill bit24shown inFIG.1comprises a slant face. The slant face is used to steer the downhole tool34as it bores. The angled nature of the slant face directs the downhole tool34in different directions depending on the tool's roll position—the direction the slant face is facing as the downhole tool34rotates about as horizontal axis.

In an alternative embodiment, the downhole tool34may be deflected in different directions by a bent sub42included in the drill string14, as shown inFIG.9. The downhole tool34may carry a traditional tri-cone bit44if the drill string14includes a bent sub42. In further alternative embodiments, an asymmetrical drill bit or a deflection shoe may be used to steer the downhole tool.

Continuing withFIG.1, the drill rig22rotates the downhole tool34by rotating the drill string14. When steering, the drill rig22pushes the drill string14forward without rotation. Once the downhole tool34has been redirected to the direction it needs to bore, the drill string14is continually rotated again. The downhole tool34bores straight in the direction and angle it is facing when the downhole tool34and drill string14are continually rotated.

By its nature, the focus of the drilling activity, the downhole tool34, is out of sight of the operators. The position and orientation of the downhole tool34is traditionally interpreted by the rig and tracker operator28and40using icons and technical data displayed on a user interface. The tracker operator40must be skilled at interpreting the technical data in order to accurately track the downhole tool34. Similarly, the rig operator28must be skilled at interpreting the technical data in order to effectively steer the downhole tool34underground along a desired borepath. The present disclosure is directed to a system that uses augmented reality to assist the tracker operator40in tracking the downhole tool34and the rig operator28in steering the downhole tool34.

With reference toFIG.3, one embodiment of an augmented reality (AR) device46is shown. The AR device46is a head-mounted device, as shown inFIG.2. The head-mounted device is a traditional heads-up-display. In alternative embodiments, the head-mounted device may be attached to a hard hat or an elastic head strap.

The AR device46comprises a camera50supported immediately adjacent a translucent lens52. The camera50comprises a lens54having a field of view. The translucent lens52has a field of view that overlaps the field of view of the camera's lens54. The AR device46further comprises a screen56that is incorporated into the translucent lens52.

Continuing withFIG.3, one or more sensors58are supported on the AR device46. The sensors58comprise one or more of the following: a GNSS receiver, an ultra-wide range beacon, an inclinometer, a compass, an accelerometer, a capacitive sensor, a resistive touch sensor, an elevation sensor, a gyroscope, a magnetometer, time-of-flight sensor, and/ or an altimeter. The sensors58may further comprise other sensors known in the art for use with augmented reality devices.

The camera50, screen56, and sensors58communicate with a controller. The controller may be supported on the AR device46, like the controller60shown inFIG.3. Alternatively, the controller may be supported remotely from the AR device46.

The controller also communicates with the tracker38. In operation, the tracker38gathers information about the downhole tool34, including its position and orientation, and transmits such information to the controller. At the same time, the sensors58measure a position and orientation of the AR device46and transmit such information to the controller. The controller analyzes information from the tracker38and the sensors58and determines a position and orientation of the downhole tool34relative to the AR device46.

Turning toFIGS.4and5, following such analysis by the controller, the controller generates a virtual image of the downhole tool62. The virtual image of the downhole tool62is incorporated into the images captured by the camera50to create a composite image. The controller displays the composite image on the screen56such that the virtual image of the downhole tool62is superimposed within the field of view of the translucent lens52. The virtual image of the downhole tool62may be an actual artistic representation of the downhole tool34, as shown inFIG.4. Alternatively, the virtual image of the downhole tool62may be a circle, as shown inFIG.5A, or a dash, as shown inFIG.5B. In further alternative embodiments, the downhole tool34may be represented by any graphic desired.

An operator wearing the AR device46views the virtual image of the downhole tool62in combination with the operator's surrounding environment. The virtual image of the downhole tool62is positioned on the screen56at its determined position relative to the ground surface26. The position and orientation of the virtual image of the downhole tool62is updated in response to new information from the tracker38or the sensors58. For example, if the tracker38sends information to the controller indicating that the downhole tool34has moved, the controller will move the position of the virtual image of the downhole tool62displayed on the screen56.

The AR device46may be worn by the tracker operator40, as shown inFIG.2. As the tracker operator40tracks the downhole tool34using the tracker38, the operator40may occasionally step away from the tracker38in order to get a realistic perspective of the downhole tool's position and depth. The AR device46may be configured to display the virtual image of the downhole tool62in 2D and 3D. For example, if the AR device46is positioned directly above the downhole tool34, the virtual image of the downhole tool62may appear in 2D, as shown inFIG.5B. As the AR device46is moved away from the downhole tool34, the virtual image of the downhole tool62may be displayed in 3D, as shown inFIGS.4and5A. The controller may be configured to automatically toggle the virtual image of the downhole tool62between 2D and 3D views based on the position of the AR device46relative to the downhole tool34.

The depth of the downhole tool34may be represented by placement of the virtual image of the downhole tool62relative to the ground surface26, as shown for example inFIG.4. The actual depth may also be identified for the tracker operator40on the screen56, as shown for example inFIG.6. Alternatively, the depth may be identified in other indirect ways, such as displaying the virtual image of the downhole tool or its surrounding environment in different colors or patterns that correspond to different depths.

With reference toFIG.5B, when tracking the downhole tool34, the tracker operator40looks for the front and rear null points of the beacon signal34. A method for identifying a location of the front and rear null point is described in U.S. Patent Publication No. 2020/0072983, authored by Cole, et al., the entire contents of which are incorporated herein by reference. A virtual image representing a front null point61and a virtual image representing a rear null point63of the beacon signal34may be displayed for reference on the screen56, as shown inFIG.5B. The null points may be displayed for reference in the various other views described herein, if desired.

Continuing, withFIGS.4-6, the controller may generate virtual images about the operating parameters of the downhole tool34and display such images on the screen56. For example, the controller may display the beacon temperature, the downhole fluid pressure, the downhole tool's roll position, battery life, and the like. A controller located at the operator station30may also transmit information about the drill rig22to the AR device46for display on the screen56. For example, the amount of torque and thrust currently being applied to the drill string14may be displayed for the tracker operator40.

The AR device46may also be worn by the rig operator28. The rig operator28may view the virtual image of the downhole tool62, as shown inFIGS.4and5. Alternatively, the rig operator28may view the downhole tool34as if the operator is positioned immediately behind the downhole tool34within the borehole12, as shown in inFIG.6. Such view is referred to herein as the “drilling view”. A position of the downhole tool34is represented by a virtual image64in the drilling view. A roll position of the downhole tool34is represented by an arrow66incorporated into the virtual image64. The depth of the downhole tool34may be stated on the screen56, as shown for example by the notation6′4″ inFIG.6. A position of the ground surface26relative to the downhole tool34may be displayed by a virtual line67in the drilling view.

Turning back toFIGS.4and5, the controller may also generate and display a virtual image of the actual borepath68created by the downhole tool34during operation. The virtual image of the actual borepath68may include an artistic representation of the drill string14, as shown inFIG.4. Alternatively, the virtual image of the actual borepath68may be a line, as shown inFIG.5. The underground position of the actual borepath is determined using information previously received about the position and orientation of the downhole tool34as it bores underground.

Continuing withFIGS.4and5, the controller may generate and display a virtual image of the planned borepath70. The virtual image of the planned borepath70may be displayed as a plurality of past and upcoming waypoints72and74. The waypoints72and74may be displayed along the ground surface26overlaying the planned borepath and relative to the ground surface26at their desired depths, as shown for example, by the upcoming waypoint74. Past waypoints72may be connected by a line to represent the actual borepath.

Turning back toFIG.6, only the upcoming waypoints74may be displayed in the drilling view. The upcoming waypoints74may be displayed as a series of rings. The rings provide targets for the rig operator28to steer the downhole tool34towards during operation. The controller may also display a set of steering instructions on the screen56, as shown for example by the instructions76. The steering instructions direct the rig operator28how to steer the downhole tool34towards the next waypoint74.

The position of the planned borepath may be determined prior to starting boring operations. For example, an operator may walk along the ground surface overlaying a desired borepath and take GPS measurements of desired waypoints. A GPS measurement may be taken every 10 feet, for example. Desired depth measurements may be associated with each waypoint. Data gathered for the planned borepath is transmitted to the controller for use in generating the virtual image of the planned borepath70. A method of planning the borepath and generating steering instructions is described in more detail in U.S. Patent Publication No. 2017/0226805, authored by Cole, the contents of which are incorporated herein by reference.

Continuing withFIGS.5and6, the controller may also generate a virtual image of a projected uncorrected borepath78. The projected uncorrected borepath represents the direction the downhole tool34will bore, based on its current position and orientation, if not steered differently. The virtual image of the projected uncorrected borepath78is a set of dashed lines inFIG.5and a straight line inFIG.6. In alternative embodiments, the virtual image of the projected uncorrected borepath may be a runway, series of rings, or the like.

During boring operations, the downhole tool34must be steered around or away from any underground obstacles, such as a utility line80shown inFIG.9. The controller may generate and display a virtual image of underground obstacles82, such as the utility line shown inFIGS.6and12. Underground obstacles may be located and displayed in accordance with the methods described in more detail later herein. If the planned borepath is nearing an underground obstacle, the upcoming waypoints74may be highlighted in some fashion, as shown inFIG.6.

The controller may be configured to alert the tracker or rig operator40and28when approaching an obstacle. For example, a warning sign array appear on the screen56. Such warning may state the current distance between the obstacle and the downhole tool34. The controller may also be configured to produce an audible alarm when approaching an underground obstacle.

With reference toFIG.7, the underground obstacles may be identified on the screen56using a heat map84. For example, the underground location of existing utility lines may be represented by shading, the ground surface26overlaying the lines in red, as shown by the dense dots86. The shading displayed on the screen56may transition to green in areas farther away from the utility lines, as shown by the less dense dots88. If the downhole tool34needs to bore below or above and exiting utility line, the same shading may be displayed in 3D when analyzing the depth of the tool34.

During operation, the rig operator28and the tracker operator40will want to maintain the downhole tool34within green shaded areas. The heat map84may also be used to assist the rig operator28in maintaining the downhole tool34on the planned borepath. Areas surrounding the planned borepath may be shaded green, while areas farther away from the planned borepath may transition to red.

Turning back toFIG.1, during operation, electromagnetic signals from nearby objects may interfere with the tracker's ability to detect the beacon signal36. The degree of interference may vary across different frequencies and at different locations along the desired borepath. Thus, it may be desirable for the tracker operator40to detect the beacon signal36on varying frequencies throughout the duration of the drilling operation. A method for determining the preferred frequencies the beacon signal36should be tuned to during the drilling operation is described in U.S. Pat. No. 9,971,013, issues to Cole et al., the contents of which are incorporated herein by reference.

The controller may be programmed with the preferred frequencies for the beacon and tracker38along a planned borepath and display on the preferred frequency on the screen56. As the tracker operator40continually tracks the downhole tool34, the controller may notify the operator40if the frequency needs to be modified. The controller may also highlight portions of the planned borepath displayed on the screen56that have high interference.

When tracking the downhole tool34, the detected position of the downhole tool34may not always be 100% accurate. For example, any interference with the beacon signal36may cause the identified position of the downhole tool34to be slightly inaccurate. The accuracy may vary over the different available frequencies based on the amount of interference present at each frequency. The controller may calculate the maximum positional deviation for the downhole tool34at each frequency using the given measurement uncertainty of the tracker38.

The controller may subsequently indicate a maximum positional deviation of the virtual image of the downhole tool62on the screen56. The controller may generate the virtual image of the downhole tool62at its detected position on each available frequency. Each image may be simultaneously displayed on the screen56so that the tracker operator40can analyze the maximum positional deviation for the downhole tool34at each available frequency. The tracker operator40may continue tracking the downhole tool34using the frequency that results in the lowest maximum positional deviation.

If any uncertainty exists as to the location of an underground obstacle, the maximum positional deviation of the virtual image of underground obstacles82may be displayed on the screen56. Positional uncertainty of the virtual images of the downhole tool62and the obstacles82may be displayed using visual indicators, such as color or graphs.

The proximity between the maximum positional deviation of the virtual image of the downhole tool62and the virtual image of underground obstacles82is monitored by the controller and the tracker operator40. The controller may warn the operator40or28on the screen56and audibly if the positional deviations for the virtual images of the downhole tool62and the obstacles82overlap or are projected to overlap. Color may also be used to indicate any overlap.

Turning back toFIG.1, after the borehole12is created, a pipeline is normally installed within the borehole12. The pipeline usually has a larger diameter than that of the borehole12. The controller may generate and display a virtual image of the pipeline to be installed within the borehole using the known measurements of such pipeline. Displaying the pipeline allows the operators40and28to visualize the location of the pipeline compared to any underground obstacles. The pipeline to be installed may be displayed when planning the borepath in order to compare the size of the pipeline to the size of existing underground obstacles. The pipeline may also be viewed while drilling by superimposing the virtual image of the pipeline over the virtual image of the actual borepath68.

Turning toFIG.8, the controller may generate virtual images of information about the operational parameters of the drill rig22. Such information may be transmitted to the controller from a controller included in the operator station30. Such information may include, for example, the drilling fluid tank level, fuel level, engine operational parameters, carriage operational parameters, pipe box parameters, numbers of pipes added to the drill string14, distance drilled, and the like. Such information may be displayed on the screen56if the operator is near the drilling rig22. For example, such information may be displayed for a rig operator28when sitting at the operator station30or walking around the drill rig22.

Various drilling parameters may be displayed in response to the corresponding component of the drill rig22coming into the field of view of the AR device46. The controller may use measurements from a time-of-flight sensor included in the sensors58to determine which parameters to display. The time-of-flight sensor may be configured to identify different features and components of the drill rig22. For example, a pipe box90is supported on the drill rig22within the rig operator's field of view, as shown inFIG.8. A virtual image of parameters for the pipe box92may be displayed for the rig operator28if the rig operator looks at the pipe box90. The virtual image92shown inFIG.8indicates that the pipe sections32are currently being removed from the second row of the pipe box90and sixteen pipe sections32remain in the pipe box90.

Any issues with the drill rig22may pop up on the screen56as the issue arises. For example, if the drill rig22is almost out of drilling fluid, a warning may pop-up on the screen56. As another example, an alert may pop-up on the screen56notifying the rig operator28that the engine needs maintenance.

The AR device46may be controlled using hand gestures. For example, the screen56may cycle through various possible information using taps, bumps, or waves. The sensors58may include ultrasonic sensors configured to recognize the hand gestures. For example, an accelerometer may sense taps or bumps to the AR device46, or capacitive or resistive touch sensors may sense pressure. Alternatively, the camera may sense motion immediately adjacent the AR device46. Buttons may also be included on the AR device46in order to cycle through information displayed on the screen56.

The same type of gestures may also be used to control the drill rig22. The controller in communication with the AR device46may communicate with the controller at the operator's station30. Using such communication, functions traditionally controlled by buttons, switches, or a touch screen in the operator station30may be controlled by the AR device46. However, extra confirmation may be required to prevent unintentional operations.

The AR device46may utilize a dead-reckoning system to locate the position of the downhole tool34rather than using data gathered by the tracker38. In a dead-reckoning system, the downhole tool34may include a plurality of sensors, such as a gyro, magnetometer, accelerometer, and the like. Data measured by the sensors may be transmitted to the controller in communication with the AR device46. The controller may analyze such information and determine a position of the downhole tool34relative to the AR device46.

Various combinations of the virtual images of the actual borepath68, planned borepath70, projected uncorrected borepath78, downhole tool62, underground obstacles82, and other parameters may be displayed in juxtaposition with one another on the screen56, as shown inFIGS.4-6. The operator40or28may select the combinations or views of the virtual images to be displayed, as desired. The virtual images may be displayed in 2D or 3D, depending on the position of the AR device46relative to the actual displayed items. Likewise, the displayed items may be modified on the screen56as the operator moves the AR device46. For example, when looking atFIG.4, if the operator moves his head to the right, more of the upcoming waypoints74may appear.

With reference toFIG.9, another embodiment of an AR device100is shown. The AR device100may comprise a computer having a camera and a display screen, such as a laptop, tablet or smartphone. A tablet102is shown inFIG.9. The tablet102has a camera and a screen. The camera includes a lens having a first field of view. Images captured by the camera within the field of view of the lens are displayed on the screen. The tablet102includes the same controller and one or more sensors58used with the AR device46. The controller creates the same virtual images displayed on the AR device's screen56and superimposes the virtual images over the images displayed on the tablet's screen. In operation, the tracker operator40, for example, may occasionally hold the tablet102above ground surface26overlaying the downhole tool34to see the virtual image of the downhole tool62, rather than wear the AR device46.

With reference toFIG.10, another embodiment of an AR device200is shown. The AR device200comprises a camera202in communication with a remote screen204. The camera202is supported on the front of the operator station30facing the desired borepath. The camera202transmits images of the above-ground operations to the screen204, as shown inFIG.11. The controller and sensors58are included in the operator station30and communicate with the camera202and the screen204. The controller generates and superimposes the previously described virtual images on the images displayed on the screen204, as shown inFIG.12. During operation, the rig operator28may manipulate the view shown on the screen204in order to see different views of the virtual images relative to the ground surface26.

The AR devices46,100, or200may be configured to allow for a virtual fly-by of the drilling operation and planned borepath. The fly-by could be used to view virtual images of upcoming waypoints74and underground obstacles82. The virtual fly-by may be generated by overlaying virtual images of waypoints74and underground obstacles82over aerial maps that have been downloaded to the corresponding controllers.

The AR devices46,100, or200may also be used for locating and mapping the underground obstacles, such as the utility line80shown inFIG.9. In locating operations, a locator operator locates the position of an underground obstacle in three dimensions using a portable, above-ground locating device. The locating device may look similar to the tracker38, shown inFIG.1.

The locator uses one or more antennas to detect active or passive electromagnetic signals emitted from an underground obstacle. Some underground obstacles, like a gas line, do not naturally emit a detectable signal. In such case, a transmitter may be coupled to the obstacle to cause it to emit an electromagnetic field having a circular field shape. The locator operator subsequently maneuvers the locator above the obstacle to locate its position and depth.

A positioning system, such as high accuracy GPS, is used to determine the position of the locating device in 3D space upon detection of the underground obstacle. The position of the locating device in combination with the data detected by the locating device is used to produce maps or models of 3D locations of the underground obstacles. The data obtained from the locator and the positioning system may be transmitted to the controller used with the AR devices46,100, and200. The controller uses the data to generate and display the virtual images of the underground obstacles82on the screen56or204, as shown inFIGS.6and12.

In some cases, the approximate location of underground obstacles may already be known. Such location may be recorded in a vector data format, such as a shapefile. The file may contain the GPS location, line type, material, age, or depth of the obstacle. Before starting a locating operation, such information may be transmitted to the controller in communication with the AR device46,100, or200. The controller may use the information to generate and display the virtual images of the underground obstacles82on the screen56or204. If the depth is included in the file, the depth may be displayed in 2D or 3D, as described above. Displaying such information on the AR device46,100, or200for the locating operator provides reference points for the operator during the locating operation.

The locating operator may determine that the actual location of the underground obstacles varies from the location identified in the vector data. If so, the controller will either update the location of the virtual image of the underground obstacle82or generate a new virtual image of the underground obstacle82using the data received from the locator. The controller may be configured to display the old virtual image in juxtaposition with the new virtual image of the underground obstacle82, if desired. Alternatively, the old virtual image of the underground obstacle82may be removed from the screen56or204. The controller may also be configured to automatically reposition the old virtual image of the underground obstacle82in response to a portion of the obstacle being located by the locating operator at a different position.

The controller may generate and display virtual images of other underground obstacles that may create electromagnetic interference, but will not necessarily impede the path of the downhole tool34. For example, a virtual image of a nearby buried electric line may be displayed on the screen56or204. Displaying such images helps the tracker operator40to be aware of any interference that may affect the operator's ability to accurately track the downhole tool34. The maximum positional deviation of the virtual image of the downhole tool62or underground obstacles may increase in areas with increased interference.

The controller may display the frequency at which different underground obstacles were located on the screen56or204. Different frequencies may be represented using different colors. For example, a gas line located at a frequency of 3.14 kHz may be highlighted blue. A single underground obstacle may also be detected by the locator at multiple frequencies along the length of the obstacle. In such case, the obstacle may be displayed having differently colored sections, each color corresponding to a different frequency.

A heat map of an underground obstacle, like the utility line80shown inFIG.9, may also be created during the locate operation. The heat map may be similar to the heat map84shown inFIG.7. As discussed above, an underground obstacle will emit an electromagnetic signal at a known frequency. The signal strength and direction of this electromagnetic signal may be detected by the locator. The signal's position in space is also recorded. Position may be recorded by the locator's onboard GNSS system. Alternatively, the AR device46may record the relative location of the locator with the camera50and/or sensors58. Once a signal is received by the locator, along with corresponding location data, the controller may display a virtual colored marker on the screen56or204.

The marker may provide an indication as to the signal strength and direction detected. As the locating operator walks forward or swings the locator from side to side, additional markers will be added to the screen. As new markers are logged, existing markers may be updated to reflect the additional data received. For example, a previously logged marker may change from green to yellow if newly added markers have a higher signal strength than the previously logged marker. The markers may also comprise a direction indicator, such as an arrow, to direct the locating operator to a position directly above the underground obstacle to be located. The locating operator may also look back at previously placed markers to ensure that he or she is consistently locating the underground obstacle.

Additional parameters related to locating the underground obstacle, may also be displayed on the screen56or204. Such parameters may include the ground speed, mode used, antenna selection, width of swing, receiver estimated depth, receiver estimated current, transmitter connection type (direct, clamp, induction, etc.), transmitter current, transmitter voltage, transmitter load resistance/impedance, age of prior locate information, name of individual or entity who previously located the obstacle, and/or models or brands of locating equipment.

The data collected during the locating operation may be stored and uploaded for future use or display. For example, a virtual underground environment can be created to visualize one or more utilities and overlay them on a street map or other drawing. Such data could be used to plan any operation that requires the ground be disturbed. Visualizing the locating infrastructure and proposed excavation geometry would aid in planning the job and make the data easy to show to non-technical personnel.

In alternative embodiments, a virtual reality system may be used in place of the augmented reality system described herein. In such system, the controller would generate a virtual image of the operator's entire surroundings, and not just information related to the drilling operation. The augment or virtual reality system may be utilized for asset management and real-time observation by training personnel. Offboard or offsite trainees or managers could be allowed to view the same screen that is viewable to the tracker or rig operator40or28.

Changes may be made in the construction, operation and arrangement of the various parts, elements, steps and procedures described herein without departing from the spirit and scope of the invention.