Patent Description:
Today, human welders are often expert welders with many years of experience in a particular type of welding. When a critical part or structure fails (e.g., in a plant, a factory, or in the field), a particular type of expert welder may be needed quickly to fix the problem. However, such an expert welder may not be readily available at the location, causing significant down time and/or a safety problem until the expert welder can travel to the location to fix the problem. There is a need to have access to such an expert welder to fix a critical part or structure in a timelier manner. Furthermore, certain areas where a critical part or structure is located may not be very hospitable to human beings. For example, environmental factors such as heat, humidity, chemicals, or radiation may pose a problem for a human welder. There is a need to have remote access to such inhospitable areas (e.g., Low Earth Orbit space). Remote and welding vision equipment is e.g. described in <CIT> (disclosing the preamble of claims <NUM> and <NUM> respectively), in <CIT> or in <CIT>.

In order to improve welding quality, especially by incorporating expert welder knowledge, a system for long distance, real time, remote welding according to the present invention is defined in claim <NUM>, and a method of remotely controlling a robotic welding system over a long distance in real time according to the present invention is defined in claim <NUM>. Preferred embodiments of the present invention are defined in the subclaims. Embodiments of the present invention include systems and methods related to remote welding. A human welder located in one place can remotely perform a welding operation in another place. For example, an available expert welder located in a first place, who is an expert in a particular type of welding associated with nuclear reactors, can fix a nuclear reactor structure in a second place which is miles away. In one embodiment, an expert welder at a first location can use a welding torch/gun to control a robot holding a similar welding torch/gun at a remote location. The welding torch/gun held by the expert welder is inactive but has accelerometers or gyros that indicate the position and orientation of the welding torch/gun in three-dimensional space. As the expert welder moves the torch/gun, the torch/gun held by the robot at the remote location moves in the same way to actively create a weld. The expert welder has a view (video and sound) of the welding environment (workpiece, torch/gun, weld puddle, etc.) at the remote location (e.g., via a head-mounted display). There is a small roundtrip communication latency (e.g., about <NUM>) between the first location and the remote location such that the expert welder does not get cyber sick (i.e., it will appear to the expert welder that what he is doing with the torch/gun at the first location is happening at the exact same time at the remote location). In this way, remote welding over longer distances can be accomplished in real time. One key to effectively welding remotely over longer distances is an ultra-low latency communication network between the two locations. Even though arc welding is mainly discussed herein, certain embodiments may be applicable to other types of welding as well such as, for example, electron beam welding or laser beam welding. The term "arc" as used herein refers to a plasma arc and the term "beam" as used herein may refer to an electron beam or a laser beam. The first controller is configured to capture the video of at least the arc or the beam between the workpiece and the welding torch during the actual welding operation via the camera observing through the auto-darkening filter. In one embodiment, the robotic welding system includes a microphone. The first controller is configured to capture audio of at least the arc or the beam between the workpiece and the welding torch during the actual welding operation via the microphone. The ultra-low-latency communication network is configured to provide communication of the audio from the first controller at the remote welding site to the second controller at the local site to be observed by the human welder via the head-mounted display device at the local site. In one embodiment, the mock welding tool includes one or more sensors configured to monitor at least one of a position and an orientation of the mock welding tool and provide corresponding position and orientation signals to the second controller for tracking the mock welding tool in three-dimensional space. In another embodiment, the one or more sensors are external to the mock welding tool.

In one embodiment, the method includes determining when a tip of the welding torch of the robotic welding system is not at a proper distance from the workpiece during the actual welding operation at the remote welding site. A feedback signal is transmitted from the remote welding site to the local site, over the ultra-low-latency communication network, when the tip of the welding torch is not at the proper distance. A haptic response is generated within the mock welding tool at the local site in response to the feedback signal. In one embodiment, the method includes determining when the welding torch of the robotic welding system is not at a proper angle with respect to the workpiece during the actual welding operation at the remote welding site. A feedback signal is transmitted from the remote welding site to the local site, over the ultra-low-latency communication network, when the welding torch is not at the proper angle. A haptic response is generated within the mock welding tool at the local site in response to the feedback signal. In one embodiment, the method includes determining when the welding torch of the robotic welding system is not moving at a proper travel speed with respect to the workpiece during the actual welding operation at the remote welding site. A feedback signal is transmitted from the remote welding site to the local site, over the ultra-low-latency communication network, when the welding torch is not moving at the proper travel speed. A haptic response is generated within the mock welding tool at the local site in response to the feedback signal. The audio from the remote welding site is transmitted to the local site over the ultra-low-latency communication network and played to the human welder at the local site in real time as the human welder moves the mock welding tool during the actual welding operation. In one embodiment, the method includes employing at least one of predictive, interpolative, or extrapolative techniques, near an edge of the ultra-low-latency communication network near the remote welding site, to anticipate the control parameters corresponding to the movements and the control of the mock welding tool over a next millisecond or more.

Numerous aspects of the general inventive concepts will become readily apparent from the following detailed description of exemplary embodiments, and from the accompanying drawings.

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate various embodiments of the present invention. It will be appreciated that the illustrated element boundaries (e.g., boxes, groups of boxes, or other shapes) in the figures represent one embodiment of boundaries. In some embodiments, one element may be designed as multiple elements or that multiple elements may be designed as one element. In some embodiments, an element shown as an internal component of another element may be implemented as an external component and vice versa. Furthermore, elements may not be drawn to scale.

Embodiments of systems and methods for performing real time, long distance, remote welding are disclosed. In one embodiment, a mock welding tool is used by a human welder at a local site to remotely control an actual welding torch/gun at a remote location to perform an actual welding operation while remotely observing (e.g., visually, audibly, and tactily) the actual welding operation. The welding torch/gun at the remote location responds immediately to the human welder's movements of the mock welding tool such that the human welder does not get "cyber sick". That is, it will appear to the human welder that what he/she is doing with the mock welding tool at the local site is happening at the exact same time with the actual welding torch/gun at the remote location. The communication latency between the local site and the remote welding site is short enough such that the human welder does not perceive any visual, audio, or tactile delay that would result in the human welder feeling disoriented or confused.

The examples and figures herein are illustrative only and are not meant to limit the subject invention, which is measured by the scope and spirit of the claims. Referring now to the drawings, wherein the showings are for the purpose of illustrating exemplary embodiments of the subject invention only and not for the purpose of limiting same, <FIG> illustrates one embodiment of a system <NUM> for performing long distance, real time, remote welding. The terms "gun" and "torch" are used interchangeably herein with respect to welding.

Referring to <FIG>, the system <NUM> includes a robotic welding system <NUM>, at a remote welding site, and a simulated welding system <NUM>, at a local site, which communicate with each other over an ultra-low-latency (ULL) communication network <NUM>. In accordance with one embodiment, the round-trip communication latency between the robotic welding system <NUM> and the simulated welding system <NUM> over the ULL communication network <NUM> is between <NUM> milliseconds and <NUM> milliseconds, and a straight-line distance (as the crow flies) between the remote welding site and the local site is at least <NUM> kilometers. As used herein, the round-trip communication latency is a measure of time it takes a packet of data to travel from the simulated welding system, at the local site, to the robotic welding system, at the remote welding site, and back again. When a human interacts with a technical system, the interaction appears to be innate and instinctive to the human only when the feedback between the human and the system corresponds to the reaction time of the human. For example, human auditory reaction time is about <NUM> milliseconds, human visual reaction time is about <NUM> milliseconds, and human tactile reaction time is about one millisecond.

In general, communication latency is limited by the components and operation of the communication network between the local site and the remote site. Ultimately, however, if a communication network contributed no latency (i.e., an ideal network), the communication latency would still be limited by the speed of light which is about <NUM>,<NUM> kilometers per second. That is, light can travel about <NUM> kilometers in one millisecond in a vacuum. Therefore, the distance between two sites would have to be less than <NUM> kilometers to achieve a round-trip communication latency of one millisecond. The greater the communication latency due to components of the communication network between the two sites, the closer the two sites have to be to maintain the round-trip communication latency of, for example, one millisecond.

A communication network that is capable of achieving ultra-low-latencies over long distances will use advanced technologies. For example, in one embodiment, the ULL communication network <NUM> may employ one or more of passive optical components, dark fiber, dispersion compensation modules, not-forward error correction transponders, software-defined networks, and network functions virtualizations techniques. Additional advanced technologies that the ULL communication network <NUM> may employ include three-dimensional integrated circuit chips, three-dimensional integrated circuit chip-stacks, optical waveguides embedded in circuit boards, optical integrated transceivers within chip-stacks, and fully wireless chip-to-chip interconnectivity of circuit boards within a chassis. Other advanced technologies are possible as well, in accordance with other embodiments. In accordance with one embodiment, such advanced technologies may be employed in a network of Low Earth Orbit (LEO) satellites which are part of a ULL communication network supporting long distance, real time, remote welding.

<FIG> illustrates an embodiment of the robotic welding system <NUM> of the system <NUM> of <FIG>. The robotic welding system <NUM> is configured to be set up at a remote welding site to perform an actual welding operation via remote control. The robotic welding system <NUM> includes a welding torch <NUM>, a video camera (or other imaging sensor) <NUM>, an auto-darkening filter <NUM>, a microphone (or other audio sensor) <NUM>, and a controller <NUM>. In one embodiment, the welding torch <NUM> is attached to an arm of the robotic welding system <NUM>. Movement of the arm (and thus the welding torch <NUM>) is commanded by the controller <NUM>. For example, in one embodiment, the arm may provide for at least six degrees of freedom of movement of the welding torch <NUM> under the control of the controller <NUM>. However, in accordance with one embodiment, the controller <NUM> controls movement of the arm (and thus the welding torch <NUM>) in response to control parameters received from the simulated welding system <NUM> over the ULL communication network <NUM> as discussed later herein.

The controller <NUM> is configured to control the robotic welding system <NUM> and capture video and audio, respectively, via the camera <NUM> and the microphone <NUM>, of a plasma arc, an electron beam, or a laser beam between a workpiece <NUM> and the welding torch <NUM> during an actual welding operation. In one embodiment, the video is captured by the camera <NUM> observing through the auto-darkening filter <NUM>. The camera <NUM>, the auto-darkening filter <NUM>, and the microphone <NUM> may be directly or indirectly attached to the arm of the robotic welding system <NUM>, in accordance with one embodiment, such that the camera <NUM>, the auto-darkening filter <NUM>, and the microphone <NUM> move to follow the point where the torch <NUM> contacts the workpiece <NUM> (e.g., via <NUM>-axes of motion, x and y, relative to a plane of the workpiece <NUM>). In accordance with another embodiment, the camera <NUM>, the auto-darkening filter <NUM>, and the microphone <NUM> are controlled in another manner to follow the point where the torch <NUM> contacts the workpiece <NUM> (e.g., via a separate motion controller tracking the motion of the torch <NUM> via input from the camera <NUM>). The video and audio are provided, respectively, from the camera <NUM> and the microphone <NUM> to the controller <NUM>. The controller <NUM> is configured to transmit the video and audio in real time to the simulated welding system <NUM> over the ULL communication network <NUM>.

<FIG> illustrates an embodiment of the simulated welding system <NUM> of the system <NUM> of <FIG>. The simulated welding system <NUM> is configured to be set up at a local site to perform a simulated welding operation that remotely controls the actual welding operation at the remote welding site over the ULL communication network <NUM>. The simulated welding system <NUM> includes a mock welding tool <NUM>, a head-mounted display device <NUM>, and a controller <NUM>. The head-mounted display device <NUM> is configured to be worn by a human welder (e.g., the local expert welder in <FIG>) to observe the video and audio from the robotic welding system <NUM> at the remote welding site. The mock welding tool <NUM> is configured to remotely control the welding torch <NUM> in response to the human welder holding and moving the mock welding tool <NUM> at the local site while observing the video and the audio. For example, the human welder may move the mock welding tool <NUM> along a simulated workpiece or coupon <NUM> which simulates the real-world workpiece <NUM> at the remote welding site. The controller <NUM> is configured to control the simulated welding system <NUM> and generate control parameters while tracking positions and movements of the mock welding tool <NUM> in three-dimensional space. For example, in one embodiment, the controller <NUM> includes a spatial tracking technology for tracking position and orientation of the mock welding tool <NUM> over time. The spatial tracking technology may be, for example, magnetically-based in accordance with one embodiment, or inertially-based, in accordance with another embodiment. Other types of spatial tracking technologies are possible as well, in accordance with other embodiments.

Referring again to <FIG>, the ULL communication network <NUM> is configured to provide communication of the video, the audio, and the control parameters between the controller <NUM> at the local site and the controller <NUM> at the remote welding site in real time. Again, in accordance with one embodiment, the round-trip communication latency between the robotic welding system <NUM> and the simulated welding system <NUM> over the ULL communication network <NUM> is between <NUM> milliseconds and <NUM> milliseconds, and a straight-line distance (as the crow flies) between the remote welding site and the local site is at least <NUM> kilometers. In this manner, the robotic welding system <NUM> is configured to form an actual weld on the workpiece <NUM> at the remote welding site, during an actual welding operation, via remote robotic control of the welding torch <NUM> via the mock welding tool <NUM> in response to the control parameters. During the actual welding operation, the human welder moves the mock welding tool <NUM> and observes the video and the audio via the head-mounted display device <NUM>. As a result, the human welder has the experience of actually welding at the remote welding site without suffering adverse effects such as cyber sickness.

In accordance with one embodiment, the controller <NUM> is configured to perform a latency test to determine the round-trip communication latency between the controller <NUM> at the remote welding site and the controller <NUM> at the local site along a path through the ULL communication network. In accordance with another embodiment, the controller <NUM> is configured to perform a latency test to determine the round-trip communication latency between the controller <NUM> at the remote welding site and the controller <NUM> at the local site along a path through the ULL communication network. Such a latency test may include sending time-stamped packets of data back and forth between the controller <NUM> and the controller <NUM> and calculating an average latency across the packets. In this manner, the desired or required latency between the two sites can be verified before attempting to perform the remote welding operation. Other types of latency tests are possible as well, in accordance with other embodiments.

A ULL communication network may provide for more than one path through the ULL communication network. Different paths may have different latencies. The controller can have prior knowledge of the various paths through the ULL communication network, in accordance with one embodiment. When a latency test fails (i.e., the latency is determined to be too long), the controller that performed the latency test can perform a subsequent latency test along a different path through the ULL communication network. Such latency testing may continue until an acceptable path is determined, or until all paths are exhausted.

<FIG> illustrates an embodiment of the head-mounted display device <NUM> and an embodiment of the mock welding tool <NUM> of the simulated welding system <NUM> of <FIG>. The mock welding tool <NUM> includes a handle <NUM>, a tip <NUM>, a gooseneck <NUM>, and a trigger <NUM>. The mock welding tool <NUM> also includes inertial sensors <NUM> (e.g., accelerometers or gyros) which are used to generate signals for tracking the position, orientation, and motion of the mock welding tool <NUM> in three-dimensional space. In accordance with another embodiment, the simulated welding system <NUM> includes sensors that are external to the mock welding tool <NUM> to track the mock welding tool <NUM> in three-dimensional space. Such external sensors may include, for example, laser devices or camera devices to sense the position, orientation, and/or movement of the mock welding tool <NUM>. In one embodiment, a pendant is attached to the mock welding tool <NUM> which is sensed by an array of sensors located above at the local site.

The mock welding tool <NUM> also includes a wireless transmitter <NUM> for wirelessly communicating with the controller <NUM>. For example, in one embodiment, data representing the position of the trigger <NUM> and the signals for tracking the position, orientation, and motion of the mock welding tool <NUM> is wirelessly communicated to the controller via the wireless transmitter <NUM>. In another embodiment, the data is communicated in a wired manner from the mock welding tool <NUM> to the controller <NUM>. The controller <NUM> generates control parameters based on the data from the mock welding tool <NUM> which is used to remotely control the robotic welding system <NUM>. The mock welding tool <NUM> includes a processor (not shown) and memory (not shown) configured to collect the signals and generate the data, in accordance with one embodiment.

In one embodiment, the head-mounted display device <NUM> includes a welding helmet or mask <NUM> and two displays <NUM> (e.g., two high-contrast SVGA 3D OLED micro-displays) configured to display fluid and real time full-motion video from the remote welding site. The head-mounted display device <NUM> also includes two speakers <NUM> configured to play audio from the remote welding site in real time. In one embodiment, the head-mounted display device <NUM> interfaces in a wired manner with the controller <NUM> to receive the video and audio. In another embodiment, the interface may be wireless via a wireless transceiver device <NUM>.

In one embodiment, the head-mounted display device <NUM> is tracked in three-dimensional space (e.g., similar to how the mock welding tool <NUM> is tracked via sensors and the controller <NUM>). Tracking of the head-mounted display device <NUM> may be used to control the position of the camera <NUM>, the microphone <NUM>, and the auto-darkening filter <NUM> at the remote welding site. In this manner, as the human welder at the local site moves his/her head, the camera <NUM>, the microphone <NUM>, and the auto-darkening filter <NUM> at the remote welding site follows the movement (similar to how the welding torch <NUM> follows the mock welding tool <NUM>). The camera <NUM>, the microphone <NUM>, and the auto-darkening filter <NUM> at the remote welding site may be operatively connected to, for example, a separate servomechanism system, in accordance with one embodiment.

<FIG> illustrates a flowchart of an embodiment of a method <NUM> of remotely controlling a robotic welding system over a long distance in real time (e.g., using the system <NUM> of <FIG>). At block <NUM>, movements and control of a mock welding tool <NUM> are tracked as the mock welding tool <NUM> is operated by a human welder at a local site. Control parameters are generated which correspond to the movements and the control of the mock welding tool <NUM>. For example, in one embodiment, the control parameters are structured to communicate, to the robotic welding system <NUM>, how to operate and control the welding torch <NUM> during an actual welding operation. The mock welding tool <NUM> may simulate characteristics (e.g., the gooseneck and the tip) of the actual welding torch <NUM>, in accordance with one embodiment.

At block <NUM>, the control parameters are transmitted from the local site to the robotic welding system <NUM> at the remote welding site over the ULL communication network <NUM>. The round-trip communication latency between the local site and the remote welding site over the ULL communication network <NUM> is between <NUM> milliseconds and <NUM> milliseconds, and a straight-line distance (as the crow flies) between the local site and the remote welding site is at least <NUM> kilometers, in accordance with one embodiment. In accordance with another embodiment, the round-trip communication latency between the local site and the remote welding site over the ULL communication network <NUM> is between <NUM> milliseconds and <NUM> milliseconds, and a straight-line distance (as the crow flies) between the local site and the remote welding site is at least <NUM> kilometers. In still another embodiment, the round-trip communication latency between the local site and the remote welding site over the ULL communication network <NUM> is between <NUM> milliseconds and <NUM> milliseconds, and a straight-line distance (as the crow flies) between the local site and the remote welding site is at least <NUM> kilometers.

At block <NUM>, an actual welding operation of the robotic welding system <NUM> is controlled at the remote welding site to form an actual weld on the workpiece <NUM> via remote robotic control of the robotic welding system <NUM> in response to the control parameters. The welding torch <NUM> of the robotic welding system <NUM> follows the movements and the control of the mock welding tool <NUM> operated by the human welder at the local site in real time. In accordance with one embodiment, the movements of the mock welding tool <NUM> operated by the human welder are along the simulated workpiece or coupon <NUM> at the local site. The simulated workpiece <NUM> may simulate the workpiece <NUM> at the remote welding site, in accordance with one embodiment.

During the actual welding operation, video and audio of at least an arc (i.e., a plasma arc) or a beam (i.e. an electron beam or a laser beam) formed between the workpiece <NUM> and the welding torch <NUM> (and/or of a region surrounding the arc or beam which includes at least a portion of the welding torch <NUM> and the workpiece <NUM>) are captured at the remote welding site and transmitted to the local site over the ULL communication network <NUM>. The video is displayed and the audio is played to the human welder at the local site in real time as the human welder moves the mock welding tool during the actual welding operation.

In one embodiment, a determination is made as to when the tip of the welding torch <NUM> of the robotic welding system <NUM> is not at a proper or specified distance from the workpiece during the actual welding operation at the remote welding site. Methods of making such a determination are well known in the art. A feedback signal is generated and transmitted (e.g., by the controller <NUM>) from the remote welding site to the local site, over the ULL communication network <NUM>, when the tip of the welding torch <NUM> is not at the specified distance from the workpiece. A haptic response is generated within the mock welding tool <NUM> at the local site in response to the feedback signal. The haptic response may be, for example, a vibration generated in the handle of the mock welding tool <NUM> (e.g., via a vibration device within the handle) which can be sensed by the human welder when holding the mock welding tool <NUM>. Other haptic responses are possible as well, in accordance with other embodiments. In this manner, the human welder can respond by adjusting how close the tip of the mock welding tool <NUM> is to the workpiece <NUM> until the haptic response ceases.

In one embodiment, a determination is made as to when the welding torch <NUM> of the robotic welding system <NUM> is not at a proper or specified angle to the workpiece <NUM> during the actual welding operation at the remote welding site. Methods of making such a determination are well known in the art. A feedback signal is generated and transmitted (e.g., by the controller <NUM>) from the remote welding site to the local site, over the ULL communication network <NUM>, when the welding torch <NUM> is not at the specified angle to the workpiece <NUM>. A haptic response is generated within the mock welding tool <NUM> at the local site in response to the feedback signal. Again, the haptic response may be, for example, a vibration generated in the handle of the mock welding tool <NUM> which can be sensed by the human welder when holding the mock welding tool <NUM>. Other haptic responses are possible as well, in accordance with other embodiments. In this manner, the human welder can respond by adjusting the angle of the mock welding tool <NUM> to the workpiece <NUM> until the haptic response ceases.

In one embodiment, a determination is made as to when the welding torch <NUM> of the robotic welding system <NUM> is not moving at a proper or specified travel speed across the workpiece <NUM> during the actual welding operation at the remote welding site. Methods of making such a determination are well known in the art. A feedback signal is generated and transmitted (e.g., by the controller <NUM>) from the remote welding site to the local site, over the ULL communication network <NUM>, when the welding torch <NUM> is not moving at the specified travel speed across the workpiece <NUM>. A haptic response is generated within the mock welding tool <NUM> at the local site in response to the feedback signal. Again, the haptic response may be, for example, a vibration generated in the handle of the mock welding tool <NUM> which can be sensed by the human welder when holding the mock welding tool <NUM>. Other haptic responses are possible as well, in accordance with other embodiments. In this manner, the human welder can respond by adjusting the travel speed of the mock welding tool <NUM> across the workpiece <NUM> until the haptic response ceases.

<FIG> illustrates an example embodiment of the controller <NUM> of the robotic welding system <NUM> of <FIG>, or the controller <NUM> of the simulated welding system <NUM> of <FIG>. The controller includes at least one processor <NUM> (e.g., a central processing unit, a tensor processing unit, a graphics processing unit) which communicates with a number of peripheral devices via bus subsystem <NUM>. These peripheral devices may include a storage subsystem <NUM>, including, for example, a memory subsystem <NUM> and a file storage subsystem <NUM>, user interface input devices <NUM>, user interface output devices <NUM>, and a network interface subsystem <NUM>. The input and output devices allow user interaction with the controller. Network interface subsystem <NUM> provides an interface to outside networks (e.g., the ULL communication network <NUM>) and is coupled to corresponding interface devices in other devices (e.g., the mock welding tool <NUM> and the head-mounted display device <NUM>). In one embodiment, at least one of the processors <NUM> is a tensor processing unit (TPU) which is an application specific integrated circuit (ASIC) created specifically for machine learning. Unlike a graphics processing unit (GPU), a TPU is structured to accommodate a larger volume of reduced precision computations.

In general, use of the term "input device" is intended to include all possible types of devices and ways to input information into the controller or onto a communication network.

In general, use of the term "output device" is intended to include all possible types of devices and ways to output information from the controller to the user or to another machine or computer system.

Storage subsystem <NUM> stores programming and data constructs that provide some or all of the functionality described herein. For example, computer-executable instructions and data are generally executed by processor <NUM> alone or in combination with other processors. The computer-executable instructions and data implementing the functionality of certain embodiments may be stored by file storage subsystem <NUM> in the storage subsystem <NUM>, or in other machines accessible by the processor(s) <NUM>.

Bus subsystem <NUM> provides a mechanism for letting the various components and subsystems of the controller communicate with each other as intended. Although bus subsystem <NUM> is shown schematically as a single bus, alternative embodiments of the bus subsystem may use multiple buses.

The various components of the controller of <FIG> may employ advanced technologies including, for example, three-dimensional integrated circuit chips, three-dimensional integrated circuit chip-stacks, optical waveguides embedded in circuit boards, optical integrated transceivers within chip-stacks, and fully wireless chip-to-chip interconnectivity of circuit boards within a chassis. Other advanced technologies are possible as well, in accordance with other embodiments.

The controller can be of varying types including a workstation, server, computing cluster, blade server, server farm, or any other data processing system or computing device. Due to the ever-changing nature of computing devices and networks, the description of the controller depicted in <FIG> is intended only as a specific example for purposes of illustrating some embodiments. Many other configurations of the controller are possible, having more or fewer components than the controller depicted in <FIG>.

<FIG> illustrates one embodiment of the ULL communication network <NUM> of <FIG>. In one embodiment, the ULL communication network <NUM> includes a first radio frequency wireless network segment <NUM>, a second radio frequency wireless network segment <NUM>, a first optical fiber network segment <NUM>, and a second optical fiber network segment <NUM>. For example, the optical fiber network segments <NUM> and <NUM> exist closer to the edge of the ULL communication network <NUM> (i.e., near the local site and the remote welding site, respectively). The radio frequency wireless network segments <NUM> and <NUM> exist between the optical fiber network segments (e.g., to provide communication over the vast majority of the distance between the local site and the remote welding site). In one embodiment, the ULL communication network <NUM> is a dedicated and private network operating between the local site and the remote welding site.

<FIG> illustrates another embodiment of the ULL communication network <NUM> of <FIG>. In one embodiment, the ULL communication network <NUM> includes a first optical fiber network segment <NUM>, a second optical fiber network segment <NUM>, a first radio frequency wireless network segment <NUM>, a second radio frequency wireless network segment <NUM>, and a Low Earth Orbit (LEO) satellite network segment <NUM>. Again, the optical fiber network segments <NUM> and <NUM> exist closer to the edge of the ULL communication network <NUM> (i.e., near the local site and the remote welding site, respectively). The radio frequency wireless network segments <NUM> and <NUM>, and the LEO satellite network segment <NUM>, exist between the optical fiber network segments (e.g., to provide communication over the vast majority of the distance between the local site and the remote welding site). In one embodiment, the ULL communication network <NUM> is a publicly accessed network operating between the local site and the remote welding site.

In one embodiment, predictive, interpolative, and/or extrapolative techniques are employed near the edge of the ULL communication network <NUM>. Such techniques allow a statistically comparable action to be automatically performed while a command for the actual action is still in transit over the ULL communication network <NUM>. That is, the system <NUM> can anticipate an action to be taken and begin performing that action before the actual command to perform that action reaches, for example, the remote welding site. For example, in response to a human welder having moved the mock welding tool <NUM> with respect to the workpiece <NUM> at the local site over the last <NUM> milliseconds (resulting in corresponding control parameters), the controller <NUM> at the remote welding site (performing predictive, interpolative, and/or extrapolative techniques) can generate control parameters for controlling the welding torch <NUM> of the robotic welding system <NUM> for the next millisecond. In this manner, the latency impact can be diminished, possibly extending the distance over which a local site can operate with a remote welding site.

Claim 1:
A system (<NUM>) for performing long distance, real time, remote welding, the system (<NUM>) comprising:
a robotic welding system (<NUM>), configured to be set up at a remote welding site to perform an actual welding operation, the robotic welding system (<NUM>) including:
a welding torch (<NUM>), a camera (<NUM>), and
a first controller configured to control the robotic welding system (<NUM>) and capture video of at least an arc or a beam between a workpiece (<NUM>) and the welding torch (<NUM>) during the actual welding operation via the camera (<NUM>);
a simulated welding system (<NUM>), configured to be set up at a local site, including:
a head-mounted display device configured to be worn by a human welder to observe at least the video at the local site,
a mock welding tool (<NUM>) configured to remotely control the welding torch (<NUM>) in response to the human welder holding and moving the mock welding tool (<NUM>) at the local site while observing at least the video, and
a second controller configured to control the simulated welding system (<NUM>) and generate control parameters while tracking movements of the mock welding tool (<NUM>);
the system (<NUM>) being characterised by the following:
an ultra-low-latency communication network (<NUM>) configured to provide communication of at least the video and the control parameters between the first controller at the remote welding site and the second controller at the local site, wherein a round-trip communication latency between the first controller and the second controller is between <NUM> milliseconds and <NUM> milliseconds, and wherein a straight-line distance between the remote welding site and the local site is at least <NUM> kilometers,
wherein the robotic welding system (<NUM>) is configured to form an actual weld on the workpiece (<NUM>) at the remote welding site, during the actual welding operation, via remote robotic control of the welding torch (<NUM>) via the mock welding tool (<NUM>) in response to at least the control parameters, and
wherein the robotic welding system (<NUM>) includes an auto-darkening filter (<NUM>), and wherein the first controller is configured to capture the video of at least the arc or the beam between the workpiece (<NUM>) and the welding torch (<NUM>) during the actual welding operation via the camera observing through the auto-darkening filter (<NUM>).