Patent Description:
Conventional systems and methods for simulating joining operations, such as welding, brazing, adhesive bonding, and/or other joining operations, require substantial use of graphical processing units (GPUs) to perform the simulation. In conventional weld simulation systems, the calculations and rendering of the simulated weld bead require substantial computational power, thereby limiting the platforms on which such simulation could be effectively implemented. For instance, mobile devices and/or web browsers could not implement conventional simulation. Furthermore, conventional weld simulation systems and methods are cumbersome, and present difficulties for development of new features and functionality such as complex welding shapes and/or simulation of different welding materials. An example of a conventional system is disclosed in <CIT>.

In conventional weld simulation, the amount of material transferred from the filler (such as the electrode for SMAW welding processes) to the weld bead cannot be accurately modeled. Accurate modeling is important for simulating root passes in multipass welding, because the root pass lays a foundation for the rest of the passes. When there is no separation between welded parts, the root pass fills up too much space. In many cases, the bead geometry can be either too convex or too concave, making the subsequent passes fit poorly. Weaved welds can produce beads that appear too flat. If welding is executed at the same place (i.e., without substantially advancing the torch or welding gun), the weld bead can grow vertically. In some occasion, there may be no transition between the coupon and weld bead. For example, the bead looks like it was glued to the coupon.

In conventional weld simulation, the melting of base workpiece material and/or underlying weld beads is very difficult to accurately simulate. In particular, conventional simulation of GTAW welding process suffers from lack of this capacity. Examples of technical infrastructures of joining operation simulators are disclosed in <CIT> and "<NPL>.

A system to simulate joining operations is disclosed, substantially as illustrated by and described in connection with at least one of the figures, as set forth more completely in the claims.

Systems and methods disclosed herein involve presenting and modeling weld workpieces and weld beads in a simulated, augmented reality training environment. More specifically, the invention refers to a method and a system according to the claims of the present document.

A disclosed system for providing a simulated, augmented-reality, training environment includes: (<NUM>) an operating system that is capable of supporting 3D (three-dimensional) graphics and communications between users of a simulation software, (<NUM>) implementation of mathematical methods and/or algorithms that simulate welding processes in three dimensions, (<NUM>) software that manages a virtual classroom, and (<NUM>) an augmented reality application that simulates a welding process on a real work piece. By using augmented reality techniques, virtual images generated by a computer are overlapped or transposed onto real environments to define and/or create a mixed reality that provides users with a tool for learning different welding techniques. In some examples, the user (e.g., welder, weld trainee) uses a commercial or custom head-mounted display, video glasses, and/or any other user-worn display device. The display or glasses may be integrated into a commercial welding mask, and presents the mixed reality to a user that wears the welding mask.

Disclosed systems and methods provide a simulated visual presentation of weld workpieces (also referred to herein as "coupons") and weld beads that realistically represents the executed weld bead. For example, such systems and methods can reproduce weld bead defects such that a user can visualize the defect and understand how and why it had occurred. As another example, such systems and methods can distinguish between different techniques used to form a weld and represent each weld differently.

Relative to conventional simulation of joining operations, disclosed systems and methods correctly calculate the amount of material transferred from the filler to the workpiece, adjust the concavity or convexity of the bead geometry to improve the accuracy of interaction between different passes, improve the representation of weaved welds, control unnatural growth resulting from lack of movement of the welding torch, provide an accurate transition between the coupon and weld bead, accurately simulate melting of base workpieces and underlying weld beads.

Disclosed systems and methods substantially increase the efficiency of modeling and simulation of joining operations, thereby providing the capability to run on previously-unused mobile devices and/or web browsers having less computational power, and/or to be ported to substantially any platform. Disclosed systems and methods simplify the creation and implementation of later-developed features and functionalities, such as importation of custom welding workpieces and/or modeling of additional variables such as adding workpiece and/or filler materials.

Disclosed systems and methods are modular, in those systems and methods enable localized changes without affecting the whole geometry of the workpiece or welding result. Disclosed systems and methods calculate weld bead geometry based on measured welding parameters in an iterative manner, for individual locations along the weld bead. Filler volume added during a simulation step is added to volume already present (e.g., from the original workpiece, from prior steps, etc.). Disclosed systems use mass-conserving techniques to accurately simulate volumes of deposited material.

Disclosed systems and methods calculate the weld bead cross-section and the weld bead lateral profile separately (e.g., independently), and combine the cross-section and lateral profile calculates to determine a resulting volume. Unlike conventional weld training techniques, disclosed systems and methods calculate and depict both welding bead surfaces and welding bead penetration into the workpiece. Depictions of the welding bead penetration may be shown using cross-sections of the welding bead at any slice location. Disclosed systems example simulation of GTAW welding without filler rod material, which can be particularly useful for simulating welding of thin sheets of metal.

Disclosed computer-implemented systems and methods for simulation of joining materials with or without filler material include: processing circuitry; and a machine readable storage device storing machine readable instructions which, when executed by the processing circuitry, cause the processing circuitry to display a visual simulation of a three-dimensional joining operation within a simulation domain by simulating the simulation domain as a set of interconnected cross-sectional slices.

Some computer-implemented systems and methods further include one or more image sensors configured to capture images of a physical workpiece and a physical joining tool corresponding to the joining operation, in which the instructions cause the processing circuitry to simulate the simulation domain based on the images. In some examples the instructions cause the processing circuitry to represent a joint result for a group of concatenated slices using one or more polygons and associated polygon data. In some such examples, the associated polygon data for the first one of the slices comprises at least one of: an indication of whether the polygon represents a workpiece, a joint filler bead, or auxiliary data; a joint pass number; joint defect data for the first one of the slices; pressure applied; time exposed to source or applied heat input for the first one of the slices.

In some embodiments, the instructions cause the processing circuitry to display a simulated joining material within a visualization of the simulation domain based on a location of the first one of the slices, the one or more polygons, and the associated polygon data. In some examples, the instructions cause the processing circuitry to display a result of the joining as a cross-section of a simulated workpiece and a simulated weld bead based on the simulated welding data for one of the slices corresponding to a location of the cross-section.

In some systems and methods, the instructions cause the processing circuitry to define one or more simulation domains with reference to a workpiece, and to enable simulated joining of one or more joints within each of the simulation domains. In some examples, the instructions cause the processing circuitry to display a result of the joining (e.g., welding) operation for at least one of the slices from any perspective.

In some systems and methods, the instructions cause the processing circuitry to store the simulation data as vector data representative of the joining operation based on the cross-sectional slices. In some such systems and methods, the instructions cause the processing circuitry to render the workpiece and the joint filler bead based on the vector data for the cross-sectional slices. In some embodiments, the instructions cause the processing circuitry to render the workpiece and the joint filler bead by: generating a three-dimensional mesh based on the vector data; and mapping at least one of color information, surface type information, heat affected zone information, heat affected zone strength information, pressure applied, time exposed to source or weld puddle information to the three-dimensional mesh.

In some systems and methods, the instructions cause the processing circuitry to calculate a volume of deposited material and determine the vector data for the cross-sectional slices based on the calculated volume. In some such systems and methods, the instructions cause the processing circuitry to calculate the volume of deposited material based on programmed simulation parameters. In some systems and methods, the instructions cause the processing circuitry to calculate at least one of a surface geometry or a weld bead penetration for the cross-sectional slices, and include the at least one of the surface geometry or the weld bead penetration in the vector data for the cross-sectional slices.

In some systems and methods, the instructions cause the processing circuitry to: define a weld joint within the simulation domain; define respective slice reference frames for the slices based on a joint reference frame of the weld joint; simulate the joining (e.g., welding) operation for the slices; and projecting simulation data from the slices onto the weld joint to visually simulate a weld bead resulting from the joining (e.g., welding) operation. In some embodiments, the instructions cause the processing circuitry to simulate the joining (e.g., welding) operation for each of the slices by determining a plurality of control points in each of the slices based on the joint parameters. In some such embodiments, the welding parameters include at least one of: weld bead width, weld bead height, weld bead convexity or concavity, reinforcement depth, penetration depth, reinforced area, penetrated area, or dilution factor.

In some embodiments, the instructions cause the processing circuitry to define one or more sequences within one or multiple simulation domains simultaneously or one after the other with in a working piece. In some systems and methods, the instructions cause two or more processors to define the same workpiece and simulate separate joining (e.g., welding) operations on different simulation domains simultaneously. In some systems and methods, the instructions are configured to cause the processing circuitry to display the visual simulation as a mixed reality display or an augmented reality display, in which only information associated with the simulation domain is rendered for display over a captured image. In some embodiments, the instructions cause the processing circuitry to simulate the cross-sectional slices as two-dimensional cross-sections of a joining operation being simulated.

Disclosed computer-implemented systems and methods for simulation of joining materials, with or without filler material, include: processing circuitry; and a machine readable storage device storing machine readable instructions which, when executed by the processing circuitry, cause the processing circuitry to: simulate of a three-dimensional joining operation within a simulation domain; and display a cross-section of a simulated result of the simulated joining operation in response to receiving a selection of a location of the cross-section.

<FIG> is a block diagram of a weld training system <NUM> including simulation of a weld result based on monitoring training weld performance. The weld training system <NUM> may be used to provide weld training to welding students and/or operators on multiple different types of workpieces. The system <NUM> may be adapted to perform simulation and display of other types of workpiece joining techniques, such as brazing or adhesive bonding. For clarity and brevity, embodiments below are described with reference to welding.

The weld training system <NUM> includes a simulator <NUM>, one or more image sensor(s) <NUM>, one or more display(s) <NUM>, and a simulator interface <NUM>. The weld training system <NUM> may communicate with a training server <NUM> and/or one or more remote display(s) <NUM>, such as via a network <NUM>.

The simulator <NUM> receives data from the image sensor(s) <NUM>, the simulator interface <NUM>, and/or the training server <NUM>. The simulator <NUM> generates and simulates a simulation domain as a set of interconnected, cross-sectional two-dimensional slices of the simulation domain. By simulating the two-dimensional slices, the simulator <NUM> generates and displays a three-dimensional welding operation within the simulation domain.

The simulator <NUM> may focus the simulation only within the simulation domain, which may be defined by one or more simulated workpiece(s) <NUM>.

The image sensor(s) <NUM> generate images for use by the simulator <NUM> in determining weld parameters and/or recognizing a perspective of a welder. For example, the image sensor(s) <NUM> may be positioned and/or oriented to obtain stereoscopic images representative of a field of view of the welder so as to enable augmented reality images to be generated from the captured image(s) and the simulation. For example, the image sensor(s) <NUM> and one or more display(s) <NUM> may be positioned on a welding helmet worn by the welder. When the welder looks at the simulated workpiece(s) <NUM>, the image sensor(s) <NUM> capture a same or similar view as would be viewed by the welder through a typical welding helmet lens.

The weld training system <NUM> may include one or more additional displays, which may be positioned to be viewed by others besides the welder, such as an instructor. In some examples, the additional display(s) <NUM> display the same augmented reality view seen from the perspective of the welder. In some other examples, the display(s) <NUM> show weld parameters, weld training exercise information, and/or any other weld training information.

As described in more detail below, the simulated workpiece(s) <NUM> and a simulated torch <NUM> used for weld training exercises are provided with markers that are recognizable by the weld training system <NUM>. The image sensor(s) <NUM> observe the markers when the markers are within the field of view of the image sensor(s) <NUM>. Each of the markers contains unique data that enables rapid identification of the marker's identity, determination of the location of the marker on the simulated workpiece <NUM> or simulated torch <NUM> (e.g., via a marker-to-workpiece map or marker-to-torch map), and/or orientation of the marker for determination of the perspective of the image sensor(s) <NUM>.

The simulator interface <NUM> includes one or more input devices, such as dials, knobs, buttons, switches, and/or any other type of input device, to enable entry of instructions or data into the simulator <NUM>. For example, the simulator interface <NUM> enables selection of a predetermined weld training program, programming of weld parameters, selection of the simulated workpiece <NUM>, identification of the welder or other user, and/or any other setup of the simulator <NUM>.

As described in more detail below, disclosed weld simulation involves defining and managing multiple slices of a welding domain to track the structure and/or shape of a weld bead or other welding result, and rendering a simulated weld bead based on the data associated with the slices.

<FIG> is a block diagram of an implementation of the simulator <NUM> of <FIG>. The simulator <NUM> receives images from the image sensor(s) <NUM> and/or receives weld parameters from the simulator interface <NUM> and/or from the training server <NUM>. The simulator <NUM> includes a marker detector <NUM>, a reference frame generator <NUM>, a weld parameter detector <NUM>, a weld solver <NUM>, a simulation database <NUM>, a slice manager <NUM>, a cross-section renderer <NUM>, and a bead renderer <NUM>.

The marker detector <NUM> analyzes images generated by the image sensor(s) <NUM> to detect markers present in the images. The markers indicate the location and/or orientation of the simulated workpiece(s) <NUM> and/or the simulated torch <NUM> relative to the image sensor(s) <NUM> (e.g., relative to the welder's point of view). <FIG> illustrate physical workpieces <NUM>, <NUM> with markers <NUM> that are recognizable by the marker detector <NUM>. <FIG> illustrates multiple views of a physical training torch <NUM> with markers <NUM> that are recognizable by the marker detector <NUM>. Each of the markers <NUM> of <FIG> is unique and has a known size. The markers <NUM> are mapped to predetermined positions on the simulated workpiece(s) <NUM> or the simulated torch <NUM>. By identifying one or more of the markers <NUM> within an image, the marker detector <NUM> can determine the portion of the workpiece(s) <NUM> and/or the torch <NUM> that are visible, the distance from the image sensor(s) <NUM> to the workpiece(s) <NUM> and/or the torch <NUM>, the location and/or orientation of the torch relative to the workpiece(s) <NUM>, and/or the orientation of the workpiece(s) <NUM> from the perspective of the image sensor(s) <NUM>.

Returning to <FIG>, the marker detector <NUM> provides detection information to the reference frame generator <NUM> and to the weld parameter detector <NUM>. The detection information may include the identities and/or characteristics of one or more identified markers.

The reference frame generator <NUM> generates a reference frame for the weld solver <NUM> based on the workpiece(s) <NUM>. For example, the reference frame generator <NUM> may use a coordinate system X, Y, Z that is defined with respect to the workpiece, which corresponds to the physical workpiece(s) <NUM>. <FIG> illustrates cross sections of workpieces <NUM>, <NUM>, and parameters that may be used by the reference frame generator <NUM> of <FIG> to define a workpiece. The reference frame generator <NUM> may define basic workpieces capable of mathematical extrusion using parameters such as plate thickness <NUM>, root face thickness <NUM>, root opening length <NUM>, a bevel angle <NUM>, and/or a bevel shape. Bevel shapes include no cut, V-cut, U-cut, symmetric, and/or asymmetric. Additionally or alternatively, basic workpieces may be defined using a set of points within the coordinate system.

The reference frame generator <NUM> may define a workpiece cross section using the parameters, and extrude the cross-section to define the full three-dimensional shape of the workpiece. <FIG> illustrates an extrusion of a workpiece cross-section <NUM> that may be performed by the reference frame generator <NUM> of <FIG> to form a resulting workpiece <NUM>. In the example of <FIG>, the reference frame generator <NUM> extrudes the groove-shaped cross-section <NUM> along a circle to form the pipe-shaped workpiece <NUM>.

The reference frame generator <NUM> defines the workpiece with reference to a simulation domain and/or a workpiece reference frame. The workpiece reference frame may be a coordinate system, such as an X, Y, Z coordinate system having a designated point as the (<NUM>, <NUM>, <NUM>) origin point. The reference frame generator <NUM> defines other reference frames with respect to the workpiece reference frame (e.g., the simulation domain). <FIG> illustrates reference frames that may be generated by the reference frame generator <NUM> for performing a simulation using the workpiece <NUM> of <FIG>. As illustrated in <FIG>, the reference frame generator <NUM> defines a workpiece reference frame <NUM> with X, Y, and Z vectors. The workpiece reference frame <NUM> generates the workpiece <NUM> based on the workpiece reference frame <NUM>. The workpiece reference frame <NUM> is absolute, in that none of the X, Y, or Z vectors change during simulation.

The reference frame generator <NUM> defines a weld joint reference frame <NUM> relative to the workpiece reference frame <NUM>. A weld joint <NUM> is defined (e.g., based on a specified training weld, a weld procedure specification, etc.) within the workpiece reference frame <NUM>, and the weld joint reference frame <NUM> is specified with reference to the weld joint <NUM>. The weld joint <NUM> goes around the circumference of the workpiece <NUM>. In the example of <FIG>, the local Y vector and/or the local X vector of the weld joint reference frame <NUM> may change with respect to the workpiece reference frame <NUM>, while the local Z vector remains the same between the reference frames <NUM>, <NUM>. While one weld joint <NUM> and one weld joint reference frame <NUM> are shown in <FIG>, the reference frame generator <NUM> may include any number of weld joints and corresponding weld joint reference frames.

The weld joint <NUM> is modeled using a number of slices that are perpendicular to the weld joint <NUM> at each point. The slices may be positioned at respective sampling intervals along the weld joint, and may be considered to be two-dimensional and/or to have a width equivalent to the distance between the sampling intervals. The reference frame generator <NUM> defines, for each of the slices, a slice reference frame <NUM>. The slice reference frame <NUM> is defined with reference to the weld joint reference frame <NUM>. As a result, the local X, Y, and/or Z vectors of the slice reference frame <NUM> may differ from the local X, Y, and/or Z vectors of the workpiece reference frame <NUM> and/or the weld joint reference frame <NUM>. In the example of <FIG>, for a given point along the weld joint <NUM>, the local Y vector and/or the local X vector of the slice reference frame <NUM> may change with respect to the workpiece reference frame <NUM> and/or with respect to the weld joint reference frame, while the local Z vector remains the same between the reference frames <NUM>, <NUM>, <NUM>.

<FIG> illustrates weld joints and corresponding weld joint reference frames that may be defined by the reference frame generator <NUM> of <FIG> for a plate-type workpiece <NUM>. To simulate welding of a workpiece, the reference frame generator <NUM> defines one or more weld joints <NUM>, <NUM>, <NUM> on the workpiece. For the purpose of simulation, the weld joints <NUM>, <NUM>, <NUM> are defined by joint trajectories <NUM>, <NUM>, <NUM> and joint zones <NUM>, <NUM>, <NUM>.

Each of the joint trajectories <NUM>, <NUM>, <NUM> is a curve in three-dimensional space that defines the centerline of the joint <NUM>, <NUM>, <NUM>. Joint trajectories can be expressed as simple mathematical expressions (e.g., a straight line, a circle, etc.), a polynomial function, a sequence of points, and/or any other technique. The joint zones <NUM>, <NUM>, <NUM> define the areas around the joint trajectories <NUM>, <NUM>, <NUM> that can be welded. In some examples, simulation is limited to the joint zones <NUM>, <NUM>, <NUM>. One or more weld beads can be placed or performed within a weld joint <NUM>, <NUM>, <NUM> (e.g., one or more passes of a weld).

As illustrated in <FIG>, multiple weld joints <NUM>, <NUM>, <NUM> can be defined on a workpiece. As illustrated in <FIG>, looping weld joints can be defined, in which the entire circumference can be welded without a gap. The reference frame generator <NUM> can place weld joints anywhere on the workpiece. The reference frame generator <NUM> defines a weld joint reference frame <NUM>, <NUM>, <NUM>, relative to a workpiece reference frame <NUM>, for each of the weld joints <NUM>, <NUM>, <NUM>.

Joint reference frames may be expressed as respective transformation matrices. The reference frame generator <NUM> defines transformations that map weld joint space (e.g., weld joint reference frames) to workpiece space (e.g., the workpiece reference frame), as well as inverse transformations from the workpiece space to the weld joint space. A simulation domain is created for each weld joint. The simulation domain is a data structure that contains the data for the simulation of welding of that weld joint. The simulation domain may be stored in, for example, the main computer memory (RAM) during execution of the simulation.

The reference frame generator <NUM> determines a transformation of a designated coordinate system (e.g., an X, Y, Z coordinate system) to the images. The bead renderer <NUM> determines, based on the detection information, the position and/or orientation of a simulation domain within the images. When rendering the weld bead (as discussed in more detail below), the bead renderer <NUM> renders the weld bead based on determining a location and/or orientation of the workpiece space within the image and applying the appropriate transformations to joint spaces and slice spaces to render the perspective to match the captured image(s).

<FIG> illustrates a weld joint <NUM> and a slice <NUM> that may be defined by the reference frame generator <NUM> of <FIG> for a tee joint-type workpiece <NUM>. The weld joint <NUM> has a weld trajectory <NUM> that is defined as a straight line (e.g., along a seam between two pieces of the workpiece <NUM>). To generate slices, the weld joint <NUM> is sampled at regular or irregular intervals. At each sample interval, the reference frame generator <NUM> defines a slice (e.g., the slice <NUM>) having a slice plane. The sampling interval can be configured depending on desired precision. A sampling interval is <NUM> samples per <NUM> (<NUM> inch). The reference frame generator <NUM> may provide the slice definitions (e.g., as a function or transformation of the weld joint space and/or as a function or transformation of the workpiece space) to the slice manager <NUM> of <FIG>. For each slice, the reference frame generator <NUM> defines a plane to be perpendicular to the joint trajectory and represents the cross-section of the workpiece and any weld beads at that point along the weld trajectory.

Each slice has a two-dimensional coordinate system and a reference frame. The position and orientation of a given slice within the three-dimensional reference frame of the weld joint may be expressed as a transformation matrix (e.g., a 4x4 transformation matrix). <FIG> illustrates a coordinate system <NUM> that may be used by the slice manager <NUM> of <FIG> for the slice <NUM> in the workpiece of <FIG>. As described in more detail below, the result of a welding simulation (e.g., as determined by the weld solver <NUM> of <FIG>) is represented within the slice <NUM> using control points and polygons. The control points and polygons are defined and/or stored with reference to the coordinate system <NUM>, from which the data can be transformed to the joint space and/or workpiece space for rendering of the resulting weld bead. The coordination system <NUM> for a slice is referred to herein as "slice space. " The reference frame generator <NUM> defines a projection that maps two-dimensional slice space to the three-dimensional joint space and/or defines an inverse projection.

The slice manager <NUM> stores a representation of the cross-section data of each slice using polygons with linear interpolation of segments. In some examples, the polygons are defined through sequences of points in two-dimensions, and may be defined to always be closed. The slice manager <NUM> stores polygons with has associated data. Polygon data includes: whether the polygon represents the coupon, weld bead, or auxiliary data; a pass number (e.g., bead number) of the polygon; a probability of simulated welding defects such as porosity or cracks in the polygon; applied heat input; and/or additional data stored per polygon segment, such as a sequence of slices that represent the volume of the simulated workpiece and all weld beads inside the weld joint area.

The slice manager <NUM> stores slice data using vector data for storage efficiency. In some examples, the slice manager <NUM> compresses the slice data (e.g., the vector data) for further storage efficiency. The efficiency of storage enables the slice manager <NUM> to transmit slice data over a network to one or more remote devices (e.g., via the network <NUM> of <FIG>, via the Internet, etc.) in substantially real time. For example, instructor software may receive the data from the simulator <NUM> and render the weld bead locally based on the received slice data. Additionally or alternatively, a mobile device located proximate the simulated workpiece may be used to provide the welder's view or a separate view based on the perspective of the mobile device, by receiving the slice data in real time from the simulator <NUM>. The analysis components that generate the volumetric data representing the slice are separable from the graphical rendering components and, therefore, can be considerably smaller in size than conventional simulation systems. In some examples, the slice manager <NUM> creates volumes using sequences of multiple slices, in which each of the slices in the sequence stores a cross-section at a respective point along the weld bead.

Disclosed simulation methods and systems using the slice data structure has multiple benefits compared with conventional weld simulation techniques. For example, the slice data structure improves the scalability of simulation, allowing the simulation to be performed at different resolutions depending on available system resources and capabilities. In some examples, the calculation of polygons can be implemented entirely on a CPU, without use of the GPU. The GPU can then be used to perform rendering, shading, and/or other graphical techniques based on available GPU resources. Disclosed simulation methods and systems using the slice data structure also improve simulation efficiency, which enables the real-time transmission of data over a network to other devices (e.g., the remote device(s) <NUM> of <FIG>). The slice data structure can then be rendered from the same or even a different point of view than the welder (e.g., from an observer's point of view), because the slice data is provided without rendering, which substantially reduces the data processing and/or transmission loads on the simulator <NUM>.

Disclosed simulation methods and systems using the slice data structure may result in improved mobility of simulation systems. For example, disclosed simulation methods are suitable for implementation on mobile devices and/or any other suitable computing device (e.g., using a processor of the computing device and instructions stored in the memory of the computing device). Such simulation methods and/or systems can generate the simplified form of the weld bead, which enables using an analysis module player without the need of downloading the data from an external service.

Disclosed simulation methods and systems using the slice data structure enable multiple weld joints on a single coupon and simulation of weld joints with complex geometry. The slice data structure separates the shape of simulated weld bead into a cross-section and a lateral profile. Each of the cross-section and lateral profile can be calculated separately and/or combined at later stage to produce a final weld bead volume. The slice data structure enables both front and back facing sides of the weld union to be stored, enabling inspection, simulation, and welding of a union from multiple sides. For example, some welding processes that require welding a root pass from both sides can be performed using disclosed simulation methods and systems.

Disclosed simulation methods and systems using the slice data structure improve flexibility of the simulation system, including enabling the use of more complex (e.g., custom and/or non-standard) workpieces. Such custom workpieces may have multiple weld joints with a variety of weld procedures, and may be particularly useful for training welders to weld parts that will be actually welded on a shop floor or other manufacturing setting. Modifications of base coupon geometry, as well as of all underlying weld beads, can be stored and visualized. The flexibility in workpiece design is also useful for simulating GTAW welding processes, in which the weld puddle might be formed from the melted metal of the workpiece (e.g., instead of the filler). The flexibility is also useful for simulating thin workpieces and/or simulating welding of certain materials such as aluminum, where there is a real risk of deformation of the workpiece. Disclosed systems and methods can simulate, store, and/or visualize perforations of the base workpiece and/or of any or all underlying weld beads, which is useful for simulating thin workpieces for which perforation of the workpiece is a substantial risk.

For each welding pass, disclosed systems and methods store the volume of the simulated weld bead. Weld bead data is stored as polygons and is therefore already efficiently stored. Also, additional data can be stored for any point along the weld bead. Such additional data might include a probability of simulated welding defects (e.g., porosity) and/or a probability of simulated applied heat input. Disclosed examples also provide the ability to undo all or part of the weld bead (e.g., removing the simulated weld bead in reverse order from which the weld bead was created) by storing changes to slices between samples and/or between passes. The ability to undo enables a review of earlier passes by the welder after a later pass has been performed.

Disclosed systems and methods enable visibility and examination of weld cross-sections and/or undercut. Because the slices extend through a cross-section of the workpiece as well as through the weld beads and/or passes, disclosed systems and methods can show a cross-section for any slice (e.g., any sample point) along the weld bead. Said systems and methods can also show cross-section of the forming weld bead in substantially real time, while the student is welding.

<FIG> illustrates a data structure of a slice <NUM> of a T-joint workpiece <NUM>. The slice <NUM> is a visual representation of a data structure including first polygons <NUM> representing the workpiece <NUM>, and second polygons <NUM> representing the weld bead. The first polygons <NUM> include data indicating that the first polygons <NUM> are representative of the workpiece <NUM>, and the second polygons <NUM> include data indicating that the second polygons <NUM> are representative of the weld bead. The first and second polygons <NUM>, <NUM> are defined using control points and/or equations, with reference to a coordinate system (e.g., the coordinate system <NUM> of <FIG>).

<FIG> illustrates a data structure of another slice <NUM> of the T-joint workpiece, including polygons <NUM>, <NUM> associated with different weld beads (and polygons associated with the workpiece). As in <FIG>, the polygons <NUM>, <NUM> are defined using control points and/or equations, with reference to a coordinate system (e.g., the coordinate system <NUM> of <FIG>). However, the polygons <NUM>, <NUM> belong to different weld beads, such that the polygons <NUM>, <NUM> may have different transformations with respect to the separate weld bead reference frames.

<FIG> illustrates a data structure of yet another slice <NUM> of a Y-shaped workpiece, including polygons <NUM>, <NUM>, <NUM> (and polygons associated with the workpiece). As in <FIG>, the polygons <NUM>, <NUM>, <NUM> are defined using control points and/or equations, with reference to a coordinate system (e.g., the coordinate system <NUM> of <FIG>). However, the polygons <NUM>, <NUM>, <NUM> each belong to a different weld bead, such that the polygons <NUM>, <NUM>, <NUM> may have different transformations with respect to the separate weld bead reference frames.

Returning to <FIG>, when the workpiece space, weld joints, weld joint space, slices, and/or slice reference frames have been generated for a weld training simulation, the simulator <NUM> may simulate a welding operation. In some examples, simulation of welding occurs in response to an initiation of the simulation by the welder. For example, the welder may press a "start simulation" button on the simulator interface <NUM>, depress a trigger on the simulated torch <NUM>, and/or otherwise initiate the simulation.

<FIG> is a flowchart representative of machine readable instructions <NUM> which may be executed by the simulator <NUM> of <FIG> to simulate a result of a training weld. The simulator <NUM> and/or the instructions <NUM> may be implemented by the computing system <NUM> of <FIG>. The instructions <NUM> of <FIG> may be executed as iterative steps to provide a sensation to the user of a continuous simulation. Each simulation step (or frame) updates the simulated weld bead and workpiece data (e.g., slice data) for a particular time differential. A frame rate or step rate may be approximately <NUM> frames per second.

At block <NUM>, the weld parameter detector <NUM> calculates input welding parameters. For example, the weld parameter detector <NUM> determines weld parameters based on images captured by the image sensor(s) <NUM> and/or recognized markers in the images. For example, using the marker information, the weld parameter detector <NUM> detects travel speed (e.g., changes in location of the torch relative to the workpiece, based on marker recognition), work angle, travel angle, arc length, and/or contact tip to work distance. The weld parameter detector <NUM> provides the detected weld parameters to the weld solver <NUM> at a designated sampling rate.

In addition to the visually-detected weld parameters, programmed weld parameters affect the simulation result. Programmed weld parameters include power source values such as weld process type (e.g., SMAW, GMAW, FCAW, GTAW, etc.), weld voltage, weld current, wire feed speed, electrode type, electrode composition, weld gas type, and/or weld gas flow rate. The weld solver <NUM> may determine and/or receive the programmed weld parameters.

At block <NUM>, the weld solver <NUM> calculates a cross-section of the simulated weld bead based on the input welding parameters for affected slices. For example, the weld solver <NUM> iteratively calculates the cross-section of the simulated weld bead (e.g., for each slice affected by the simulated arc at a given time) and calculates characteristics of the weld puddle, based on the input parameters. An implementation of block <NUM> is described below with reference to <FIG>.

At block <NUM>, the weld solver <NUM> solves a weld puddle shape and volume based on the cross-sections of affected slices. For example, the weld solver <NUM> may use the cross-sections of the slices and/or an expected shape of a weld puddle to simulate a resulting shape of the weld puddle. An implementation of block <NUM> is described below with reference to <FIG>.

To calculate the cross-sections and the weld puddle, the weld solver <NUM> accesses the simulation database <NUM> using the input parameters and data from one or more prior iterations. For example, prior iterations may determine a shape of the workpiece and filler, to which new filler may be added during a subsequent iteration.

The simulation database <NUM> of <FIG> includes a lookup table that maps combinations of input parameters and polygon data to weld puddle shapes and/or weld results. The simulation database <NUM> may be populated based on empirical test data, computer modeling, and/or any other method of determining relationships between available input parameters, polygon data, weld puddle shapes, and/or weld results. The weld solver <NUM> queries the simulation database <NUM> at each iteration or sample, using detected input parameters including detected weld parameters and/or programmed weld parameters.

In addition to querying the simulation database <NUM>, the weld solver <NUM> applies rules to evaluate and determine the weld result at a given point during the welding simulation. The slice data (e.g., weld cross-section) is represented in simplified terms, using control points and relations between the control points.

<FIG> is a flowchart representative of machine readable instructions <NUM> which may be executed by the weld solver <NUM> of <FIG> to calculate a cross-section of a simulated weld bead. The weld solver <NUM> and/or the instructions <NUM> may be implemented by the computing system <NUM> of <FIG>.

The instructions <NUM> may be executed to implement block <NUM> of <FIG> to calculate a cross-section of the simulated weld bead based on the input welding parameters for affected slices. The instructions <NUM> are described below with reference to a single slice of a set of affected slices during a single step of the simulated weld, but instances of the instructions <NUM> are executed (e.g., serially and/or in parallel) for each of the affected slices in the step. <FIG> illustrate an example of calculating a polygon for a cross-section (e.g., a slice <NUM>) during a simulation. The process illustrated in <FIG> may be performed for multiple slices at each step of the simulation. <FIG> are discussed below with reference to the instructions <NUM> of <FIG>.

At block <NUM> of <FIG>, the weld solver <NUM> samples surface(s) proximate the welding deposit source. <FIG> illustrates sampling of a surface <NUM> around a welding deposit source <NUM> (e.g., an electrode). As illustrated in <FIG>, the weld solver <NUM> samples the surface at multiple intervals, which includes sampling a workpiece <NUM> and any weld bead surface that may be present (from prior steps).

After sampling, the weld solver <NUM> determines a set of control points for each affected slice. At block <NUM>, the weld solver <NUM> determines whether the current iteration is the first iteration (e.g., for the current simulation step). If the current iteration is the first iteration (block <NUM>), at block <NUM> the weld solver <NUM> places control points for the slice based on the welding parameters using default values. The default values may be determined using one or more rules. At block <NUM>, the weld solver <NUM> generates a cross-section outline from the control points.

If the current iteration is not the first iteration (block <NUM>), at block <NUM> the weld solver <NUM> places control points for the slice based on the welding parameters using control points from one or more prior iteration(s). At block <NUM>, the weld solver <NUM> generates a cross-section outline from the control points and data from the prior iteration(s).

<FIG> illustrates a placement of control points <NUM>. <FIG> illustrates a first iteration (e.g., blocks <NUM>, <NUM>), in which the weld solver <NUM> places control points <NUM> based on default values. For subsequent iterations, the control points are placed based on data from one or more prior iterations (see <FIG> for control points <NUM> set during a subsequent iteration (e.g., blocks <NUM>, <NUM>). The control points function as an intermediate shape of the weld bead cross-section within the slice.

The weld solver <NUM> generates a cross-section polygon based on the control points. <FIG> illustrates a generation of the cross-section outline <NUM> from control points and data from zero or more prior iterations. The weld solver <NUM> may add and/or remove control points <NUM>, <NUM> and relationships between control points as needed to represent the cross-section (e.g., as limited by the cross-section area determined above). The weld solver <NUM> may process the intermediate shape at a higher resolution than the stored data to improve the quality of the resulting weld bead shape.

After generating the cross-section outline (block <NUM> or block <NUM>), at block <NUM> the weld solver <NUM> calculates an area within the cross-section outline that was not occupied during the prior simulation step. The area that was not previously occupied represents added volume (e.g., due to added filler metal). For example, the weld solver <NUM> calculates a volume of material deposited from the filler using a filler deposit rate (e.g., a programmed deposit rate such as wire feed speed, an observed value based on changes in arc length and/or CTWD using visual information, and/or a combination of programmed deposit rate and visually observed data) and travel speed of the simulated torch <NUM> (e.g., visually observed data). The weld solver <NUM> determines a corresponding area in affected slices based on the calculated volume of material. The weld solver <NUM> uses the calculated area as the primary parameter controlling the resulting shape of the cross-section (e.g., the weld bead polygon shapes). As illustrated in <FIG>, the weld solver <NUM> may calculate a total area <NUM> within the slice <NUM> that was not occupied by any material (e.g., workpiece material or filler material) during a previous step. The total area <NUM> is bound by the outline <NUM>.

At block <NUM>, the weld solver <NUM> evaluates the current cross-section based on one or more specified rule(s) and parameter(s). The weld solver <NUM> evaluates the current slice <NUM> based on rules and/or parameters, which may be applied based on the welding process and/or filler material used for the simulated weld. The rules establish restrictions on the cross-section (e.g., polygon) shape. Example rules include: requiring that the volume of resulting weld bead must match (e.g., within a margin of error) a volume of material deposited from the filler within the slice and/or across affected slices for the iteration; the extremities of the polygon must be connected to material that was present in a previous iteration (e.g., the bead cannot float in the air, disconnected from the workpiece); the polygon must form a continuous, non-overlapping curve; the polygon must be aligned with the welding source (e.g., an electrode, a welding gun, etc.); and/or the base estimation for the cross-section shape is derived from the underlying surface. Additional or alternative rules may be added.

At block <NUM>, the weld solver <NUM> determines whether the solution for the current cross-section meets threshold requirements. If the solution for the current cross-section does not meet threshold requirements (block <NUM>), at block <NUM> the weld solver <NUM> determines whether a number of performed iterations for the cross-section during the current step is at least a threshold number of iterations. If the number of performed iterations for the cross-section during the current step is less than the threshold number of iterations (block <NUM>), control returns to block <NUM> to place control points for another iteration.

For example, after evaluating a slice based on the rules, the weld solver <NUM> may start a new iteration at the placement of control points (e.g., <FIG>). Each iteration is based on data from the previous iteration. The weld solver <NUM> iteratively refines the placement of control points based on the rules, parameters specific to each welding process, filler type and/or diameter, and on real-time input parameters. Parameters specific to welding process and filler type/diameter include: weld bead width and/or height; weld bead convexity and/or concavity; reinforcement and penetration depth; and/or reinforced area, penetrated area, and/or dilution factor.

<FIG> illustrate characterizations of weld beads <NUM>, <NUM>, <NUM> based on calculated control points <NUM> for a simulated weld bead within a slice. The characterizations represented in <FIG> may be used by the weld solver <NUM> to evaluate an iteration of control point placement within a slice.

<FIG> illustrates a characterization of a root pass weld bead <NUM> for a butt joint <NUM>. The characterization of the weld bead <NUM> includes calculations of a bead width <NUM>, a throat depth <NUM>, a reinforcement depth <NUM>, and a penetration depth <NUM>. <FIG> illustrates similar characterizations <NUM>-<NUM> for a corner bead <NUM> (e.g., a T-joint <NUM>). <FIG> illustrates similar characterizations <NUM>-<NUM> for a flat bead <NUM> (e.g., a plate <NUM>), with a throat depth of <NUM>. Each of the characterizations <NUM>-<NUM> may be defined by particular control points and/or polygons, which have different interpretations for the different types of joints <NUM>, <NUM>, <NUM>.

<FIG> illustrate weld beads for slices of multiple workpiece and/or joint types based on calculated control points and vectors. <FIG> illustrates a slice <NUM> with a calculated butt joint root bead. <FIG> illustrates a slice <NUM> with a calculated T-joint root bead. <FIG> illustrates a slice <NUM> with a calculated plate flat bead. <FIG> illustrates a slice <NUM> with a T-joint bevel bead. <FIG> illustrates a slice <NUM> with a lap joint edge bead. <FIG> illustrates a lap joint <NUM> second pass bead over a first pass.

If the solution for the current cross-section meets the threshold requirements (block <NUM>), or if the number of performed iterations for the cross-section during the current step meets the threshold number of iterations (block <NUM>), at block <NUM> the weld solver <NUM> selects the best solution from the iterations. The instructions <NUM> may then iterate for another slice and/or end and return control to block <NUM> of <FIG>. The iterations may end when a threshold number of iterations are performed and/or when a solution has been identified that meets selected (or all) of the applied rules.

<FIG> is a flowchart representative of machine readable instructions <NUM> which may be executed by the weld solver <NUM> of <FIG> to solve a weld puddle shape and volume. The weld solver <NUM> and/or the instructions <NUM> may be implemented by the computing system <NUM> of <FIG>.

The instructions <NUM> may be executed to implement block <NUM> of <FIG> to solve a weld puddle shape and volume. The instructions <NUM> do not directly calculate the entire volume of the weld puddle. Instead, the instructions <NUM>, when executed, cause the weld solver <NUM> to calculate only the cross-section of the resulting simulated weld bead and then approximate the shape of the weld puddle.

The instructions <NUM> are described below with reference to a single slice of a set of affected slices during a single step of the simulated weld, but instances of the instructions <NUM> are executed (e.g., serially and/or in parallel) for each of the affected slices in the step. <FIG> illustrates a calculation of a simulated weld puddle <NUM> across multiple slices of a simulated weld bead <NUM>. The instructions <NUM> are described below with reference to the weld puddle calculation of <FIG>.

At block <NUM>, the weld solver <NUM> determines a set of slices <NUM> affected by the welding zone of the current step. For example, the set of slices <NUM> are determined based on measured weld parameters (e.g., the location of the simulated weld torch <NUM> relative to the simulated workpiece <NUM> as determined by the marker detector <NUM>) and/or programmed weld parameters. The affected slices are illustrated in <FIG> as vertical lines separated by intervals.

At block <NUM>, the weld solver <NUM> defines a lateral interpolation curve for the weld puddle. The lateral interpolation curve determines the lateral shape of the weld puddle (e.g., across the affected slices <NUM>). A lateral interpolation curve is illustrated in <FIG> as a surface curve <NUM> over the affected slices <NUM> and a penetration curve <NUM> on a workpiece side of the slices. The weld solver <NUM> may select or calculate the lateral interpolation as a predetermined relationship based on measured and/or programmed weld parameters to provide a characteristic slope and curvature of a real-world weld puddle, which often takes a consistent shape.

At block <NUM>, the weld solver <NUM> determines a primary slice cross-section based on the affected slices. For example, the primary slice cross-section may be selected by simulating the slice cross-sections as described above with reference to <FIG>. A primary slice cross-section <NUM> is illustrated in <FIG>.

At block <NUM>, the weld solver <NUM> derives an intermediate slice from the bead cross-section based on the lateral interpolation curve and the primary slice cross-section. At block <NUM>, the weld solver <NUM> derives a distal slice from the bead cross-section based on the lateral interpolation curve and the primary slice cross-section. For example, the weld solver <NUM> fits the lateral interpolation curve <NUM>, <NUM> to the primary slice cross-section to determine the locations and set the cross-section polygon data of the intermediate slice and the distal slice. An intermediate slice <NUM> and a distal slice <NUM> are illustrated in <FIG>. However, the positions of the intermediate slice <NUM> and/or the distal slice <NUM> may be selected to be different from the slices <NUM>, <NUM>.

At block <NUM>, the weld solver <NUM> interpolates the remaining affected slices based on the primary slice, the intermediate slice, the distal slice, and the lateral interpolation curve. For example, the weld solver <NUM> may reconfigure the slice geometries and/or control points to fit the lateral interpolation curve as set based on the primary slice, the intermediate slice, and the distal slice.

After interpolating the weld puddle, the example instructions may end and return control to block <NUM> of <FIG>.

Returning to <FIG>, at block <NUM>, the weld solver <NUM> merges a current weld puddle volume with a weld puddle volume from the prior step. For each affected slice of the weld puddle (block <NUM>), the weld solver <NUM> merges the cross-section polygon of the weld puddle with any previous weld puddle polygon. The merging of weld puddles improves the simulation accuracy of weaved welds.

<FIG> is a flowchart representative of machine readable instructions <NUM> which may be executed by the weld solver <NUM> of <FIG> to merge a new weld puddle volume with a previous weld puddle volume. The weld solver <NUM> and/or the instructions <NUM> may be implemented by the computing system <NUM> of <FIG>. The instructions <NUM> may be executed to implement block <NUM> of <FIG> to merge a new weld puddle volume with a previous weld puddle volume.

At block <NUM>, the weld solver <NUM> selects an affected slice in the weld puddle. The affected slice may be one of the slices <NUM> of <FIG>. At block <NUM>, the weld solver <NUM> samples the cross-section outline to determine a cross-section outline area.

At block <NUM>, the weld solver <NUM> samples the bead polygon to determine a bead polygon area.

At block <NUM>, the weld solver <NUM> determines whether the cross-section outline area is greater than or equal to the bead polygon area. If the cross-section outline area is greater than or equal to the bead polygon area (block <NUM>), at block <NUM> the slice manager <NUM> stores the cross-section outline for the selected slice. On the other hand, if the cross-section outline area is less than the bead polygon area (block <NUM>), at block <NUM> the slice manager <NUM> stores the bead polygon for the selected slice.

After storing the cross-section outline (block <NUM>) or storing the bead polygon (block <NUM>), at block <NUM> the weld solver <NUM> determines whether there are any additional affected slices for merging. If there are additional slices for merging (block <NUM>), control returns to block <NUM> to select another affected slice.

When there are no more slices for merging (e.g., all of the affected slices have been merged with the prior weld puddle) (block <NUM>), at block <NUM> the weld solver <NUM> resamples all of the affected slice polygons to a selected resolution. Sampling resolutions may be between <NUM> and <NUM> (<NUM> inches and <NUM> inches). A sampling resolution is <NUM> (<NUM> inches). <FIG> illustrates a sampling of visible surfaces of a welding result (e.g., a bead) for a slice <NUM>. The slice of <FIG> illustrates a sampling resolution as vertical lines corresponding to a sampling function <NUM>. A front (or top) surface <NUM> and a rear (or bottom) surface <NUM> are sampled to determine the respective positions.

The instructions <NUM> then end and return control to block <NUM> of <FIG>.

Returning to <FIG>, following block <NUM>, the slice manager <NUM> has stored slice geometry data and sample data for the previous slices and for slices within the weld puddle for the current step. At block <NUM>, the bead renderer <NUM> generates a coarse triangle mesh based on cross-sections (e.g., slices) of the simulated weld bead and the updated weld puddle.

<FIG> illustrates a construction of a triangular mesh <NUM>, which may be performed by the bead renderer <NUM> of <FIG>, for rendering two adjacent slices <NUM>, <NUM> based on polygonal weld bead shapes calculated for the slices. Each of the slices <NUM>, <NUM> is represented be multiple sampled points 2006a-2006i, 2008a-2008i. The sampled points 2006a-2006i, 2008a-2008i are generated based on the polygonal data for the slices (e.g., block <NUM> of <FIG>). For ease of reference, certain ones of the sampled points of <FIG> are marked A1, A2, A3 (for sampled points on Slice A <NUM>) and B1, B2, B3 (for sampled points on Slice B <NUM>).

The bead renderer <NUM> constructs the triangle mesh by connecting sets of three points, such that connections (i.e., vertices in <FIG>) do not overlap. As illustrated in <FIG>, a first triangle <NUM> is constructed between samples A1, B1, and B2. A second triangle <NUM> is constructed between samples A1, A2, and B2. A third triangle <NUM> is constructed between samples A2, B2, and B3. A fourth triangle <NUM> is constructed between samples A2, A3, and B3. Additional triangles are constructed for the remaining sets of samples 2006c-2006i, 2008c-2008i to form a triangle mesh between the adjacent slices <NUM>, <NUM>.

<FIG> illustrates a triangular mesh <NUM>, which may be constructed by the bead renderer <NUM> of <FIG>, for rendering a weld bead over multiple slices including the slices <NUM>, <NUM> of <FIG>. Adjacent slices in <FIG> are connected with a triangle mesh in an identical manner as described above with reference to <FIG>. The triangular mesh of <FIG> is scalable. The number of triangles can be increased or decreased by increasing or decreasing (respectively) the number of samples of the slice polygon surfaces.

Returning to <FIG>, at block <NUM> the bead renderer <NUM> projects the weld result (e.g., the three-dimensional representation of the weld bead and the weld puddle, as the generated triangular mesh) onto a two-dimensional image space. <FIG> illustrates a projection <NUM> by the bead renderer <NUM> of a weld result <NUM> onto a two-dimensional image space <NUM>. The weld result <NUM> of <FIG> includes a weld bead portion <NUM> and a weld puddle portion <NUM>. <FIG> illustrates an outline <NUM> of the weld bead portion <NUM> and an outline of the weld puddle portion <NUM> as projected onto the two-dimensional image space <NUM>. In addition to generating the projection, the bead renderer <NUM> may also generate an inverse projection.

Following block <NUM>, the bead renderer <NUM> has generated a coarse geometry of the simulated weld bead. At block <NUM>, the bead renderer <NUM> draws finer surface detail onto a set of images. The surface detail may be overlaid onto the projection generated in block <NUM> to improve a visual quality of the weld bead. The bead renderer <NUM> generates additional data for the surface detail using a two-dimensional image format, which improves performance of the simulation. Surface detail includes: a color map, a normal map, a heat affected zone (HAZ) map, and a weld puddle map. The bead renderer <NUM> stores the color map as a Red, Blue, Green, Opacity (RGBA) image, in which color information is stored using the RGB channels and surface type information is stored in the A channel. The normal map is stored as an RGB image, in which micro-geometry information is stored using the RGB channels. The HAZ map is stored as an RGBA image, in which the HAZ color is stored in RGB channels and HAZ strength is stored in the A channel. The weld puddle map is stored as an RGB image, in which the RGB channels store information used to render the weld puddle and incandescence of the simulated weld bead.

The color map, the normal map, the HAZ map, and/or the weld puddle map may be calculated by referencing the simulation database <NUM> based on weld bead characteristics. The simulation database <NUM> may be populated with color information, micro-geometry information, surface type information, heat affected zone color, heat affected zone strength, weld puddle color, and/or weld puddle incandescence information by using empirical observations performed with real welding. For example, a real weld (or other filler operation) may be performed with known parameters. By capturing images before, during and/or after the operation, color information, micro-geometry information, surface type information, heat affected zone color, heat affected zone strength, weld puddle color, and/or weld puddle incandescence information may be extracted from the images and correlated with the known parameters. While larger numbers of samples of known parameters are generally better for accuracy, some blending and/or extrapolation between samples of known parameters may be used to calculate color information, micro-geometry information, surface type information, heat affected zone color, heat affected zone strength, weld puddle color, and/or weld puddle incandescence information from stored data in the simulation database <NUM>.

Each of the images representing the color map, the normal map, the heat affected zone (HAZ) map, and the weld puddle map may use a different resolution, based on the desired amount of detail. In some examples, the images have pixel counts that are a power of two to improve performance (e.g., both width and height, in pixels, or <NUM>, <NUM>, <NUM>, <NUM>, <NUM> or <NUM> pixels). The images provide additional surface detail to the appearance of the simulated weld bead. The resolutions of the surface detail maps may be substantially higher (e.g., several times higher) than the resolution of the coarse geometry. At each step of the simulation (e.g., each frame), the set of images is updated and new information is generated.

At block <NUM>, the bead renderer <NUM> projects surface detail from the set of surface detail images (e.g., as mapped to the two-dimensional image space <NUM> of <FIG>) onto the weld bead surface. For example, the outline of current weld puddle is projected into the two-dimensional image space to map onto surface detail images (e.g., for consistent orientation of the images in the set). The surface detail information projected from the set of images onto the workpiece and simulated weld bead are used to determine illumination and final color of the workpiece and the simulated weld bead. Illumination may be based on, for example, the presence or absence of a simulated weld arc and a simulated darkening of a welding helmet.

At block <NUM>, the bead renderer <NUM> combines the triangle mesh geometry and the projected surface detail to produce and output the weld result (e.g., to the display(s) <NUM> of <FIG>). For example, the set of images are projected to the triangle mesh geometry using the inverse projection generated by the bead renderer <NUM>. While the weld bead and weld puddle information have been determined, the final image produced on the display(s) <NUM> may involve further processing such as resampling based on the display resolution, applying bloom filters based on pixel energy, and/or generating and rendering additional information such as virtual indicators or helper icons to assist the welder with the weld. Virtual indicators may be displayed to assist with work angle, travel angle, travel speed, contact tip to work distance, and/or any other information.

When the welder is viewing a back side of the workpiece (e.g., a side of the workpiece opposite the side where the weld bead was placed), the instructions <NUM> execute in the same manner, but display the opposite surface of the weld bead due to the recognition by the marker detector <NUM> of markers placed on the back side of the workpiece.

While simulation techniques are described, additional or alternative characteristics or considerations may be added to the simulation techniques. Due to the calculation of bead cross-section and lateral profile separately, as well as modular performance of weld bead geometry calculation and weld bead rendering, disclosed systems and methods are easily updated to take into account such newly-developed characteristics. Examples of such characteristics may include considering the influence of gravity on the weld puddle, indicating additional and/or specific welding defects such as under-bite, and/or considering the adequacy of preparation of a workpiece prior to welding when determining weld geometry and/or defects.

<FIG> illustrate an example calculated and rendered root pass using the systems and methods disclosed herein. The realism of the illustrated weld bead is superior to that of conventional simulation techniques utilizing comparable computing resources.

<FIG> illustrates a slice <NUM> onto which multiple simulated weld passes <NUM>-<NUM> are made for a single weld joint. In <FIG>, as many as <NUM> weld passes may be simulated. When simulating subsequent weld passes, the same or similar weld simulation techniques and rendering techniques described herein may be used as for the initial weld pass. For example, the sampling of the surface <NUM> illustrated in <FIG> is performed based on slice data stored by the slice manager <NUM>. Instead of the sampled surface being identical to the surface of the underlying workpiece <NUM>, at least a portion of the sampled surface would include the resultant surface of previously deposited weld passes.

The simulation database <NUM> stores data representative of the effects of adding a weld pass to weld joint having one or more prior weld passes. The volume of prior passes (e.g., a combination of workpiece and filler volume) is also considered when adding filler volume during subsequent passes, such as by adding, moving, and/or removing control points, and/or by adjusting the polygons and/or outlines from prior passes to subsequent passes.

<FIG> is a block diagram of a computing system <NUM> that may be used to implement the marker detector <NUM>, the reference frame generator <NUM>, the weld parameter detector <NUM>, the weld solver <NUM>, the simulation database <NUM>, the slice manager <NUM>, the cross-section renderer <NUM>, the bead renderer <NUM> and/or, more generally, the simulator <NUM> and/or the weld training system <NUM> of <FIG>. The computing system <NUM> may be, an integrated computing device, a computing appliance, a desktop, or all-in-one computer, a server, a laptop or other portable computer, a tablet computing device, a smartphone, and/or any other type of computing device.

The computing system <NUM> of <FIG> includes a processor <NUM>. The processor <NUM> may be any general purpose central processing unit (CPU) from any manufacturer. In some other examples, the processor <NUM> may include one or more specialized processing units, such as RISC processors with an ARM core, graphic processing units, digital signal processors, and/or system-on-chips (SoC). In some examples, the computing system <NUM> includes two or more processors. Two or more processors may execute separate sets of instructions to enable the two or more processors to define the same workpiece, and to simulate separate welding operations on different simulation domains simultaneously. Simulating the separate welding operations would enable multiple operators to simultaneously perform simulated welding operations on the same workpiece.

The processor <NUM> executes machine readable instructions <NUM> that may be stored locally at the processor (e.g., in an included cache or SoC), in a random access memory <NUM> (or other volatile memory), in a read only memory <NUM> (or other nonvolatile memory such as FLASH memory), and/or in a mass storage device <NUM>. The mass storage device <NUM> may be a hard drive, a solid state storage drive, a hybrid drive, a RAID array, and/or any other mass data storage device.

A bus <NUM> enables communications between the processor <NUM>, the RAM <NUM>, the ROM <NUM>, the mass storage device <NUM>, a network interface <NUM>, and/or an input/output interface <NUM>.

The network interface <NUM> includes hardware, firmware, and/or software to connect the computing system <NUM> to a communications network <NUM> such as the Internet. For example, the network interface <NUM> may include IEEE <NUM>. X-compliant wireless and/or wired communications hardware for transmitting and/or receiving communications.

The I/O interface <NUM> of <FIG> includes hardware, firmware, and/or software to connect one or more input/output devices <NUM> to the processor <NUM> for providing input to the processor <NUM> and/or providing output from the processor <NUM>. For example, the I/O interface <NUM> may include a graphics processing unit for interfacing with a display device, a universal serial bus port for interfacing with one or more USB-compliant devices, a FireWire, a field bus, and/or any other type of interface. The computing system <NUM> includes a display device <NUM> (e.g., an LCD screen) coupled to the I/O interface <NUM>. Other example I/O device(s) <NUM> may include a keyboard, a keypad, a mouse, a trackball, a pointing device, a microphone, an audio speaker, an optical media drive, a multi-touch touch screen, a gesture recognition interface, a magnetic media drive, and/or any other type of input and/or output device.

The computing system <NUM> may access a non-transitory machine readable medium <NUM> via the I/O interface <NUM> and/or the I/O device(s) <NUM>. Examples of the machine readable medium <NUM> of <FIG> include optical discs (e.g., compact discs (CDs), digital versatile/video discs (DVDs), Blu-ray discs, etc.), magnetic media (e.g., floppy disks), portable storage media (e.g., portable flash drives, secure digital (SD) cards, etc.), and/or any other type of removable and/or installed machine readable media.

The present methods and/or systems may be realized in a centralized fashion in at least one computing system, or in a distributed fashion in which different elements are spread across several interconnected computing systems.

Claim 1:
A computer-implemented system (<NUM>) for simulation of joining materials with or without filler material, the system comprising:
processing circuitry;
a physical workpiece (<NUM>);
a physical joining tool (<NUM>);
one or more image sensors (<NUM>) configured to capture images of the physical workpiece and the physical joining tool corresponding to a joining operation; and
characterised in that the system further comprises:
a machine readable storage device storing machine readable instructions which, when executed by the processing circuitry, cause the processing circuitry to:
define a plurality of slice sampling intervals depending on a desired precision; define a slice plane according to a desired orientation with respect to a weld trajectory;
display a visual simulation of a three-dimensional joining operation within a simulation domain by simulating the simulation domain as a set of interconnected cross-sectional slices;
wherein said slices are oriented according to the defined slice plane along the weld trajectory;
wherein the slices are two-dimensional and have a width equivalent to a distance between the defined sampling intervals; and
wherein the simulation domain is based on images of the physical workpiece (<NUM>) and the physical joining tool (<NUM>) captured by the image sensors (<NUM>).