Convolutional neural network evaluation of additive manufacturing images, and additive manufacturing system based thereon

An additive manufacturing system uses a trained artificial intelligence module as part of a closed-loop control structure for adjusting the initial set of build parameters in-process to improve part quality. The closed-loop control structure includes a slow control loop taking into account in-process build layer images, and may include fast control loop taking into account melt pool monitoring data. The artificial intelligence module is trained using outputs from a plurality of convolutional neural networks (CNNs) tasked with evaluating build layer images captured in-process and images of finished parts captured post-process. The post process images may include two-dimensional images of sectioned finished parts and three-dimensional CAT scan images of finished parts.

FIELD OF THE INVENTION

The present invention relates to the field of additive manufacturing (AM).

BACKGROUND OF THE INVENTION

AM machines are useful in building finished parts according to a layer-by-layer build process. For example, laser powder bed fusion AM machines use either a laser or an electron beam to melt and fuse powder material. Powder bed fusion processes involve spreading thin layers of powder material over previous layers using a roller or a blade, and scanning the laser or electron beam in a controlled manner over the powder layer to form the layer according to a desired geometry of the part. A geometric computer model of the part is converted to an AM build parameter file in which various control parameters of the AM machine are defined for controlling the scanning and fusion operations for each build layer.

While AM shows great promise for manufacturing parts that are difficult and/or time consuming to manufacture by traditional subtractive manufacturing, and for manufacturing parts “on demand” at remote locations where an AM machine is present, concerns about the quality of parts made by AM have slowed its widespread adoption in critical industries. For example, parts made by AM sometimes exhibit porosity, voids, and poor surface finish, thus hampering acceptance of AM for safety critical applications such as aerospace and medical applications. This places an added burden on quality control inspection of finished AM parts, especially for parts intended for safety critical applications such as medical devices and aircraft parts.

It has been suggested in various publications that artificial intelligence can be applied to AM to improve the quality of finished parts. However, the publications lack any useful details or practical description of how to apply artificial intelligence to AM to improve the quality of finished parts.

SUMMARY OF THE INVENTION

The present disclosure provides an AM system for building a part layer-by-layer in an AM machine according to an AM build process, wherein the system includes a closed-loop control structure for adjusting an initial set of build parameters in-process. As used herein, the term “in-process” refers to a time period during which the part is in the process of being built in the AM machine. The term “in-process” is distinguished from the term “post-process,” which is used herein to refer to a time period after the part has been built in the AM machine.

The closed loop control structure of the present disclosure includes a slow control loop having a trained artificial intelligence module, and may further include a fast control loop having a state machine. As used herein, “slow control loop” means a control loop having a controller gain update period on the order of whole seconds, and “fast control loop” means a control loop having a controller gain update period on the order of microseconds. The trained artificial intelligence module may be a deep learning module having a recurrent artificial neural network.

In one embodiment, the AM system includes a melt-pool monitoring system arranged to acquire real-time melt pool data representative of a melt pool formed by the energy source in-process, and a build layer image sensor arranged to acquire layer images of the part layers in-process. An initial set of build parameters, a time-based sequence of adjusted build parameters corresponding to the build process, the layer images, and the melt pool data are transmitted as inputs to the trained artificial intelligence module of the slow control loop. The melt pool data may be transmitted as an input to the state machine of the fast control loop.

In accordance with the present disclosure, the trained artificial intelligence module may be trained using evaluation data from a first convolutional neural network (CNN) configured to evaluate layer images acquired in-process, and at least one second CNN configured to evaluate images of finished parts acquired post-process. For example, a CNN may be configured to evaluate two-dimensional images of sectioned finished parts acquired post-process, and another CNN may be configured to evaluate three-dimensional images of parts acquired post-process by computer tomography (CT) scanning of a finished part.

DETAILED DESCRIPTION OF THE INVENTION

An AM system10formed in accordance with an embodiment of the present invention is shown inFIG. 1. AM system10comprises an AM machine20, shown in greater detail inFIG. 2. AM machine20may be in the form of a laser powder bed machine of a type including a powder reservoir22, a powder bed24in which a part P is built, and a powder scraper26for transferring a new layer of powder from powder reservoir22into powder bed24. The elevation of powder reservoir is adjusted by means of a powder delivery actuator23and the elevation of powder bed24is adjusted by means of a fabrication actuator25. AM machine20further includes an energy source in the form of a laser28, and a scanner system30for redirecting and scanning a beam32from energy source28over each new layer of powder in powder bed24in a controlled manner to form part P. As will be understood, beam30interacts with powder layer in powder bed24and forms a trailing melt pool33which solidifies and fuses with part P to build the part. AM machines of the type described above are available from Renishaw plc of the United Kingdom.

AM machine20may be equipped with a melt-pool monitoring system35having one or more melt pool sensors37arranged to acquire real-time melt pool data39representative of melt pool33in-process. AM machine20is also equipped with a build layer image sensor38arranged to acquire layer images of part layers in-process. Additionally, spatial frequency modulated imaging (SPIFI) may be utilized to glean information about the state of the melt pool33directly through the beam32; see, e.g., Young, Michael D., et al, Spatial Frequency Modulated Imaging (SPIFI) with amplitude or phase grating from a spatial light modulator, Proceedings of the SPIE, Vol. 10069, id. 100692P 8 pp. (2017). The various components of AM machine20are connected to a microprocessor-based controller21configured to control the build process.

AM system10comprises a closed-loop control structure42for adjusting the initial set of build parameters41in-process. In a basic embodiment shown in FIG.3, the closed loop control structure42includes a trained artificial intelligence (AI) module in the form of a CNN46trained and configured to evaluate layer images48of part P acquired in-process by build layer image sensor38. The evaluation result provided by CNN46, which may indicate a degree to which each captured layer image48corresponds to an expected or desired appearance of the layer, is used in block50to calculate adjusted build parameters of AM machine20in-process to influence building of subsequent layers as the build process continues in block52. The evaluation result may be in the form of an assigned classification of each build layer image48into a predetermined category (e.g. very good, good, fair, bad, etc.).

In another embodiment corresponding toFIG. 1, closed loop control structure42includes a slow control loop54having a trained AI module in the form of a deep learning recurrent AI module56, and a fast control loop58having a state machine60.

In slow control loop54, the initial AM build parameters41generated by build parameter configuration module40are inputted to deep learning recurrent AI module56. Other inputs to trained AI module56may include sequential time-based data62representing AM process variables and parameters over time (e.g. argon flow, temperature, sound/vibration transducer levels, voltage, current, etc.), build layer images48acquired in-process by build layer image sensor38, and melt pool data39acquired in-process by melt pool monitoring system35. The melt pool data39may be preconditioned by a preconditioner64before input to deep learning recurrent AI module56. For example, preconditioner64may be programmed to accumulate and average melt pool data39over each build layer or a set of build layers. The preconditioning may be adjustable to have a shorter or longer frame rate.

Deep learning AI module56may have a recurrent neural network (RNN) component combined with one or more CNNs to form a committee of neural networks. The RNN component may be implemented, for example, as long short-term memory (LSTM) to overcome the so-called “vanishing or exploding gradient problem,” or a gated recurrent unit (GRU), which will allow the use of a large stack of recurrent networks that add process states and long-term memory capabilities to learn the complex, noisy and non-linear relationship between the fast in-process update data and the slow process output data, and predict the correct AM build parameters needed to build good quality parts. GRUs are described, for example, in Chung, et al, Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling, arXiv:1412,3555v1 [cs.NE] 11 Dec. 2014. The trained deep learning AI module56may be used to close the slow layer-to-layer evaluation of part quality for enhanced slow process feedback control. AI module56may be configured as a computer or network of computers running AI intelligence software. For example, the software may be programmed in Python™ programming language supported by the Python Software Foundation, using, as examples, TensorFlow (Google's open source artificial neural network (ANN) software library at https://www.tensorflow.org), Theano (University of Montreal's Deep Learning Group's open-source ANN software library at http://deeplearning.net/software/theano/index.html), or CNTK (Microsoft's Cognitive Toolkit at https://www.microsoft.com/en-us/cognitive-toolkit/) to actually implement the artificial neural network AI. Alternatively or additionally, more traditional programming languages such as C and C++ may be used. With regard to hardware, because AI module56may be running as an inference-only AI, the trained neural network could be run using fixed-point math or even lower bit-count (for example BNNs or Bitwise Neural Networks; see, e.g., Kim, Smaragdis, Bitwise Neural Networks, arXiv:1601.06071v1 [cs.LG] 22 Jan. 2016 (https://arxiv.org/pdf/1601.06071.pdf)) on dedicated computing platforms, and this may dramatically improve the processing-throughput of the AI module.

In fast control loop58, melt pool data39may be inputted to state machine60along with output from deep learning AI module56. A state machine output from deep learning AI module56may be used as part of the fast control loop58, which may be configured as a separate state-variable inner control loop on the fast process control gain update. For example, a state machine output from the LSTM mentioned above may be inputted to state machine60and used to facilitate fast-loop closure of the melt pool control.

InFIG. 6, a simple example of state machine60is shown with three different states as represented by a Mealy FSM, where the outputs from each state depend on the current state and the inputs to the FSM. The three states are “Hold” where the control scheme is maintained, “Lower Energy Density” (Lower ED) where the control scheme favors lowering the specific energy density (ED) being input to the powder bed24by beam32, and “Higher Energy Density” (Higher ED) where the control scheme favors elevating the specific ED being input to the powder bed24by beam32. Also in this example, the input to the FSM is an output from trained RNN56that predicts the condition of the melt pool33. The prediction is based on theFIG. 5training imparted to RNN56by theFIG. 4augmented data.

Each state in theFIG. 6example represents a different or altered control scheme. These control schemes could be implemented as simple gain-controlled feedback loops or as complex stochastic optimal controllers. Those skilled in the art will recognize that this is merely a simplified example of how a state machine60for fast-loop58control could be interfaced with the output from a RNN56, and that many other and more complex configurations are possible, including different control scheme states, as well as the way the control scheme states alter the many possible implementations of the underlying controllers.

As may be seen inFIG. 1, slow loop feedback from trained deep learning AI module56and fast loop feedback from state machine60may be used to calculate adjusted AM build parameters in block50for operating AM machine20in a manner which improves part quality.

An approach to training deep learning AI module56in accordance with an embodiment of the invention is now described with reference toFIGS. 4 and 5. Teacher data for training deep learning AI module56may be collected by operating AM machine20to build parts in a data augmentation mode represented byFIG. 4. As may be understood, basic CNN46tasked with evaluating in-process build-layer images48may be augmented by one or more further CNNs72and82configured to evaluate images of finished parts acquired post-process as indicated by blocks70and80, respectively. The actual images48may also be collected in a build layer image database49.

In block70, parts P built by AM machine20are sectioned post-process, for example by cutting the part and polishing an exposed sectional surface at a known layer depth, and then capturing a two-dimensional (2D) image74of the exposed surface using an imaging camera. The 2D images74captured post-process may then be evaluated and classified by CNN72. For example, possible classifications76may include under-melt, just right, and over-melt. The post-process 2D image at a given layer depth may be directly related to the associated image48of the layer acquired in-process. This relation may be controlled by a software application programmed to synchronize the data augmentation inFIG. 4to allow the RNN56to be trained on the reconstructed virtual part build from actual data. The number of virtual part builds will be limited only by how much data is available for collection.

The virtual part build aspect of the software application may allow simulations of how a trained RNN56will act using actual data, and may allow integrated computational materials engineering (ICME) models to be improved and/or validated. Additionally, better predictive models may be constructed using the virtual build data to implement advanced control schemes such as model predictive control (MPC) into the fast58loop control schemes illustrated inFIG. 6.

In block80, parts P built by AM machine20are scanned post-process, for example using computer-aided tomography (CAT) equipment, to capture a three-dimensional (3D) image84of the entire part. The 3D images84captured post-process may then be evaluated and classified by CNN82. For example, the classification86may indicate a degree of porosity of the finished part and/or an extent to which voids are present in the finished part.

As mentioned above, in-process build layer images48may be collected in build layer image database49. Other in-process data may also be collected for use in training deep learning AI module56. For example, the fast process melt pool data39acquired in-process by melt pool monitoring system35may be stored in a binary database67, and the sequential time-based data62generated by AM machine20while a layer is being fabricated may be stored in a sequential time-based parameter database68.

As shown inFIG. 5, the data collected as described in connection withFIG. 4may be used as inputs to train deep learning AI module56. The output of CNN46characterizing build layer images48may act as one teacher input provided to deep learning AI module56in a training mode of operation. Similarly, outputs from CNN72and CNN82respectively characterizing post-process images72and82may act as further teacher inputs provided to deep learning AI module56during the training mode of operation. Fast process melt pool data39may be preconditioned by preconditioner64and inputted to deep learning AI module56during the training mode of operation. Sequential time-based data62stored in sequential time-based parameter database68may also be provided as an input to deep learning AI module56during the training mode of operation. The initial AM build parameters41may be provided as a further input to deep learning AI module56during the training mode of operation.

The various inputs to deep learning AI module56should be synchronized correctly to perform the training, and enough data must be available to make the training effective. An output from an LSTM component of deep learning AI module56may be provided to state machine60during the training mode of operation to later facilitate fast-loop closure of the melt pool control when AM system10is operated in a regular production mode. The input to state machine60provides a record that may allow the changing control scheme states (e.g. inFIG. 6) to be evaluated against control simulations to help evaluate the effect of the trained RNN56on the fast control loop58.

Training AI module56using in-process and post-process information as described above will enable reliable determination of whether or not an AM part and corresponding AM process are good from several perspectives associated with good manufacturing practice. The entire set of data for the part build will be captured for the production record. First, the integrity of AM configuration data files used to manufacture a part (i.e. “data integrity”) may be demonstrated and certified. Second, the integrity of the AM process used to build the part (i.e. “process integrity”) may be demonstrated and certified. Third, it may be demonstrated and certified that the process performance generates good parts having high density, minimal or no porosity, and good internal grain structure (i.e. “performance integrity”). By way of analogy, the mentioned process certification for AM parts may be similar to the Design Quality (DQ), Installation Quality (IQ), Operational Quality (OQ), and Performance Quality (PQ) metrics for providing verification and validation evidence that a medical device is functioning correctly to specification. IQ, OQ and PQ are analogous to data, process and manufacturing integrity, respectively. In this case, installation of the correct AM build file is the IQ. Real-time verification that process integrity (OQ) is correct, and near real-time verification that manufacturing integrity (PQ) will come from the in-process and post-process components of the machine learning AI. The measure of goodness would be used by the machine learning AI module56to decide what level of goodness we actually have (through the learned recurrent memory of the non-linear relationship between the in-process measurements and the post-process measurements), and to then make automatic corrections to the process in real time such that goodness (indirectly estimated through non-linear correlation) will be maximized. DQ is equivalent to the AM design rule checks associated with a design/build file, which may integrate ICME for metals or some other physics-based design protocols.

The invention is intended to advance the manufacture of large and complex components by AM methods. This invention would result in higher quality parts made at the additive manufacturing machine and reduce the inspection burden.

While the invention has been described in connection with exemplary embodiments, the detailed description is not intended to limit the scope of the invention to the particular forms set forth. The invention is intended to cover such alternatives, modifications and equivalents of the described embodiment as may be included within the scope of the claims.