Emulating a PID Controller with Long Short-term Memory: Part 2 | by Nicholas Lewis | Towards Data Science

Welcome to Part 2 of this exciting project! The results have looked great so far, and now we can get into the meat of what we’re trying to accomplish: emulating the behavior of a PID controller with an LSTM. As a recap, here’s what we’ve explored so far and where we’re going:
Using the Temperature Control Lab to create Proportional-Integral-Derivative controller dataTraining a Long Short-term Memory neural network in Keras to emulate a PID controller (this article)Controlling the Temperature Control Lab with an LSTMPractical applications of temperature control using LSTM controller instead of PID controller
Using the Temperature Control Lab to create Proportional-Integral-Derivative controller data
Training a Long Short-term Memory neural network in Keras to emulate a PID controller (this article)
Controlling the Temperature Control Lab with an LSTM
Practical applications of temperature control using LSTM controller instead of PID controller
If you want to run this code on your own, you can find it on Github. It’s heavily dependent on using the TCLab device on your computer, but eventually I plan to generate data and add some notebooks so you can run it as a standalone project as well. Of course, I always encourage you to see what principles you can take from this and apply them to your own project.
After completing the last segment, we have some data to work with, and we want to see if an LSTM can emulate the behavior of the PID controller. There are some excellent articles about how LSTMs work, so I’ll point you to those if you haven’t worked with them before. Here is one of the places I went when looking at some of the math behind them, and this is a fantastic Towards Data Science article that introduced me to how they’re implemented in Python with Keras. Here’s another great tutorial that has a video. As usual, APMonitor.com is a rich resource in all things relevant to machine learning and process control.
LSTMs have been a popular approach to all types of machine learning models because of their versatility. I’ve used them in sequence prediction tasks, natural language processing, and anomaly detection; others have found applications ranging from image processing to speech recognition. What sets it apart from a standard recurrent neural network is the presence of its cell memory unit, which helps to address the vanishing gradient problem.
LSTMs work well on this type of problem because we want to remember what has been happening previously in the control system. If we just barely changed the setpoint from 35°C to 70°C, we know the heater is going to be put on max for a while to get to that setpoint. However, if we just drop from 45°C down to 42°C, or have been holding steady at say 55°C for a while, the controller will have to account differently.
In context of emulating the PID controller, we want to input a window of data, such as the temperature, setpoint, error, or heater values, and predict what the next heater value should be to reach the desired setpoint. This prediction is emulating the output that the PID controller with the given tuning constants would give us. Of course, if the tuning constants change, we’re no longer looking at the same type of controller behavior. That’s an interesting idea to explore in Part 4.
The first step is to look at what features are useful to feed into the model. Intuitively, the PID controller takes the error between the sensor temperature and the setpoint as the input, and so it’s likely that our LSTM will need those. Scikit-learn has lots of useful feature selection models to help us understand some good inputs. Let’s use the SelectKBest method to narrow down our options. We have the sensor temperature, setpoint, and heater output; we’ll also derive the error, and pass these into the SelectKBest function:
# Create new feature: setpoint errordf['err'] = df['Tsp'] - df['T1']# Load possible featuresX = df[['T1','Tsp','err']]y = np.ravel(df[['Q1']])# SelectKBest feature selectionbestfeatures = SelectKBest(score_func=f_regression, k='all')fit = bestfeatures.fit(X,y)plt.bar(x=X.columns,height=fit.scores_)
Plotting the results shows that Tsp (setpoint temperature) and err (error between setpoint and sensor) are the most important features by far, which validates our intuition.
For the LSTM, I’ll be using Keras with a Tensorflow backend; I find this is an intuitive and efficient way of building models. Before building the model, I need to scale the data and format it to how the LSTM expects it. In this case, I’ll take my Tsp and err features, and arrange them in such a way that the past 15 time points are fed as an input to predict the heater output Q1. Be sure to also set aside some data to test the model performance.
X = df[['Tsp','err']].valuesy = df[['Q1']].values# Scale datas = MinMaxScaler()Xs = s.fit_transform(X)# Each input uses last 'window' number of Tsp and err to predict the next Q1X_lstm = []y_lstm = []for i in range(window,len(df)):    X_lstm.append(Xs[i-window:i])    y_lstm.append(y[i])# Reshape data to format accepted by LSTMX_lstm, y_lstm = np.array(X_lstm), np.array(y_lstm)# Split into train and test Xtrain, Xtest, ytrain, ytest = train_test_split(X_lstm,y_lstm,test_size=0.2,shuffle=False)
There are tons of hyperparameters we could look at to optimize the fidelity of the LSTM to the PID controller. I typically will use an extremely useful Python package called Hyperopt to tune these, especially with a really complex problem. Here are some of the hyperparameters that worked well:
window = 15layers = 2batch_size = 100drop = 0.1units = 100
Now, we’re ready to construct the LSTM model. There are lots of tutorials on how to do this, so I’ll go ahead and show the code, then comment on a few highlights.
# Keras LSTM modelmodel = Sequential()if layers == 1:    model.add(LSTM(units=units,                    input_shape=(Xtrain.shape[1],Xtrain.shape[2])                  )             )    model.add(Dropout(rate=drop))else:    # First layer specifies input_shape and returns sequences    model.add(LSTM(units=units,                    return_sequences=True,                    input_shape=(Xtrain.shape[1],Xtrain.shape[2])                  )             )    model.add(Dropout(rate=drop))    # Middle layers return sequences    for i in range(layers-2):        model.add(LSTM(units=units,return_sequences=True))        model.add(Dropout(rate=drop))    # Last layer doesn't return anything    model.add(LSTM(units=units))    model.add(Dropout(rate=drop))model.add(Dense(1))model.compile(optimizer='adam', loss='mean_squared_error')es = EarlyStopping(monitor='val_loss',                   mode='min',                   verbose=1,                   patience=25                  )result = model.fit(Xtrain, ytrain,                    verbose=0,                    validation_split=0.2,                   callbacks = [es,TqdmCallback(verbose=1)],                   batch_size=batch_size,                   epochs=350)# Show resultsepochs = es.stopped_epochplt.semilogy(result.history['loss'],label='loss')plt.semilogy(result.history['val_loss'],label='val_loss')plt.legend();
After constructing the LSTM and Dropout layers, we have a Dense layer with a value of 1, since the model is only predicting 1 output value. We use the adam optimizer and mean_squared_error as the loss function. I like to set aside some of the training data for validation with the validation_split argument. This allows me to monitor the loss on a separate set of data and stop the model training early if my loss plateaus (you can adjust how long you wait after it plateaus with the patience argument), which prevents overfitting the model. The TqdmCallback is a handy progress bar that I like better than the default one that comes with Keras. Finally, it’s always good to plot the loss function to make sure that both the training loss and validation loss show a general decreasing trend. It will take a bit of time to train the LSTM, but we’re not working with huge datasets, so it’s not prohibitive.
Before we use the LSTM to control the TCLab, we want to make sure its behavior approximates what the PID controller would do. Not only is this important for a sanity check, but it’s also an important safety issue. Can you imagine using a temperature controller on a reactor that you weren’t sure would work? If it didn’t work as intended, and there was a runaway reaction, it could cause a big mess.
Luckily, we already have some sample data that we set aside, all in the correct format that the LSTM expects for input. We just need to see that, given the inputs, the predicted heater outputs from the LSTM align with what the PID controller would do. Be sure to undo the scaling of the data so we have the real values.
# Predict using LSTMyp_s = model.predict(Xtest)# Unscale dataXtest_us = s_x.inverse_transform(Xtest[:,-1,:])ytest_us = s_y.inverse_transform(ytest)yp = s_y.inverse_transform(yp_s)# Derive Tsp (setpoint) and T1 (sensor) from X datasp = Xtest_us[:,0]pv = Xtest_us[:,0] + Xtest_us[:,1]
This is what we see:
That’s amazing! You can see the setpoint and T1 data (from which we derived the error), and the real data from the PID controller. The green then shows how the LSTM would behave, given the exact same set of input data. It looks like it follows the PID controller behavior with close fidelity, so I’d be confident trying this out as a proxy controller, with just one adjustment. Notice that the LSTM output sometimes goes beyond the range [0, 100] that the heater is bound to. We’ll have to remember to clip the results when we code it in as the controller.
We’ve got a working LSTM model, and the next step is to code it in as the controller. That’s a pretty big project, so we’ll save that for the next post in the series (coming October 2020). Until then, just be sure to save your model and any parameters you used for preprocessing.
# Save Keras LSTM modelmodel.save('pid_emulate.h5')# Save preprocessing parametersmodel_params = dict()model_params['Xscale'] = s_xmodel_params['yscale'] = s_ymodel_params['window'] = windowpickle.dump(model_params, open('model_params.pkl', 'wb'))
And there we have it. We successfully trained an LSTM model to emulate the behavior of a PID controller. The next big step is to put it to use.