Emulating a PID Controller with Long Short-term Memory: Part 4
By: Nicholas Lewis
Tags: Lstm, Machine Learning, Pid Controller, Python, Lstm Control Emulation

Here we are in the last section of this fun project! Here’s where we’ll see some practical applications of using the LSTM to emulate the PID controller, as well as some potential shortcomings. If you haven’t read the previous articles in the series, I highly recommend going back so you can have some context. And of course, the neat thing about this project is that you can run all the code on your own Temperature Control Lab device, simulating something that you might see in a real refinery or chemical plant. So here’s a quick recap of this series, then we’ll get started!

Changing PID tuning parameters

If you’re familiar with PID controllers, the first thing you may have been wondering during the problem setup is what happens if you change the tuning parameters? Recall the PID controller equation:

If we change K_c, τ_I, or τ_D, the heater output changes, and then the LSTM that we trained no longer matches the output. One could look at this as a huge downside — if we change the tuning parameters for the PID controller, we’d have to go to all the work to retrain the LSTM to emulate the new behavior (back to Part 2). On the other hand, we could use this to our advantage. Once a PID controller is tuned to a system, it’s rare that we’d need to change the tuning parameters, so changes to the PID behavior might indicate a malicious attack from an outsider (or potentially an insider such as a disgruntled employee).

Suddenly, we have an anomaly detection method out of the box. We can do what we did as a check in Part 3, where we run the controller off the PID, but also check the LSTM output to make sure they’re similar. If they’re not, then we know something has set the PID controller out of whack, and we can investigate to fix it.

Let’s take a look at our setup, which should be familiar by now, but with a few changes:

#### Set up run #### # Import model and model parameters

model = load_model('pid_emulate.h5')

model_params = pickle.load(open('model_params.pkl', 'rb')) s_x = model_params['Xscale']

s_y = model_params['yscale']

window = model_params['window'] # Run time in minutes

run_time = 45.0 # Number of cycles

loops = int(60.0*run_time) # arrays for storing data

T1 = np.zeros(loops) # measured T (degC)

Qpid = np.zeros(loops) # Heater values for PID controller

Qlstm = np.zeros(loops) # Heater values for LSTM controller

tm = np.zeros(loops) # Time

t_pid = np.zeros(loops) # Time to compute PID controller output

t_lstm = np.zeros(loops) # Time to compute LSTM controller output

Q2 = np.zeros(loops) # Time range to introduce anomaly (turn on heater 2, change PID tuning constants)

start_anom = int(0.7*loops) # Heater 2 turned on during anomaly window

Q2[start_anom:] = 80 # Temperature set point (degC)

with tclab.TCLab() as lab:

Tsp = np.ones(loops) * lab.T1 # vary temperature setpoint

end = window + 15 # leave 1st window + 15 seconds of temp set point as room temp

while end <= start_anom:

start = end

# keep new temp set point value for anywhere from 4 to 10 min

end += random.randint(240,600)

Tsp[start:end] = random.randint(30,70)



while end <= loops:

start = end

# keep new temp set point value for anywhere from 4 to 10 min

end += random.randint(240,600)

Tsp[start:end] = random.randint(30,50)

As usual, we import our model and accompanying parameters, set a run time, initiate the arrays to store data, and create a setpoint profile. There are also a few things to point out. We have t_pid and t_lstm , which we’ll be using later on to time the controllers. We have the start_anom variable, which I use to indicate when the anomalous section of data should begin. Notice that I also setup an array for heater 2 ( Q2 ), which we’ll be using for another test later on. And finally, I kept the setpoint for the heater a bit lower during the anomalous section of data — it will work with higher temperatures, but it’s easier to see when the temperature stays lower.

I also made a change to the pid(sp,pv,pv_last,ierr,dt) function, so it now takes the tuning “constants” as additional inputs (Kc,tauI,tauD) . This will let us change the PID controller output mid-run.

And finally, here’s what the code will look like as we run the simulation:

# Run test

with tclab.TCLab() as lab:

# Find current T1, T2

print('Temperature 1: {0:0.2f} °C'.format(lab.T1))

print('Temperature 2: {0:0.2f} °C'.format(lab.T2)) # Integral error

ierr = 0.0

# Integral absolute error

iae = 0.0 start_time = time.time()

prev_time = start_time for i in tqdm(range(loops)):

# Delay 1 second

if time.time() > prev_time + 1.0:

print('Exceeded cycle time')

else:

while time.time() < prev_time + 1.0:

pass



# Record time and change in time

t = time.time()

dt = t - prev_time

prev_time = t

tm[i] = t - start_time # Read temperature (C)

T1[i] = lab.T1 # Integral absolute error

iae += np.abs(Tsp[i]-T1[i]) # Perturb PID tuning parameter

if i > start_anom:

Kc, tauI, tauD = 3.0*Kc0, 0.5*tauI0, tauD0 + 2.0

else:

Kc, tauI, tauD = Kc0, tauI0, tauD0



# Calculate PID output (and time)

t0_pid = time.time()

[Qpid[i],P,ierr,D] = pid(Tsp[i],T1[i],T1[i-1],ierr,dt,

Kc=Kc,tauI=tauI,tauD=tauD)

tf_pid = time.time() # Write heater output (0-100)

lab.Q1(Qpid[i])



# Run LSTM model to get Q1 value for control

if i >= window:

# Load data for model

T1_m = T1[i-window:i]

Tsp_m = Tsp[i-window:i]

# Predict and store LSTM value for comparison

t0_lstm = time.time()

Qlstm[i] = lstm(T1_m,Tsp_m)

tf_lstm = time.time()



# Save controller times

t_pid[i] = tf_pid - t0_pid

t_lstm[i] = tf_lstm - t0_lstm

There’s a lot in there, but it should look familiar from Part 3. The main difference is the line where we change the PID tuning parameters: Kc, tauI, tauD = 3.0*Kc0, 0.5*tauI0, tauD0 + 2.0 when our time count goes beyond our start_anom specification. You’ll also notice a few time recordings in there, which we’ll address in the next section.

After letting this run, we can plot the results and put them into a video to visualize. Let’s see what happened!

As we saw in Part 3, the LSTM output closely resembles that of the PID, with obvious exceptions during the transient timeframe. This abruptly changes, as expected, when the PID tuning parameters change. You’ll notice that since the PID controller is the one actually writing the output, that the heater output also starts to go erratic, and the temperature starts to fluctuate a lot more.

We could easily plot the error between Qpid and Qlstm and use that to detect anomalous behavior. It’s also convenient to have a backup in this case; if we notice that the PID controller starts behaving oddly, we can quickly switch control over to the LSTM controller, just by changing lab.Q1(Qpid[i]) to lab.Q1(Qlstm[i]) . Pretty neat that what initially might be thought of as a shortcoming actually turns out to be quite useful!

Computation Time

If you recall, the primary reason we started this project was from a paper talking about using a neural network to emulate a model predictive controller, with the idea that the neural network would be faster than the controller. From the last run where we were computing both the LSTM and PID controller, we conveniently saved the time each one took. Let’s see what the results are:

print('LSTM:',t_lstm.mean())

print('PID:',t_pid.mean()) >>> LSTM: 0.03118442685515792

PID: 2.043927157366717e-05

Well, that’s honestly what I expected. The PID is actually significantly faster than the LSTM, although the LSTM controller still computes in well under 1 second. There’s a reason that PID controllers are used, and that’s because they’re fast and perform quite well for several control problems. If you have a more complex controller, though, we might expect the LSTM controller to still only take a fraction of a second for each computation, assuming the model inputs and parameters are similar. So while it actually has a time disadvantage over a PID controller, there are still some uses (check back for a bonus article coming November 2020 about a more complex controller and how the LSTM controller could be advantageous in that case).

Using Heater 2 to simulate additional anomalies

Finally, let’s take another look at how the LSTM controller could be used to detect other types of anomalies. We’ll simulate a situation where the ambient conditions are no longer standard in the refinery. Maybe this is from something as benign as different weather, or something as malicious as a cyber attack that is targeting another part of the plant (or even this specific process). How would the controller behave differently?

Let’s simulate this anomalous event by turning on heater 2 for part of the time. While heater 2 is physically separated from the sensor on heater 1, it is still close enough to affect the conditions around heater 1 which we’re trying to control. Our LSTM controller is trained on a specific set of data that was operating under standard room temperature, so if we turn on heater 2, we have a situation that the LSTM controller isn’t trained to control. Similar to the case of changing the PID tuning parameters, this initially seems like a downfall of using the LSTM controller. However, if we have a carefully controlled ambient environment in our plant, then we’d expect the controller to behave consistently; any deviation from that would indicate something anomalous is happening.

Here’s a look at the code for the run (recall that we already specified the Q2 parameters in the initial setup).

# Run test

with tclab.TCLab() as lab:

# Find current T1, T2

print('Temperature 1: {0:0.2f} °C'.format(lab.T1))

print('Temperature 2: {0:0.2f} °C'.format(lab.T2)) start_time = time.time()

t = start_time for i in tqdm(range(loops)):

# Delay 1 second

if time.time() > t + 1.0:

print('Exceeded cycle time by ',time.time()-t-1.0)

else:

while time.time() < t + 1.0:

pass # Record time and change in time

t = time.time()

tm[i] = t - start_time # Read temperature (C)

T1[i] = lab.T1 # Run LSTM model to get Q1 value for control

if i >= window:

# Load data for model

T1_m = T1[i-window:i]

Tsp_m = Tsp[i-window:i]

# Timer for LSTM controller output

t0_lstm = time.time()

# Predict and store LSTM value for comparison

Qlstm[i] = lstm(T1_m,Tsp_m)

tf_lstm = time.time() # Write heater output (0-100)

lab.Q1(Qlstm[i])

lab.Q2(Q2[i])

This probably all looks familiar by now, with the notable exception of the last two lines: we’ve turned over control to the LSTM, and we also read the Q2 value in. After running the simulation, we again can plot the results and put them into an animation.

Very interesting. Notice that once we turn on Q2, the temperature tends to overshoot the setpoint, and the controller isn’t able to properly account for the overshoot. That’s because the controller isn’t trained to account for extra heat coming from the nearby heater.

That gives me an idea, but I think I’ll let you try it out, now that you know how to set up the TCLab, train an LSTM, and run a temperature control simulation with it. Could you use the skills you learned from this series to set up a control system that turns on both of the heaters and tries to control both temperatures to a certain setpoint? This would be a Multiple Input, Multiple Output (MIMO) controller. For a prod in the right direction, visit the MIMO lab at APMonitor.com.

Final Thoughts

That brings us to the conclusion of this project. We started with a basic PID controller, generated some data, used that data to train an LSTM to emulate the PID controller, turned over control to the LSTM, and then looked at some useful applications such as detecting various anomalies and potential computation time savings.

So how did you do? Did you get excited about this project like I did? What other applications can you think of for this? I’m happy to hear suggestions for areas that could use more clarification. Thanks for following along on this project, and I look forward to future interactions. If you enjoyed this, feel free to connect with me on LinkedIn.