Emulating a PID Controller with Long Short-term Memory: Part 1 | by Nicholas Lewis | Towards Data Science

Do you ever just get really excited about an idea? Maybe it’s a new DIY project you’re tackling, or a cool assignment at work. Maybe you’re crazy like me and want to hike the Pacific Crest Trail (as I’m moving to Seattle soon, so I can’t help but get excited about the idea of flying down to San Diego and walking home). Well, this project is one of those types of ideas for me, and I hope you enjoy the ride!
Before I get started, though, I want to warn you that this is quite an extensive project, and so I’m breaking it up into parts. They will be as follows:
Using the Temperature Control Lab to create Proportional-Integral-Derivative controller data (this article)Training a Long Short-term Memory neural network in Keras to emulate a PID controllerControlling the Temperature Control Lab with an LSTMPractical applications of temperature control using LSTM controller instead of PID controller
Using the Temperature Control Lab to create Proportional-Integral-Derivative controller data (this article)
Training a Long Short-term Memory neural network in Keras to emulate a PID controller
Controlling the Temperature Control Lab with an LSTM
Practical applications of temperature control using LSTM controller instead of PID controller
While working on a project for work one day, I came across a paper that introduced a novel idea. Some researchers had the idea to emulate the behavior of a model predictive control (MPC) strategy using a neural network. The idea was that some MPC problems are extremely computationally expensive, but the behavior is predictable; if we could emulate that behavior by training a deep neural network, we could, in theory, replace the computationally expensive MPC with a much faster machine learning model. The only computationally expensive part of the process would be the model training.
The really neat part of this is that I test the idea here at my desk. The Temperature Control Lab is perfectly designed to prototype this kind of problem. I introduced the TCLab and some basic anomaly detection you can do with machine learning in an earlier post. If you’re new to the TCLab, you can check out my article, or look at the abundant resources at APMonitor.com.
So here’s what I decided to try: I wanted to tune a Proportional-Integral-Derivative (PID) controller so I could set a temperature on the TCLab and the controller would adjust the heater until it hit that setpoint. Then, I wanted to see if I could emulate this behavior with a Long Short-term Memory neural network. If this prototype worked, it would open the doors for not only improving computational speed, but also applications such as anomaly detection.
This series is about the machine learning, so if you’re not familiar with PID controllers or control theory, that’s perfectly fine; we’ll just hit a few highlights so you understand the context. There are some amazing resources at APMonitor.com to help you understand PID controllers and how to tune them. These controllers are ubiquitous in engineering industry, and so the potential applications for this are numerous.
Control systems are everywhere. Think about your thermostat: all you have to do is plug in the temperature, and somehow it knows when to turn the heat on and off to keep your house at that temperature (except, of course, for the room that’s always an ice box...). How does it do that? Control systems can be broken down into three parts: a sensor (something that reads the temperature in the house), an actuator (the A/C or furnace that turns on cold or hot air), and a controller (the computer in your thermostat that is the brains behind the operation). The sensor data feeds into the controller, which then tells the actuator what to do.
PID controllers are ubiquitous for a reason: they’re straightforward and extremely effective, especially when paired with a first order plus dead time (FOPDT) model. In the case of our TCLab, it takes the difference between the temperature setpoint and temperature sensor (represented as e(t) in the equation below) as an input, and outputs a heater value (labeled output, or OP, below) that will allow the temperature to approach the setpoint. As the name suggests, there is a proportional, integral, and derivative portion to this.
K_c, τ_I, and τ_D are called the tuning constants, and determine how the controller behaves. You can generate a step response for your TCLab, and then obtain tuning parameters for a PID controller graphically. For my TCLab, some effective tuning parameters were K_c = 6.0, τ_i =75.0, and τ_d = 0.0. Yours will likely be different since each TCLab behaves slightly differently. Let’s take a look at the code now for the PID controller. I put it in the function pid(sp,pv,pv_last,ierr,dt) .
The first part is to load your tuning parameters and other constants:
# PID ParametersKc   = 6.0tauI = 75.0 # sectauD = 0.0  # sec# PID coefficients in terms of tuning parametersKP = KcKI = Kc / tauIKD = Kc * tauD# OPbias for controller (initial heater)op0 = 0# upper and lower bounds on heater levelophi = 100oplo = 0
Then we want to calculate the error between the setpoint (sp) and temperature (also referred to as process variable, pv):
# calculate the errorerror = sp - pv
The integral portion is calculated numerically, adding whatever the previous error was to the current time step error:
# calculate the integral errorierr = ierr + KI * error * dt
And finally, the derivative portion of the controller (also calculated numerically) looks like this:
# calculate the measurement derivativedpv = (pv - pv_last) / dt
Putting it all together now, we can calculate the PID output (op):
# calculate the PID outputP = KP * errorI = ierrD = -KD * dpvop = op0 + P + I + D
And finally, just to make sure we account for the heater limit between 0 and 100, we add an anti-reset windup condition:
# implement anti-reset windupif op < oplo or op > ophi:    I = I - KI * error * dt    # clip output    op = max(oplo,min(ophi,op))
That’s not so bad! If nothing else, you learned how to program a PID controller function, which is extremely useful for industrial controllers.
Now we need to actually run the PID controller on the TCLab. We’ll set up several arrays to store data as we go, and run the controller for 90 minutes to make sure we have plenty of data. We’ll change the temperature setpoint intermittently so we have a good mix of steady-state and transient behavior. Here’s what my setup looks like:
##### Set up run parameters ###### Run time in minutesrun_time = 90.0# Number of cyclesloops = int(60.0*run_time)# arrays for storing dataT1 = np.zeros(loops) # measured T (degC)error_sp = np.zeros(loops) # Setpoint errorQ1 = np.zeros(loops) # Heater valuestm = np.zeros(loops) # Time# Temperature set point (degC)with tclab.TCLab() as lab:    Tsp1 = np.ones(loops) * lab.T1# vary temperature setpointend = 30 # leave 1st 30 seconds of temp setpoint as room tempwhile end <= loops:    start = end    # keep new temp set point value for anywhere from 4 to 10 min    end += random.randint(240,600)     Tsp1[start:end] = random.randint(30,70)
It never hurts to plot what you expect should happen. Over the course of 90 minutes, we intend for the PID controller to set the heater such that the temperature follows this profile:
We’re all set! I went into more details of how to set up a run on the TCLab in another article I wrote, but this application is slightly more advanced. Most notably, we call our pid(sp,pv,pv_last,ierr,dt) function at each time step to return the heater output so that we stay at the right temperature. We’re updating this every second; the PID isn’t too computationally expensive, so it can easily solve within that time.
# Run testwith tclab.TCLab() as lab:    # Find current T1, T2    print('Temperature 1: {0:0.2f} °C'.format(lab.T1))    print('Temperature 2: {0:0.2f} °C'.format(lab.T2))    # Integral error    ierr = 0.0    # Integral absolute error    iae = 0.0    start_time = time.time()    prev_time = start_time    for i in range(loops):        # Delay 1 second        if time.time() > prev_time + 1.0:            print('Exceeded cycle time by ',                   time.time()-prev_time-1.0)        else:            while time.time() < prev_time + 1.0:                pass                # Record time and change in time        t = time.time()        dt = t - prev_time        prev_time = t        tm[i] = t - start_time        # Read temperatures in Kelvin         T1[i] = lab.T1        # Integral absolute error        iae += np.abs(Tsp1[i]-T1[i])        # Calculate PID output        [Q1[i],P,ierr,D] = pid(Tsp1[i],T1[i],T1[i-1],ierr,dt)        # Write heater output (0-100)        lab.Q1(Q1[i])
Let’s take a look at how the controller did:
Pretty neat! We want the red line, which is the temperature sensor, to follow the black line, which is the temperature setpoint. The blue shows how the heater adjusted so that the sensor would match the setpoint. We could always tune the PID parameters for different performance, but we want to get to the machine learning part, so for this exercise, this is perfect. And I don’t know about you, but we haven’t even started the machine learning, but just programming our own controller at our desk is already exciting!
First things first, you can download the code for this from Github. This exercise depends a lot on the TCLab, but I’m hoping in the future to add some alternative notebooks with raw data so that you can run the code without one.
How neat is it that we can program a device to control the temperature? There are so many applications — think of your thermostat, or oven, or even a fancy barbeque grill. Not to mention the industrial applications of reactor temperature or feed stream temperature. What other applications can you think of for this type of controller?
Thanks for bearing with me through this introductory article, especially if you’ve never seen a controller. I know I promised some machine learning applications, and that’s exactly what we’re working up to. We’ll dive into that first thing in the next section (coming October 2020). If anything, though, I feel that it’s important to understand the background behind machine learning applications. Not only does it make them relevant to our work or daily lives, but it also opens up the door for greater insights. I’m happy to hear what ideas this might open up for you!