12th International Symposium on Turbulence and Shear Flow Phenomena (TSFP12)
Osaka, Japan (Online), July 19-22, 2022

METHODOLOGY TO ACHIEVE ROBUST CLOSED-LOOP TURBULENTFLOW CONTROL USING MACHINE LEARNING

Calin G. Gaina Ghiroaga1 , Matthew R. Singbeil, Chris Morton, Robert J. Martinuzzi
Department of Mechanical and Manufacturing Engineering
University of Calgary
2500 University Drive NW, Calgary, Alberta T2L 1N4, Canada
calin.gainaghiroaga@ucalgary.ca, mrsingbe@ucalgary.ca, chris.morton@ucalgary.ca, rmartinu@ucalgary.ca

ABSTRACT

MPC is able to achieve control, it can be a computationallyexpensive procedure. In particular, MPC must perform an iterative online optimization which involves numerically solving
the model for several candidate actuations in series to determine the optimal actuation based on a cost function. This process is then repeated until the prediction horizon is achieved,
which is the number of time steps the controller is set to predict.

A methodology to achieve robust closed-loop feedback
control of a turbulent flow using machine learning is outlined.
The chosen candidate system is a square cross-sectional cylinder with two moving surface actuators embedded in the windward face at the leading corners. A Long Short-Term Memory (LSTM) Neural Network is trained using motor actuationand pressure sensor data to forecast future system states. This
LSTM model is then implemented with Model Predictive Control (MPC) in order to achieve closed-loop flow control. The
derived controller performance was tested experimentally using three objective functions: recovery of mean-base pressure
set-point from perturbation, and drag or wake fluctuation intensity optimizations. An adaptive learning algorithm, which
adjusts the model to new Reynolds number (Re) conditions
without user intervention, is implemented to extend controller
performance and achieve robust control.

An alternative approach can be to incorporate LSTM networks, a type of RNN whose architecture is more suitable
for long term dependencies. LSTMs do not require numerical solving, can evaluate all candidate actuations in parallel
and can forecast several future states, thus removing all areasof iteration within the MPC framework. In addition to reducing the computational complexity of MPC, LSTMs are a goodcandidate predictive models as they excel at forecasting time
evolution data from smaller training sets when compared to
traditional techniques (Siami-Namini et al., 2018). Therefore,
by using a time series of past system states and actuations as an
input, future system states can be predicted using the LSTM.
Moreover, the LSTM is capable of learning dynamics associated with new Re conditions when they arise, without user intervention, allowing for the model to adapt and achieve robust
control over a Re range.

INTRODUCTION
Control of bluff body flows is an important research area
impacting many industry applications such as efficiency of energy harvesting systems, stability of high-lift devices or dragreduction of high-speed ground vehicles (Choi et al., 2008).
Achieving robust control of fluid flows is challenging as the
response to an actuation is non-linear, thus linear control methods are generally not successful. Additionally, the high dimensionality of the Navier-Stokes equations implies a large parameter space and thus real-time optimization is difficult. Finally,
turbulence increases complexity as fluctuations are stochastic
which effectively is equivalent to reducing signal quality, thus
making model identification more difficult.
The use of machine learning in closed-loop flow control
is appealing as it is a strategy used to identify patterns andrelations within large stochastic datasets. Neural Networks(NN) are machine learning architectures that mimic the interconnected nature of neurons in the brain and are capable offorming input and output relationships through trial and error. Recurrent Neural Networks (RNN) are a type of NN thatspecialize at identifying and correlating patterns through timeby recurrently using earlier outputs as inputs for subsequent
computations. MPC is a control method that uses a predictive model of the system being controlled to optimize future
actuations.
The predictive model is generally a dynamical system of
non-linear ordinary differential equations which need to be numerically solved to achieve predictions of future states. While

Studies that have achieved flow control are usually restricted to a narrow range of Re, and application to turbulent
flows have had limited success. For example, Bieker et al.
(2019) applied a RNN in conjunction with MPC to a computational model of a fluidic pinball system which was able toachieve closed-loop control authority at low Re = 100 − 200,with performance lowering as Re increased. Additionally, thesystem controlled was laminar and computational. This novel
approach aims to achieve robust real-time experimental control of turbulent fluid systems by reducing optimization timeand using high performing predictive model architectures.
In this work, the LSTM-MPC approach is implemented
to control the turbulent wake of a square cross-sectional cylinder. The actuators are rotating cylinders at the leading edges
which are modulated to achieve forcing frequencies which excite the separated shear layer. These shear layer excitations
can manipulate the circulation generation and transport to exploit flow instabilities and alter the wake dynamics. Singbeil
et al. (2021) showed that frequency-based actuation achieves
authority of the flow at lower surface velocity to free-stream
ratios (vsur f /U∞ = 0.2 to 0.4) compared to no forcing frequency (vsur f /U∞ = 3) (Munshi et al., 1997). A model relating cylinder RPM and obstacle surface pressure will be deter-112th International Symposium on Turbulence and Shear Flow Phenomena (TSFP12)
Osaka, Japan (Online), July 19-22, 2022

Figure 1.

a) Side View of Wind Tunnel Facility. b) Obstacle with surface mounted pressure taps (P1 through P6).

MPC Methodology

mined experimentally and used to achieve flow control using
MPC.

MPC was used to achieve closed-loop feedback controlof the system. Closed-loop control works by measuring thesystem output or current system state and feeding it back into
the controller, allowing the controller to improve future actuations. A flow diagram of the entire closed loop control systemand the MPC controller is shown in Fig. 4.
Traditional MPC uses an optimizer which suggests several actuation strategies for the next time step based on thecurrent actuation and the actuation constraints. The suggested
actuations and previous system states are input into a model,
generally a system of ODEs, which gets numerically solved
for each potential actuation. The entire process gets repeateduntil the prediction horizon, N, has been reached. By using a
pre-trained LSTM as the predictive system model, the traditional MPC framework can be altered to reduce computational
complexity since all areas of iteration are eliminated. For thiswork, as an input to the LSTM, the optimizer suggests a libraryof actuations, created a-priori, containing all possible actuations discritized between minimum and maximum actuations.
The LSTM then computes (in parallel) all of the candidate actuations inside this library and outputs the predicted system
state for the entire prediction horizon. Afterwards, the optimizer, through the use of a cost function, locks in an actuation.
The cost function is a relationship comparing the systems statepredictions to a desired state which is optimized in order tochoose the optimal future actuation.
The quantity of interest which will be controlled willbe the leeward face base pressure on the obstacle (Cpbase =(Cp3 + Cp4 + Cp5 + Cp6 )/4). The reason for this is to influence important characteristics that are linked to the leeward
face base pressure such as drag and fluctuating loads on the
obstacle. Thus, the output of the system will be Cpbase , and it
will be controlled with the two rotating cylinders at the leading edges of the obstacle. In order to use MPC to control thisquantity, a model that relates the system actuation, cylinder
RPM, and the system state, Cpbase , must be developed. Cpbase
will be treated as the system state that will be measured andfed back into the controller, where it will be used as an input
to an LSTM, and the output will be future values of Cpbase .

METHODOLOGY
Experimental Facility
The experiments were performed in a small-scale suction
type wind tunnel shown schematically in Fig. 1a. The windtunnel is a = 2510 mm in length consisting of a square crosssectional inlet (b = 690 mm), interchangeable test-section (c= 510 mm), square to the circular cross sectional expansion(d = 1150 mm), and fan shroud (e = 160 mm). Air is drawn
through a honeycomb and four conditioning grids (dscreen =0.22mm, ∆screen = 1.18mm) at the inlet before undergoing a
9:1 area contraction to the square test section with a wetted
cross-sectional area of 203.2 mm × 203.2 mm.
The obstacle is a square cross-section cylinder 3D printed
from PLA (Polylactic Acid) with a characteristic length D =25.4mm and thickness T = 25.4mm (Fig 1b). The obstacle
was printed with the layers being oriented in the freestream
direction, and the surface was sanded to minimize variation insurface roughness. Installed 127.0 mm downstream of the test
section inlet, the obstacle spans the test section for an aspectratio of 8:1 and blockage of 12.5%.
The actuators are two dact = 10 mm diameter rotating
cylinders, embedded as shown in Fig. 1b, at the leading edgesof the square cylinder. Each cylinder was printed from PLA
and a tight tolerance tool steel shaft was inserted through thecenter of the cylinders. Each cylinder was sanded to have surface roughness similar to the obstacle. The cylinders have ninespan wise sinusoidal grooves that have an amplitude of 0.5mm.
The grooves create a forcing frequency equal to nine times therate of rotation. One end of each cylinder was connected tothe drive shaft of a brushless servo-motor mounted external to
the test section. The other end of the steel shaft is supported
externally from the test-section using radially loaded ball bearings. Additionally, the cylinders experience a whirling motion
which introduces a forcing frequency and harmonics of therate of rotation ( fw = RPM/60) (Baek & Sung, 2000). The
whirling arises due to the center of inertia not coinciding withthe axis of rotation. The whirling is beneficial as it introduces
new forcing frequencies that can be exploited.
Six differential pressure transducers monitor surface pressure at locations P1 to P6 indicated in Fig. 1b. On the
top/bottom surfaces, transducers P1 and P2 are 1-inch watercolumn (All Sensors D-4V - Sensitivity = 124 Pa/V) and on
the leeward face P3 to P6 are 5-inch (All Sensors Dx-4v Mini
- Sensitivity = 622 Pa/V) transducers. An Advantech PCIE1812 DAQ was used to measure all the pressure signals, RPM
signals and output RPM signals.

LSTM Methodology
An LSTM was used to create a model of the system which
will be used inside the MPC controller. The LSTM was chosen since it has the capability of making use of history when
given an input sequence to better inform the prediction. A diagram of an LSTM showing how inputs, outputs are handledis shown in Fig. 5. The diagram shows the flow of information through time (horizontally) and how information is remembered and forgotten through forget and input gates. The

2

12th International Symposium on Turbulence and Shear Flow Phenomena (TSFP12)
Osaka, Japan (Online), July 19-22, 2022
0.35

4000

0.3

3500

-0.9

-1

3000

0.25

-1

2500
Cp

0.2
-1.1

-1.1

2000
0.15
1500

-1.2

-1.2

Cp

-1.3

base

RPM1

a)

Cp

set

RPM2

0.1
Cp

-1.3

base

RPM1

b)

Global Max/Min

1000

0.05

RMS of Cp

RPM2

base

RPM1

c)

Global Max/Min

0
0

1000

2000

3000

4000

5000

6000

7000

0

1000

2000

3000
t*

t*

4000

5000

6000

0

200

400

500

RPM2

600

800

1000

1200

0
1400

t*

Figure 2. Motor Actuation Speed (RPM1 and RPM2) and Pressure Response for a) Recovery of mean-base pressure set-point b)
Optimization of Drag through mean Cpbase c) Optimization of wake fluctuation through RMS of Cpbase . Cpbase is defined as (Cp3 +Cp4 +Cp5 +Cp6 )/4. t ∗ is the time non-dimensionalized by the shedding period. The RMS is computed as a windowed average of 17
shedding cycles.

goal of the LSTM model is to accurately forecast future system
states, Cpbase (t), in order for the controller to better optimize
the next actuation (RPM). The LSTM model predicts futuresystem states, Cpbase (t), which are represented as a vector array of size 1 × N, where N is the prediction horizon. The input
to the LSTM is an array of size 3 × M, where M is the input sequence length. The three rows of the input array are thepast system states (Cpbase (t)), the past actuation (RPM), andthe future actuation (Candidate RPM). Past system states andpast actuation were decided upon as inputs as they provide thenecessary history and feedback to predict future system statesin a closed loop control system. The future actuation input isused in order for the optimizer to suggest future actuations,and thus allow the controller optimize the future actuation inorder to achieve the control goal.
The LSTM model is trained offline using time series ofpressure and actuation data. To begin, input and output data arecollected, where the system is randomly actuated between theminimum and maximum RPM = [1000, 3800] for a set timeinterval of 10 minutes, changing RPM every 2 seconds, at aconstant Re = 12500. Pressure and RPM data are collected at asampling frequency of 3000Hz. 50% of the data obtained fromthe data collection are used for training and 50% are saved formodel validation. This split was empirically tested to be suitable and was not optimized. The training data are split into
inputs (3 × M) and outputs (1 × N). The choice of N and M
is important as the LSTM needs sufficient past information tomodel the characteristic time-scale of the flow. The relevanttime scale is that of the vortex shedding cycle-to-cycle variations and it is found that this requires at least two shedding
periods. Each input and output are normalized to be within[-1, 1], by their minimum and maximum values.

and actual pressure by correcting model errors at the current
conditions.
The purpose of transfer learning adaptation is to improve
controller performance for large Re changes. The algorithm
begins by collecting a small set of data at new conditions
which span the entire actuation space. The model is re-trainedusing the pre-trained data as a starting point and adapting to the
new conditions. Through this transfer learning, a higher accuracy can be obtained in a shorter amount of time if the data set
used to pre-train the model exhibit similar system dynamics tothe new data set (Torrey & Shavlik, 2010).

RESULTS
The performance of the closed-loop control wasmeasured by subjecting the controller to: (i) Recovery ofmean-base pressure set-point from perturbation, (ii) drag orwake fluctuation intensity optimization, and (iii) robustness
test at small and large Re changes, and a difficult control
scenario.
(i) Recovery of Mean-base Pressure Set-point from Perturbation
This experiment tests the controller ability to recover
the system state to a specific value when started at a random
condition, and with naturally occurring perturbations whichcome from the turbulent flow development. Additionally,
it tests whether the LSTM is capable of accurately predicting the system dynamics in real time. To confirm thesetwo capabilities, the controller optimizes the cost function(minu∈Rn Cp f uture (t) −Cpset (t) ). Figure 2a shows how thecontroller is able to keep the short term average of the basepressure Cpbase (t) (blue) to the prescribed set-point (red).
The optimized cylinder actuation is shown through RPM1
(yellow) and RPM2 (orange). This confirms that the controller
is capable of recovering the mean-base pressure to a set-pointwith constant perturbations and that the LSTM accurately
predicts system dynamics real time.

Adaptive Learning
A strategy is developed so that the model trained for agiven Re can adapt autonomously to flows at different Re conditions. The algorithm is split into two main functions; a continuous adaptation, and a transfer learning adaptation. The
purpose of the continuous adaptation is to improve controller
performance for small Re changes, and for when the model has
not perfectly captured the dynamics. In order to improve thecontroller performance in these scenarios, the model is continually adapted and trained using feedback data, as seen in
Fig. 4, before the feedback data enters the controller. This
quickly corrects any discrepancies between model prediction(ii) Drag and Wake Fluctuation Intensity Optimization
This experiment attempts to determine the global
optima of the drag or wake fluctuation intensities
through Cpbase (t).
For drag intensity, the mean of
Cpbase (t) is optimized through the cost function:(minu∈RnOR
maxu∈Rn Cp f uture (t) −Cpcurrent (t) ).
Similarly, for wake fluctuation intensity, the RMS
of Cpbase (t) is optimized through the cost function:

3

Motor RPM

-0.9

12th International Symposium on Turbulence and Shear Flow Phenomena (TSFP12)
Osaka, Japan (Online), July 19-22, 2022
(minu∈Rn OR maxu∈Rn CpRMS f uture (t) −CpRMScurrent (t) ).
This problem is traditionally difficult since iteration isnecessary to find global optima, which is computationally expensive (Choi et al., 2008). Sub-optimal control, which stops
iteration early, is faster and less computationally expensive but
can get trapped in local minima and relies heavily on a costfunction. Iteration and sub-optimal solutions are eliminatedby the LSTM since the prediction horizon is computed in one
step and the entire actuation space can be evaluated in one input. By evaluating the entire actuation space, the cost function
is easily optimized to determine the global optima.
The test requires the controller to maximize andthen minimize the mean/RMS of Cpbase (t).
Figure2b shows the max/min (t ∗ = 0 − 2500/2500 − 5000)
mean of Cpbase (t). Next, Figure 2c shows the max/min(t ∗ = 200 − 750/750 − 1300) RMS of Cpbase (t). These two
figures show the controller’s capability to find the global
max/min for both the mean and RMS of Cpbase (t). Figure 6
shows the results of the grid search, confirming the locationsof the optima and proving that the global optima were foundand accurately captured in the model.

-1

-1.05

-1.1

Cp

-1.15

-1.2
Collect Data (30s)

Retraining (60s)

-1.25

-1.3

Cpbase
Cpset

-1.35

0

1000

2000

3000

4000

5000
t*

6000

7000

8000

9000

10000

Figure 3. Closed-Loop Controller Adaptive Learning Test.
Cpbase is defined as (Cp3 + Cp4 + Cp5 + Cp6 )/4. t ∗ is thetime non-dimensionalized by the shedding period.
actuated vortex shedding frequency of 60Hz, the model learns
the new dynamics quickly (60s) based on its experience learning from the training set at a different Re. If trained without
prior learning, it would need at least one order of magnitudelarger training time.

CONCLUSIONS(iii) Control Robustness
One important challenge for robust control is the sensitivity of the system dynamics to changes in Re. The challenge isresolved by adapting the LSTM to unseen conditions. Threetests are done to show how effective the adaptive learning algorithm is. The first test is to use continuous adaptation in aregion in the parameter space that is difficult to control, suchas high gradients of pressure with respect to RPM. Discerning
small pressure differences in large pressure gradient regions isdifficult in real experimental systems due to the compounding
effects of sensor noise and model inaccuracy. Fig. 7 shows
how the controller performs when asked to discern small pressure differences without adaptation on the left and with adaptation on the right. The adaptive learning algorithm improves
controller accuracy in challenging situations as the controller
is better able to regulate the pressure response to the control
goal.
The second test is to use continuous adaptation when
making small Re changes to see if the controller can recover mean-base pressure set-point to a constant value. Without adaptation, this is difficult as dynamics change when Rechanges, and thus the LSTM model will be inaccurate. Figure
8 shows how the controller quickly reacts to a 10% change in
Re and recovers the pressure back to the desired set-point, at
a different Re. With a 10% change in Re, a 100% change in
RPM was needed to maintain the same pressure, which shows
the system sensitivity to Re changes and the necessity for continuous adaptation in real-world systems when Re is not constant.
Finally, a recovery of mean-base pressure set-point test
was run at Re = 22250, a significantly different Re compared
to Re = 12500 which the model was previously trained on, tosee how transfer adaptive learning performs. This procedureincludes collecting a small set of data and using it to retrain themodel on the new conditions. The set-point test is run before
and after re-training to show the difference in performance.
Figure 3 shows how the model initially performs poorly at the
new Re when the controller is asked to regulate Cpbase (t). After adaptive learning, the controller performs well at regulating the pressure to the desired set-point. This shows that themodel has indeed learned the new dynamics associated withthe unseen conditions. The time to train depends on the integral scales of the flow, the size of the actuation space, andhardware implementation. In this implementation, with an un-

A methodology has been presented for achieving closedloop turbulent flow control using Machine Learning. MPC
optimization time was improved through the use of LSTMs,
by eliminating all sources of iteration in traditional MPC.
Recovery of mean-base pressure set-point from perturbation,and drag or wake fluctuation intensity optimizations were performed. In addition, an adaptive learning algorithm was introduced to make the control robust to changing Re and increase
accuracy in difficult to control scenarios. The novelty of thiswork is the development of a methodology which achieves robust closed-loop feedback control of an experimental high Re
turbulent fluid system. This can help advance the field towards
the control of complex systems in industry.
Future challenges to be addressed include increasing actuation degrees of freedom, and increasing MPC cost function
complexity by including actuator power.

REFERENCES
Baek, S. & Sung, H. J. 2000 Quasi-periodicity in the wake of arotationally oscillating cylinder. Journal of Fluid Mechanics 408, 275–300.
Bieker, K., Peitz, S., Brunton, S. L., Kutz, J. N. & Dellnitz, M. 2019 Deep model predictive control with online learning for complex physical systems. arXiv preprint
arXiv:1905.10094 .
Choi, H., Jeon, W. & Kim, J. 2008 Control of flow over a bluff
body. Annu. Rev. Fluid Mech. 40, 113–139.
Munshi, S. R., Modi, V. J. & Yokomizo, T. 1997 Aerodynamics and dynamics of rectangular prisms with momentum injection. Journal of Fluids and Structures 11 (8), 873–892.
Siami-Namini, S., Tavakoli, N. & Namin, A. S. 2018 A comparison of arima and lstm in forecasting time series. In 2018
17th IEEE International Conference on Machine Learning
and Applications (ICMLA), pp. 1394–1401. IEEE.
Singbeil, M., Ghiroaga, C., Morton, C. & Martinuzzi, R. 2021
Moving surface actuation, effects of frequency-based shear
layer excitation on the response of a bluff body wake. In
14th International Symposium on Particle Image Velocimetry, , vol. 1, No. 1.
Torrey, L. & Shavlik, J. 2010 Transfer learning. In Handbookof research on machine learning applications and trends:
algorithms, methods, and techniques, pp. 242–264. IGI
global.

4

12th International Symposium on Turbulence and Shear Flow Phenomena (TSFP12)
Osaka, Japan (Online), July 19-22, 2022

Figure 4. MPC Controller Diagram. The main feedback control loop works by measuring previous system states and actuations, andfeeding them back into the controller, where the controller optimizes future actuations. The adaptive learning loop uses feedback data
to update the model on current conditions to improve robustness and accuracy.

Figure 5.

LSTM Architecture. The LSTM uses forget, input and output gates to control the flow of information.

-0.85

0.4

0.35

-0.9

CpBase

0.25
-1
0.2

RMS of CpBase

0.3
-0.95

-1.05
0.15
-1.1

-1.15
500

0.1

1000

1500

2000

2500

3000

3500

0.05
4000

RPM

Figure 6.

Grid Search Results for Mean and RMS Base Pressure, Cpbase = (Cp3 +Cp4 +Cp5 +Cp6 )/4.

5

-1

-1

-1.05

-1.05

-1.1

-1.1

-1.15

-1.15

-1.2

-1.2

Cp

Cp

12th International Symposium on Turbulence and Shear Flow Phenomena (TSFP12)
Osaka, Japan (Online), July 19-22, 2022

-1.25
-1.3

-1.25
-1.3

-1.35

-1.35

-1.4

-1.4

Cp base
Cp

-1.45

set

Cp baseCp-1.45

Mean of Cp base

-1.5

set

Mean of Cp base

-1.5
0

1000

2000

3000

4000

5000

6000

0

Non Dimensionalized Time (Shedding Cycles)

1000

2000

3000

4000

5000

6000

Non Dimensionalized Time (Shedding Cycles)

Figure 7. Pressure Response of Setpoint Test in High Gradient Location without (left) and with (right) Adaptive Learning. Cpbase isdefined as (Cp3 +Cp4 +Cp5 +Cp6 )/4. t ∗ is the time non-dimensionalized by the shedding period.

104
1.6

Motor Response

2400
2200

1.5
1.4

1800
1600

1.3

1400

Re

Motor RPM

2000

1.2
RPM1
RPM2
Reynolds Number
Re Change

1200
1000
800
0

5000

1.1

1
15000

10000

Non Dimensionalized Time (Shedding Cycles)

Pressure Response
-1
-1.05
-1.1

Cp

-1.15
-1.2
-1.25

Cp base-1.3

Cp set-1.35

Mean of Cp base
Re Change

-1.4
0

5000

10000

15000

Non Dimensionalized Time (Shedding Cycles)

Figure 8. Motor Actuation Speed and Re (top) and Pressure Response (bottom) of Small Re Change Test with Adaptive Learning.
Cpbase is defined as (Cp3 +Cp4 +Cp5 +Cp6 )/4. t ∗ is the time non-dimensionalized by the shedding period.

6