# Data-Driven Approximated Optimal Control of Sulfur Flotation Process

Authors: Mingfang He
Journal: Complexity
 (2019)

---

## Abstract

Sulfur flotation process is a typical industry process with complex dynamics. For a sulfur flotation cell, the structure of the system model could be derived using first-principles and reaction kinetics. However, the model parameters cannot be obtained under certain working conditions. In this paper, by using adaptive dynamic programming (ADP), we establish a data-driven optimal control approach for the operation of a sulfur flotation cell without knowing the model parameters. By learning from the online production data, an initial admissible control policy iteratively converges to an approximated optimal control law, and the dependence of optimal control design on the full model knowledge is eliminated. A simulation environment of sulfur flotation process is constructed based on phenomenological model and industrial data. Some practical problems in the implementation of ADP, i.e., selection of basis functions, how to use the model structural information in the ADP-based control design, are investigated. The feasibility and performance of the proposed data-driven optimal control are tested in the simulation environment. The results indicate the potential of applying bioinspired control methods in flotation process.


---

## Body

## 1. Introduction

Sulfur is an important element with wide applications in pharmacy industry, chemical engineering, light industry, and food industry, to name a few. In most cases, sulfur exists in the form of sulfide or sulfate minerals. So sulfur flotation process is usually integrated into mineral processing and hydrometallurgy plants to recover the valuable sulfur from the mineral residue as a secondary product. On the other hand, by using sulfur flotation, more advanced and economic technologies can be applied in metallurgy plants, e.g., direct leaching technology. The grade and recovery are two main technical indexes which define the overall performance of a sulfur flotation process. The operation objective of a sulfur flotation process is to recover as much as high-graded sulfur as possible from the mineral residue by finding the optimal combination of the manipulated variables in a real-time manner.Sulfur flotation is a typical three-phase (solid-liquid-gas) process. Generally, the operation of this type of process is nontrivial due to its complex and nonlinear dynamics which is a synthetical outcome of the intricate interactions between the reactants, inlet conditions, manipulated variables, and disturbances. Therefore, the economical operation of flotation processes has aroused the interest from both mineral processing community and control community [1–3].The flotation modeling approaches can be categorized into first-principle modeling [4] and soft-computing-based modeling [5]. Lynch et al. [6] and Gorain et al. [7] developed empirical models for the entire flotation plant and subprocesses in a flotation system. Yianatos proposed a first-order kinetic flotation model [8]. Koh and Schwarz studied the probabilistic modeling of the microprocesses, e.g., adhesion of particles to the bubbles in the pulp, in the flotation [9]. The flotation rate constant in the model varies with feeding conditions and working conditions. Therefore, to precisely determine the flotation rate constant remains an active research topic [4]. Bascur proposed a phenomenological model which considers both the particle population balance and hydraulic behavior [10]. The phenomenological model combined population balance, transfer of particles between phases and liquid balance, and is able to simulate the effects of manipulated variables (e.g., tailing and air flow rate) on a flotation process [2].In flotation plants, the operation of a flotation process is conducted in a “setting and control” manner. The basic control loops maintain the primary variables at set-points [2], which is achieved by conventional PID control. On a higher level, machine vision-based control systems, model-based control methods, e.g., Model Predictive Control (MPC), adaptive control, expert systems, neural network, fuzzy-logic, and their commercial software are developed and applied [11]. The performance of the above model-based controllers relies on the model quality. Although the structure of a mathematical model could be obtained by applying physical/chemical principles and reaction kinetics, the model parameters vary with the working conditions and are not available under certain working conditions. Robust control approaches are the natural choice for these working conditions with modeling uncertainties. However, in some cases, the conservative feature of the robust control approaches keeps the derived controller away from optimal which in turn leads to economical loss.In recent years, bioinspired control approaches have been wide-spreading. Learning is the basic feature of bioinspired control methods. It obtains the optimal or nearly optimal controller by learning from the “excitation-response” interactions with the environment [12], of which “try and error” is a typical example. Reinforcement learning (RL) is a bioinspired machine learning method. It is originally observed from the learning behavior in mammals [13]. RL represents how an agent/animal interacts with the unknown environment and modifies its action to obtain a maximized reward. Adaptive dynamic programming is an integration of adaptive control, dynamic programming, and reinforcement learning [14, 15]. By using an “actor-critic” structure, it can approximate the solution of the Hamilton-Jacobi-Bellman (HJB) equation online without knowing the full knowledge of the system model, or even when the system model is unknown [16, 17]. Thus, if an initial admissible control policy is available, an approximated optimal control can be obtained by solving the HJB equation iteratively. The industrial applications of ADP include oil production [18], coal gasification [19], and energy scheduling [20], to name a few. In recent years, learning-based control has been applied in the operation and control of froth flotation process. In [21, 22], a dual-rate and a two-layer control strategy based on Q-learning and off-policy reinforcement learning were proposed and tested via experimental study, respectively.The aim of this study is to design an alternative optimal control approach for a sulfur flotation cell when the model parameters are unknown and the manipulated variables are constrained. The rest of this paper is organized as follows. In Section2, the classical flotation kinetic model developed by Bascur [10] is introduced and used in the analysis of the optimal control problem in Section 3. Then the model is adopted to simulate the real dynamics of a flotation cell in Section 4. An optimal control based on ADP is designed to iteratively improve the performance of an initial admissible controller by learning from the online operational data generated from the simulation environment. The performance of the proposed optimal control is compared with the traditional PI controller. The simulation results, advantages, and shortcomings as well as future extensions of the proposed control are analyzed in Section 6.


## 2. Process Description

Typically, a sulfur flotation process is composed of three stages with specific functions, namely, roughing, scavenging, and cleaning (Figure1). The roughing stage performs the initial separation of sulfur from gangue. Recovery rate is the main concern of roughing stage. The tailing of roughing stage is delivered to scavenging stage to seek for the remaining sulfur. The tailing of scavenging stage is also the final tailing of the entire flotation process. The froth of roughing stage overflows to the cleaning stage. The scavenging froth as well as the cleaning tails are returned to the roughing stage. The overflow of the cleaning stage is the final product, i.e., the sulfur concentrate. The three stages work collaboratively to maximize the sulfur recovery and concentrate grade and minimize the grade of final tails.Figure 1
Flow-diagram of a sulphur flotation circuit.The economical production of a flotation process relies on the optimal control of a single flotation cell which is the basic flotation unit. As shown in Figure2, the operation of sulfur flotation in a flotation cell is based on the natural hydrophobicity of sulfur. First, slurry containing both valuable sulfur and base gangue is fed into the flotation cell. Then, by blowing air into the flotation cell, the sulfur particles in the slurry are attached to the bubbles. The bubbles float upward and carry the particles to the froth layer. Finally, the sulfur particles are collected in the concentrate launder via the natural overflow of the froth. Focused on a single flotation cell, the rest of this section presents the model structure which is then used along with the flotation mechanism to analyze the optimal control problem.Figure 2
Structure of a sulphur flotation cell.


### 2.1. Flotation Behavior Analysis

The physical activities in a sulfur flotation cell involve the movement of solid particles driven by the flow of air and liquid. As illustrated in Figures2 and 3 [10], considering the different roles of pulp and froth in determining the cell dynamics, the flotation cell is divided into two sections: froth volume and pulp volume. The froth volume and pulp volume are then subdivided in the liquid and air phases with which the mineral particles are associated. The states and movements of particles in a flotation cell mainly include the following:(i)
SLP: free in the pulp (particles in the liquid phase of the pulp volume)(ii)
SBP: attached in the pulp (particles attached to the bubbles in the pulp volume)(iii)
SLF: free in the froth (particles in the liquid of the froth volume)(iv)
SBF: attached in the froth (particles attached to the bubbles of the froth volume)Figure 3
Particle movement in a flotation cell.and(i)
SLP→SBP: free particles in the pulp attached to the bubbles in the pulp volume with the attachment rate constant kijPAT(ii)
SBP→SLP: particles associated with the bubbles detached and enter the liquid phase in the pulp volume with the detachment rate constant kijPDT(iii)
SLF→SBF: free particles in the froth layer attached to the bubbles in the froth volume with the attachment rate constant kijFAT(iv)
SBF→SLF: particles associated with the bubbles detached and enter the liquid phase in the froth volume with the detachment rate constant kijFDT(v)
Feed→SLP: the particles in the input slurry (with liquid volume flow rate QFeed and ΨijFeed the number of particles of size i and mineralogical species j per unit volume) feed into the flotation cell(vi)
SLF→SLP: particles draining from the liquid phase of the froth volume (with the flow rate of QR and ΨijLF the number of particles of size i and mineralogical species j per unit volume of liquid in the froth volume, kijR a constant indicating the segregation mechanism in the draining)(vii)
SLP→SLF: particles entrained by the liquid flow from pulp volume to the froth volume (with the flow rate of QE)(viii)
SBP→SBF: transport of particles attached on the bubbles in the pulp volume to the froth volume (with the air flow rate of QA′)(ix)
SLP→Tailings: the particles in the pulp leave through the tailing port (with the flow rate of QT and ΨijLP the number of particles of size i and mineralogical species j per unit volume of liquid in the pulp)(x)
SBP→Tailings: exit of particles attached on the bubbles in the pulp volume to the tailings (with the aeration rate of QAT)(xi)
SLF→Concentrate: the particles free in the froth overflow to the concentrate lander (with the flow rate of QC)(xii)
SBF→Concentrate: the particles attached to the bubbles in the froth overflow to the concentrate lander (with the flow rate of QAC)


### 2.2. Phenomenological Model of a Flotation Cell

Based on the above simplifications, consider the hydraulic behavior, and apply the following assumptions:Assumption 1.
Each phase in the pulp and froth volume of the flotation cell is perfectly mixed.Assumption 2.
The particle size and shape are uniform.Assumption 3.
The changes in the air volumes occur much faster than the liquid volumes, and steady state relations are used; i.e.,QA=QA′+QAT, QA′=QAC [10, 23].then a process model, which is a combination of the particle population balance in the liquid phase and the air phase for the pulp and froth volume, is built to describe the dynamics of a flotation cell:(1a)dCjPdt=QFeedVLPCjFeed-QE+QAαjPVLP/VBP1+αjPVLPCjP+QRkjR1+αjFVLPCjF-QT1+αjPVLPCjP(1b)dCjFdt=QE+QAαjPVLP/VBP1+αjPVLFCjP-QRkjR1+αjFVLFCjF-QTαjP1+αjPVLFCjP-QAVBFCjF+QTVBPVBFVLPCjF(1c)dVLPdt=QFeed-QT-QE+QR(1d)dVLFdt=QE-QR-QC(1e)dhdt=VLPS2VLP-ShQFeed-QT-QE+QRwhere (1a) and (1b) present the mass balance for mineralogical species j, while (1c) and (1d) describe the liquid volume balance at the pulp volume and froth volume, and(2)VBP=Sh-VLP(3)VBF=V-Sh-VLF(4)QC=VLFVBFQA-VBPVLPQTThe model (1a), (1b), (1c), (1d), and (1e) is a combination of the froth flotation models proposed in [10, 24]. It keeps the phenomenological behavior in [10], while it also approximates the dynamics of the pulp level in [24]. The physical meanings of the model parameters and the variables in Figure 3 are listed in Table 1. In Section 5, this model is applied to simulate the real flotation cell.Table 1
List of model parameters and process variables.
Parm Unit Physical meaning Q F e e d m 3/h Liquid volume flow rate of the slurry feed Q C m 3/h Liquid volume flow rate of the concentrate Q T m 3/h Liquid volume flow rate of the tailings Q E m 3/h Liquid volume flow rate of the entrainment from pulp to froth volume Q R m 3/h Liquid volume flow rate of the drainage from froth to pulp volume Q A m 3/h Air flow rate of the aeration Q A ′ m 3/h Air flow rate that leaves the pulp volume into the froth volume Q A C m 3/h Air flow rate that leaves the froth volume into the concentrate launder Q A T m 3/h Air flow rate that leaves the pulp volume into the tailings V m 3/h Volume of the flotation cell V T F m 3/h Volume of the froth layer in the flotation cell V L F m 3/h Volume of liquid phase in the froth layer V B F m 3/h Volume of air phase in the froth layer V T P m 3/h Volume of the pulp layer in the flotation cell V L P m 3/h Volume of liquid phase in the pulp layer V B P m 3/h Volume of air phase in the pulp layer C j F e e d t/m3 Concentration of speciesj in the liquid volume of the feed slurry C j F t/m3 Concentration of speciesj in the liquid volume of the froth layer C j P t/m3 Concentration of speciesj in the liquid volume of the pulp layer Ψ i j F e e d - Number of particles of sizei and mineralogical species j per unit volume in the feed Ψ i j L F - Number of particles of sizei and mineralogical species j per unit volume of liquid phase in the froth layer Ψ i j B F - Number of particles of sizei and mineralogical species j per unit volume of air phase in the froth layer Ψ i j L P - Number of particles of sizei and mineralogical species j per unit volume of liquid phase in the pulp layer Ψ i j B P - Number of particles of sizei and mineralogical species j per unit volume of air phase in the pulp layer k i j P A T - Rate constant of particles of sizei and mineralogical species j attach from the pulp to the bubble in the pulp volume k i j P D T - Rate constant of particles of sizei and mineralogical species j detach from the bubble to the pulp in the pulp volume k i j F A T - Rate constant of particles of sizei and mineralogical species j attach from the pulp to the bubble in the froth volume k i j F D T - Rate constant of particles of sizei and mineralogical species j attach from the bubble to the pulp in the froth volume α j F - Rate constant of the attachment and detachment for speciesj in the froth α j P - Rate constant of the attachment and detachment for speciesj in the pulp k j R - Rate constant of draining S m 2 Cross-sectional area of the flotation cell h m Pulp level


## 2.1. Flotation Behavior Analysis

The physical activities in a sulfur flotation cell involve the movement of solid particles driven by the flow of air and liquid. As illustrated in Figures2 and 3 [10], considering the different roles of pulp and froth in determining the cell dynamics, the flotation cell is divided into two sections: froth volume and pulp volume. The froth volume and pulp volume are then subdivided in the liquid and air phases with which the mineral particles are associated. The states and movements of particles in a flotation cell mainly include the following:(i)
SLP: free in the pulp (particles in the liquid phase of the pulp volume)(ii)
SBP: attached in the pulp (particles attached to the bubbles in the pulp volume)(iii)
SLF: free in the froth (particles in the liquid of the froth volume)(iv)
SBF: attached in the froth (particles attached to the bubbles of the froth volume)Figure 3
Particle movement in a flotation cell.and(i)
SLP→SBP: free particles in the pulp attached to the bubbles in the pulp volume with the attachment rate constant kijPAT(ii)
SBP→SLP: particles associated with the bubbles detached and enter the liquid phase in the pulp volume with the detachment rate constant kijPDT(iii)
SLF→SBF: free particles in the froth layer attached to the bubbles in the froth volume with the attachment rate constant kijFAT(iv)
SBF→SLF: particles associated with the bubbles detached and enter the liquid phase in the froth volume with the detachment rate constant kijFDT(v)
Feed→SLP: the particles in the input slurry (with liquid volume flow rate QFeed and ΨijFeed the number of particles of size i and mineralogical species j per unit volume) feed into the flotation cell(vi)
SLF→SLP: particles draining from the liquid phase of the froth volume (with the flow rate of QR and ΨijLF the number of particles of size i and mineralogical species j per unit volume of liquid in the froth volume, kijR a constant indicating the segregation mechanism in the draining)(vii)
SLP→SLF: particles entrained by the liquid flow from pulp volume to the froth volume (with the flow rate of QE)(viii)
SBP→SBF: transport of particles attached on the bubbles in the pulp volume to the froth volume (with the air flow rate of QA′)(ix)
SLP→Tailings: the particles in the pulp leave through the tailing port (with the flow rate of QT and ΨijLP the number of particles of size i and mineralogical species j per unit volume of liquid in the pulp)(x)
SBP→Tailings: exit of particles attached on the bubbles in the pulp volume to the tailings (with the aeration rate of QAT)(xi)
SLF→Concentrate: the particles free in the froth overflow to the concentrate lander (with the flow rate of QC)(xii)
SBF→Concentrate: the particles attached to the bubbles in the froth overflow to the concentrate lander (with the flow rate of QAC)


## 2.2. Phenomenological Model of a Flotation Cell

Based on the above simplifications, consider the hydraulic behavior, and apply the following assumptions:Assumption 1.
Each phase in the pulp and froth volume of the flotation cell is perfectly mixed.Assumption 2.
The particle size and shape are uniform.Assumption 3.
The changes in the air volumes occur much faster than the liquid volumes, and steady state relations are used; i.e.,QA=QA′+QAT, QA′=QAC [10, 23].then a process model, which is a combination of the particle population balance in the liquid phase and the air phase for the pulp and froth volume, is built to describe the dynamics of a flotation cell:(1a)dCjPdt=QFeedVLPCjFeed-QE+QAαjPVLP/VBP1+αjPVLPCjP+QRkjR1+αjFVLPCjF-QT1+αjPVLPCjP(1b)dCjFdt=QE+QAαjPVLP/VBP1+αjPVLFCjP-QRkjR1+αjFVLFCjF-QTαjP1+αjPVLFCjP-QAVBFCjF+QTVBPVBFVLPCjF(1c)dVLPdt=QFeed-QT-QE+QR(1d)dVLFdt=QE-QR-QC(1e)dhdt=VLPS2VLP-ShQFeed-QT-QE+QRwhere (1a) and (1b) present the mass balance for mineralogical species j, while (1c) and (1d) describe the liquid volume balance at the pulp volume and froth volume, and(2)VBP=Sh-VLP(3)VBF=V-Sh-VLF(4)QC=VLFVBFQA-VBPVLPQTThe model (1a), (1b), (1c), (1d), and (1e) is a combination of the froth flotation models proposed in [10, 24]. It keeps the phenomenological behavior in [10], while it also approximates the dynamics of the pulp level in [24]. The physical meanings of the model parameters and the variables in Figure 3 are listed in Table 1. In Section 5, this model is applied to simulate the real flotation cell.Table 1
List of model parameters and process variables.
Parm Unit Physical meaning Q F e e d m 3/h Liquid volume flow rate of the slurry feed Q C m 3/h Liquid volume flow rate of the concentrate Q T m 3/h Liquid volume flow rate of the tailings Q E m 3/h Liquid volume flow rate of the entrainment from pulp to froth volume Q R m 3/h Liquid volume flow rate of the drainage from froth to pulp volume Q A m 3/h Air flow rate of the aeration Q A ′ m 3/h Air flow rate that leaves the pulp volume into the froth volume Q A C m 3/h Air flow rate that leaves the froth volume into the concentrate launder Q A T m 3/h Air flow rate that leaves the pulp volume into the tailings V m 3/h Volume of the flotation cell V T F m 3/h Volume of the froth layer in the flotation cell V L F m 3/h Volume of liquid phase in the froth layer V B F m 3/h Volume of air phase in the froth layer V T P m 3/h Volume of the pulp layer in the flotation cell V L P m 3/h Volume of liquid phase in the pulp layer V B P m 3/h Volume of air phase in the pulp layer C j F e e d t/m3 Concentration of speciesj in the liquid volume of the feed slurry C j F t/m3 Concentration of speciesj in the liquid volume of the froth layer C j P t/m3 Concentration of speciesj in the liquid volume of the pulp layer Ψ i j F e e d - Number of particles of sizei and mineralogical species j per unit volume in the feed Ψ i j L F - Number of particles of sizei and mineralogical species j per unit volume of liquid phase in the froth layer Ψ i j B F - Number of particles of sizei and mineralogical species j per unit volume of air phase in the froth layer Ψ i j L P - Number of particles of sizei and mineralogical species j per unit volume of liquid phase in the pulp layer Ψ i j B P - Number of particles of sizei and mineralogical species j per unit volume of air phase in the pulp layer k i j P A T - Rate constant of particles of sizei and mineralogical species j attach from the pulp to the bubble in the pulp volume k i j P D T - Rate constant of particles of sizei and mineralogical species j detach from the bubble to the pulp in the pulp volume k i j F A T - Rate constant of particles of sizei and mineralogical species j attach from the pulp to the bubble in the froth volume k i j F D T - Rate constant of particles of sizei and mineralogical species j attach from the bubble to the pulp in the froth volume α j F - Rate constant of the attachment and detachment for speciesj in the froth α j P - Rate constant of the attachment and detachment for speciesj in the pulp k j R - Rate constant of draining S m 2 Cross-sectional area of the flotation cell h m Pulp level


## 3. Optimal Control Problem Analysis

To start with the optimal control problem analysis, the model is reformulated first to investigate how the manipulated variables affect the flotation performance.As indicated in [10], QE and QR can be formulated as (5)QE=6δdBPQA=r1QA(6)QR=1-μFμFQAS=r2QAwhere δ is the thickness of film surrounding the bubble, dBP is the mean size bubble diameter in the pulp region, μF is the average air holdup in the plateau border of froth volume, and r1 and r2 are two variables changing with working conditions.Combining (1a), (1b), (1c), (1d), and (1e), (5), (6), assume there are two mineralogical classes in the pulp, i.e., sulfur rich mineral (class 1) and sulfur poor mineral (class 2); the original model can be reformulated as (7a)dC1Pdt=1VLPQFeedC1Feed-β1QA-QT1+α1PC1P(7b)dC1Fdt=1VLFβ1QA-VLFVBFQAC1F-α1P1+α1PQTC1P+VLFVBPVBFVLPQTC1F(7c)dC2Pdt=1VLPQFeedC2Feed-β2QA-QT1+α2PC2P(7d)dC2Fdt=1VLFβ2QA-VLFVBFQAC2F-α2P1+α2PQTC2P+VLFVBPVBFVLPQTC2F(7e)dVLPdt=QFeed-QT-r3QA(7f)dVLFdt=r3QA-VLFVBFQA+VLFVBPVBFVLPQT(7g)dhdt=VLPS2VLP-ShQFeed-QT-r3QAwhere(8)r3=r1-r2and for the mineralogical class i(9)βi=11+αiPr1+αiPVLPVBPCiP-r2αikFCiFwhere(10)αikF=kiR1+αiFQA and QT are constrained in reasonable operation ranges and rationally adjusted to avoid too high or too low pulp level.The reformulated model (7a), (7b), (7c), (7d), (7e), (7f), and (7g) indicates that air flow rate QA, tailing flow rate QT, and pulp level h are important factors that determine the flotation efficiency. This conclusion is consistent with the site experience. In a flotation plant, the operators adjust these variables to obtain the desired recovery γ∗ and concentrate grade η∗ which are determined by a higher-level decision maker. The concentrate grade η and recovery γ are defined by (11a)η=VLFC1Fg1+VLFC2Fg2VLFC1F+VLFC2F(11b)γ=ηin-ηtailηη-ηtailηinwhere CF is the density of the final concentrate (t/m3), g1 and g2 are the sulfur grade of the two mineralogical classes, ηin is the grade of the feed, and(12)ηtail=VLPC1Pg1+VLPC2Pg2VLPC1P+VLPC2Pis the grade of the tailings. In practice, the optimal control of the entire flotation process can be transformed to the setting and control of VLP and VLF (determine the pulp level) and CF (determines the concentrate grade) of each cell to assure that, in each cell, a sufficient part of the froth is collected while undesired situations, e.g., pulp tuning, are avoided.


## 4. ADP-Based Optimal Control Design

The dynamic of a flotation cell can be expressed in a state-space format:(13)x˙=fx+gxuin which, the states and the controlled and manipulated variables are(14)x=C1P,C1F,C2P,C2F,VLP,VLF,hTu=QTQATand(15)fx=C1FeedVLPQFeed0C2FeedVLPQFeed0QFeed0VLPS2VLP-ShQFeed(16)gx=-C1P1+α1PVLP-β1VLP-α1PC1P1+α1PVLF+VBPC1FVBFVLPβ1VLF-C1FVBF-C2P1+α2PVLP-β2VLP-α2PC2P1+α2PVLF+VBPC2FVBFVLPβ2VLF-C2FVBF-1-r3VLFVBPVBFVLPr3-VLFVBF-VLPS2VLP-Sh-r3VLPS2VLP-ShIf the model parameters[r1,r2,α1P,α1F,k1R,α2P,α2F,k2R] are unknown under certain working conditions, then the model is only partially known and the performance of traditional optimal control method cannot be guaranteed. This situation also exists in other industry processes. To provide an optimal control alternative when the model parameters are unknown, this section studies the ADP-based optimal control of a flotation cell. ADP does not rely on a precise system model. It can, starting from an initial admissible controller, iteratively improve the controller via learning from the online operational data.


### 4.1. Preliminaries on Policy Iteration

Consider the following objective function:(17)J·=∫t0tfQx+Wuτdtwhere Q(x) is the performance integrand of the system states, and W(u) is the nonquadratic performance integrand of the input variables:(18)Wu=2∫0uH-1vTRdvwhere(19)Hv=h1v1h2v2⋯hmvmT∈RmH-1v=h1-1v1h2-1v2⋯hm-1vmT∈Rmhi(·) is a bounded, monotonic, odd Cp(p⩾1) function (i=1,2,…,m) and maps the control bounds imposed, hi-1(·) is the inverse function of hi(·), v=v1v2⋯vmT, and R∈Rm×m is a diagonal matrix with its diagonal elements all positive.The optimal control policy is(20)u∗x=-H12R-1gTx∇V∗xwhere V∗(x) is the solution of a Hamilton-Jacobi-Bellman (HJB) equation:(21)∇VxTfx-gxH12R-1gTx∇Vx+Qx+2∫0-H1/2R-1gTx∇VxH-1vTRdv=0with V(0)=0.The HJB equation is generally difficult to solve analytically. Therefore, Police Iteration (PI) approach is required:(1)
Apply an initial stabilizing admissible controlleru0(x) to the system(2)
Fori⩾0, solve the following Lyapunov equation for V(i):(22)∇VixTfx+gxuix+Qx+2∫0uixH-1vTRdv=0(3)
Update the control law(23)ui+1x=-H12R-1gTx∇Vix(4)
if stop criterion is met, then stop; otherwise,i=i+1; go to Step (2).


### 4.2. ADP-Based Input-Constrained Optimal Control

In the police iteration, full knowledge off(x) and g(x) is required. In this study, a model-free PI technique is extended to the input-constrained case to eliminate this requirement.To start with, reformulateu as(24)u=ui+viThen(25)x˙=fx+gxui+gxviwhere u is the control forced actually on the system, and u(i) is the control policy to be improved iteratively.Consider a cost functionV(i) satisfying (22):(26)Vi˙=VixTfx+gxui+gxvi=-Qx-Wui-2H-Tui+1RviIntegrateV(i)˙ on a time interval [t,t+Δt]:(27)Vit+Δt-Vit=∫tt+Δt-Qx-Wui-2H-Tui+1RvidτAssume{ϕj(x)}j=1∞ and {ψj(x)}j=1∞ are two infinite sequences of linearly independent smooth basis functions on X, where ϕj(0)=0 and ψj(0)=0 for all j=1,2,⋯. Approximate the cost function and the control policy as(28)V^i=∑j=1N1c^jiϕjx=ΦxTc^iwhere c^(i)=c^1(i)c^2(i)⋯c^N1(i)T, Φx=ϕ1(x)ϕ2(x)⋯ϕN1(x)T, and(29)u^i=-H12R-TΥ^ixwhere(30)Υ^ix=k^iΨxwith(31)k^i=k^11ik^12i⋯k^1N2ik^21ik^22i⋯k^2N2i⋮k^m1ik^m2i⋯k^mN2i(32)Ψx=ψ1xψ2x…ψN2xTNote that the input is approximated by (29). This eliminates the requirement on the knowledge on g(x), as indicated by (23). This problem can also be handled by constructing a NN-based approximation system, such as the method proposed in [25].In (28)-(32), N1>0 and N2>0 are two sufficiently large integers. Substitute Eqs. (28)-(32) into (27), and denote t and t+Δt as tk and tk+1:(33)∑j=1N1c^jiϕjxtk+1-ϕjxtk-∫tktk+1k^i+1Ψxujx-ujixdt=-∫tktk+1Qx+Wuidt+εi,kIn (33), there are N1+N2 variables to be determined in c^(i) and k^j(i+1)  (j=1,2,…,m), and consider a sufficient long time sequence {tk}k=0L with L⩾N1+N2; then according to (33)(34)IΦx-IΨu+IΨuic^iveck^i+1T=-IQx-IWui+Δiwhere(35)IΦx=Φx1T-Φx0TΦx2T-Φx1T⋮ΦxLT-ΦxL-1TL×N1(36)IΨu=∫t0t1uTIm⊗ΨxTdt∫t1t2uTIm⊗ΨxTdt⋮∫tL-1tLuTIm⊗ΨxTdtL×mN2(37)IΨui=∫t0t1H12R-1k^iΨxTIm⊗ΨxT∫t1t2H12R-1k^iΨxTIm⊗ΨxTdt⋮∫tL-1tLH12R-1k^iΨxTIm⊗ΨxTL×mN2(38)IQx=∫t0t1Qxdt∫t1t2Qxdt⋮∫tL-1tLQxdtL×1(39)IWui=∫t0t1Wuidt∫t1t2Wuidt⋮∫tL-1tLWuidtL×1and Δi=εi,1εi,1…εi,LT∈RL is the approximation error.From (34), if(40)rankΘk^i=N1+mN2where Θk^i=IΦx-(IΨu+IΨu(i)), then c^(i) and k^(i+1) can be obtained as(41)c^iveck^i+1T=Θk^iTΘk^i-1Θk^iTΠk^iwhere Π(k^(i))=[-IQx-IWu(i)]. (41) along with (28)-(32) serves as a computational approach to approximate V(i) and generate an improved controller u(i+1) when an admissible control policy u(i) is given. By using this approach iteratively, starting from an admissible bounded control u(0), a control sequence {u(i)(x)}i=0∞ could be obtained and satisfies the following theorem. As the iteration is conducted after the online stage when the information needed for the calculation of Θ(k^(i)) and Π(k^(i)) is collected, this method belongs to off-policy reinforcement learning [26, 27].Theorem 4.
Consider system (13) with input constraints; if there exists an optimal controller and Assumptions 1 and 2 hold, then(42)ui+1=-H12R-Tk^iΨxwhich is generated by the PI ((41), (28), and (29)) is an admissible control sequence for (13) on X. The cost function V(i)=J(t0)u=u(i) satisfies the following Lyapunov equation (LE):(43)VixTfx+gxui+Qx+Wui=0Then V∗(x)⩽V(i+1)(x)⩽V(i)(x) for ∀x∈X. In addition, V(i)→V∗, u(i)→u∗ uniformly on X.Proof.
The proof of this theorem follows the same lines of reasoning as in the proof of Theorem 1 and Theorem 3.1 in [28, 29], and it is omitted here for brevity.Assumption 5.
There existL0>0 and δ>0, such that for all L>L0(44)1L∑k=1Lθi,kTθi,k⩾IN1+N2where θi,k is the kth row of Θ(k^(i)).Assumption 6.
The closed loop system is composed of (16) and(45)u=u0+eis ISS (Input-to-State Stable) [30] when the exploration noise e is considered as the input.


## 4.1. Preliminaries on Policy Iteration

Consider the following objective function:(17)J·=∫t0tfQx+Wuτdtwhere Q(x) is the performance integrand of the system states, and W(u) is the nonquadratic performance integrand of the input variables:(18)Wu=2∫0uH-1vTRdvwhere(19)Hv=h1v1h2v2⋯hmvmT∈RmH-1v=h1-1v1h2-1v2⋯hm-1vmT∈Rmhi(·) is a bounded, monotonic, odd Cp(p⩾1) function (i=1,2,…,m) and maps the control bounds imposed, hi-1(·) is the inverse function of hi(·), v=v1v2⋯vmT, and R∈Rm×m is a diagonal matrix with its diagonal elements all positive.The optimal control policy is(20)u∗x=-H12R-1gTx∇V∗xwhere V∗(x) is the solution of a Hamilton-Jacobi-Bellman (HJB) equation:(21)∇VxTfx-gxH12R-1gTx∇Vx+Qx+2∫0-H1/2R-1gTx∇VxH-1vTRdv=0with V(0)=0.The HJB equation is generally difficult to solve analytically. Therefore, Police Iteration (PI) approach is required:(1)
Apply an initial stabilizing admissible controlleru0(x) to the system(2)
Fori⩾0, solve the following Lyapunov equation for V(i):(22)∇VixTfx+gxuix+Qx+2∫0uixH-1vTRdv=0(3)
Update the control law(23)ui+1x=-H12R-1gTx∇Vix(4)
if stop criterion is met, then stop; otherwise,i=i+1; go to Step (2).


## 4.2. ADP-Based Input-Constrained Optimal Control

In the police iteration, full knowledge off(x) and g(x) is required. In this study, a model-free PI technique is extended to the input-constrained case to eliminate this requirement.To start with, reformulateu as(24)u=ui+viThen(25)x˙=fx+gxui+gxviwhere u is the control forced actually on the system, and u(i) is the control policy to be improved iteratively.Consider a cost functionV(i) satisfying (22):(26)Vi˙=VixTfx+gxui+gxvi=-Qx-Wui-2H-Tui+1RviIntegrateV(i)˙ on a time interval [t,t+Δt]:(27)Vit+Δt-Vit=∫tt+Δt-Qx-Wui-2H-Tui+1RvidτAssume{ϕj(x)}j=1∞ and {ψj(x)}j=1∞ are two infinite sequences of linearly independent smooth basis functions on X, where ϕj(0)=0 and ψj(0)=0 for all j=1,2,⋯. Approximate the cost function and the control policy as(28)V^i=∑j=1N1c^jiϕjx=ΦxTc^iwhere c^(i)=c^1(i)c^2(i)⋯c^N1(i)T, Φx=ϕ1(x)ϕ2(x)⋯ϕN1(x)T, and(29)u^i=-H12R-TΥ^ixwhere(30)Υ^ix=k^iΨxwith(31)k^i=k^11ik^12i⋯k^1N2ik^21ik^22i⋯k^2N2i⋮k^m1ik^m2i⋯k^mN2i(32)Ψx=ψ1xψ2x…ψN2xTNote that the input is approximated by (29). This eliminates the requirement on the knowledge on g(x), as indicated by (23). This problem can also be handled by constructing a NN-based approximation system, such as the method proposed in [25].In (28)-(32), N1>0 and N2>0 are two sufficiently large integers. Substitute Eqs. (28)-(32) into (27), and denote t and t+Δt as tk and tk+1:(33)∑j=1N1c^jiϕjxtk+1-ϕjxtk-∫tktk+1k^i+1Ψxujx-ujixdt=-∫tktk+1Qx+Wuidt+εi,kIn (33), there are N1+N2 variables to be determined in c^(i) and k^j(i+1)  (j=1,2,…,m), and consider a sufficient long time sequence {tk}k=0L with L⩾N1+N2; then according to (33)(34)IΦx-IΨu+IΨuic^iveck^i+1T=-IQx-IWui+Δiwhere(35)IΦx=Φx1T-Φx0TΦx2T-Φx1T⋮ΦxLT-ΦxL-1TL×N1(36)IΨu=∫t0t1uTIm⊗ΨxTdt∫t1t2uTIm⊗ΨxTdt⋮∫tL-1tLuTIm⊗ΨxTdtL×mN2(37)IΨui=∫t0t1H12R-1k^iΨxTIm⊗ΨxT∫t1t2H12R-1k^iΨxTIm⊗ΨxTdt⋮∫tL-1tLH12R-1k^iΨxTIm⊗ΨxTL×mN2(38)IQx=∫t0t1Qxdt∫t1t2Qxdt⋮∫tL-1tLQxdtL×1(39)IWui=∫t0t1Wuidt∫t1t2Wuidt⋮∫tL-1tLWuidtL×1and Δi=εi,1εi,1…εi,LT∈RL is the approximation error.From (34), if(40)rankΘk^i=N1+mN2where Θk^i=IΦx-(IΨu+IΨu(i)), then c^(i) and k^(i+1) can be obtained as(41)c^iveck^i+1T=Θk^iTΘk^i-1Θk^iTΠk^iwhere Π(k^(i))=[-IQx-IWu(i)]. (41) along with (28)-(32) serves as a computational approach to approximate V(i) and generate an improved controller u(i+1) when an admissible control policy u(i) is given. By using this approach iteratively, starting from an admissible bounded control u(0), a control sequence {u(i)(x)}i=0∞ could be obtained and satisfies the following theorem. As the iteration is conducted after the online stage when the information needed for the calculation of Θ(k^(i)) and Π(k^(i)) is collected, this method belongs to off-policy reinforcement learning [26, 27].Theorem 4.
Consider system (13) with input constraints; if there exists an optimal controller and Assumptions 1 and 2 hold, then(42)ui+1=-H12R-Tk^iΨxwhich is generated by the PI ((41), (28), and (29)) is an admissible control sequence for (13) on X. The cost function V(i)=J(t0)u=u(i) satisfies the following Lyapunov equation (LE):(43)VixTfx+gxui+Qx+Wui=0Then V∗(x)⩽V(i+1)(x)⩽V(i)(x) for ∀x∈X. In addition, V(i)→V∗, u(i)→u∗ uniformly on X.Proof.
The proof of this theorem follows the same lines of reasoning as in the proof of Theorem 1 and Theorem 3.1 in [28, 29], and it is omitted here for brevity.Assumption 5.
There existL0>0 and δ>0, such that for all L>L0(44)1L∑k=1Lθi,kTθi,k⩾IN1+N2where θi,k is the kth row of Θ(k^(i)).Assumption 6.
The closed loop system is composed of (16) and(45)u=u0+eis ISS (Input-to-State Stable) [30] when the exploration noise e is considered as the input.


## 5. Experimental Study

### 5.1. Simulation Setup

In order to test the feasibility and performance of the proposed optimal control, a testing environment based on the model introduced in Section2 was constructed to simulate a real flotation cell. Consider the available instrumentation in the real flotation plant; the model (7a), (7b), (7c), (7d), (7e), (7f), and (7g) was reformulated to (46a), (46b), (46c), (46d), (46e), (46f), and (46g), which makes the simulation more close to practical. In the reformulated model, new system states M1P=VLPC1P, M1F=VLFC1F, M2P=VLPC2P, and M2F=VLFC2F are introduced. So C1P=M1P/VLP, C1F=M1F/VLF, C2P=M2P/VLP, and C2F=M2F/VLF.(46a)dM1Pdt=C1PQFeed-QT-r3QA+QFeedC1Feed-β1QA-QT1+α1PC1P(46b)dM1Fdt=C1Fr3QA-VLFVBFQA+VLFVBPVBFVLPQT+β1QA-VLFVBFQAC1F-α1P1+α1PQTC1P+VLFVBPVBFVLPQTC1F(46c)dM2Pdt=C2PQFeed-QT-r3QA+QFeedC2Feed-β2QA-QT1+α2PC2P(46d)dM2Fdt=C2Fr3QA-VLFVBFQA+VLFVBPVBFVLPQT+β2QA-VLFVBFQAC2F-α2P1+α2PQTC2P+VLFVBPVBFVLPQTC2F(46e)dVLPdt=QFeed-QT-r3QA(46f)dVLFdt=r3QA-VLFVBFQA+VLFVBPVBFVLPQT(46g)dhdt=VLPS2VLP-ShQFeed-QT-r3QAThe parameters of the model[r1,r2,α1P,α1F,k1R,α2P,α2F,k2R] were identified from industrial data collected from a real plant [10]. The initial value of system states is (47)M1P0,M1F0,M2P0,M2F0,VLP0,VLF0,h0=4.4531,2.1558,8.3847,0.6074,33.0105,0.8893,3.0The model parameter set is (48)r1,r2,r3,α1P,α2P,α1kF,α2kF=0.1667,0.0417,0.125,4.2060,0.2921,0.5335,4.6816the concentration and sulfur grade of the two mineralogical classes are:(49)C1Feed,C2Feed=0.22,0.27g1,g2=12%,0.5%In order to test two different ways of control, i.e., setting the recovery rate and grade and setting the concentrations of each mineralogical species in both pulp and froth layer, the experimental study includes two tasks: (i)
Task 1: the control objective is to obtain the desired recovery rate and grade:(50)γobj,ηobj=41%,0.09%(ii)
Task 2: the control objective is to obtain desired concentrations of each mineralogical species in both pulp and froth layer:(51)C∗=C1P∗,C1F∗,C2P∗,C2F∗=0.1290,2.5000,0.2545,0.6600In Task 1 and Task 2, the performance integrandQ(x) was chosen as (52)Qx=q1objγ-γobj2+q2objη-ηobj2and(53)Qx=C-C∗TQC-C∗respectively, where(54)q1obj=2×106q2obj=1Q=diag1,1,1,1C=C1P,C1F,C2P,C2FTo handle the input constraint,h(·) was selected as hyperbolic tangent function, and R=I2.The basis functions for the approximation of the control input were selected as(55)QT=QTmax+QTmin2+QTmax-QTmin2tanh⁡k^11x1+k^12x2+k^13x3+k^14x4+k^15x5+k^16x6+e1(56)QA=QAmax+QAmin2+QAmax-QAmin2tanh⁡k^21x1+k^22x2+k^23x3+k^24x4+k^25x5+k^26x6+e2where e1 and e2 are the explosion noises, and(57)e1=100rand11-0.5rand12-0.5sin⁡t+rand13-0.5sin⁡10t+rand14-0.5sin⁡100te2=100rand21-0.5rand22-0.5sin⁡t+rand23-0.5sin⁡10t+rand24-0.5sin⁡100t{randij} are random variables, i=1,2, j=1,2,3,4.The basis functions for the approximation of the value function were selected as:(58)V^=c^1x12+c^2x22+c^3x32+c^4x42+c^5x52+c^6x62+c^7x72+c^8x12x22+c^9x12x32+c^10x12x42+c^11x12x52+c^12x12x62+c^13x12x72+c^14x22x32+c^15x22x42+c^16x22x52+c^17x22x62+c^18x22x72+c^19x32x42+c^20x32x52+c^21x32x62+c^22x32x72+c^23x42x52+c^24x42x62+c^25x42x72+c^26x52x62+c^27x52x72+c^28x62x72for Task 1, and(59)V^=c^1z12+c^2z22+c^3z32+c^4z42+c^5z14+c^6z24+c^7z34+c^8z44+c^9z12z22+c^10z12z32+c^11z12z42+c^12z22z32+c^13z22z42+c^14z32z42+c^15z16+c^16z26+c^17z36+c^18z46+c^19z12z24+c^20z12z34+c^21z12z44+c^22z22z34+c^23z22z44+c^24z32z44+c^25z14z22+c^26z14z32+c^27z14z42+c^28z24z32+c^29z24z42+c^30z34z42for Task 2, respectively, where(60)z1=x1x5-x1objx5The learning phase is 30 minutes. After the operational data was collected, an offline policy iteration was conducted until the stop criterion was met.


### 5.2. Simulation Results and Discussion

In Task 1, the initial admissible controller was selected as:(61)k^11,k^12,k^13,k^14,k^15,k^16=-0.01,-0.01,-0.01,-0.01,0,0k^21,k^22,k^23,k^24,k^25,k^26=-0.01,-0.01,-0.01,-0.01,0,0The resulting controller was(62)QT=965+10tanh⁡133.8804x1-13.3021x2-21.5489x3+19.2795x4-58.2083x5-4.5132x6QA=240+10tanh⁡-283.1446x1+32.4985x2+51.4569x3-31.8602x4+96.6250x5+9.2052x6The simulation results of Task 1 are shown in Figures4–14. Figure 12 shows that the recovery raised to 41% after 1 hour and kept stable since then, while the variation of concentrate grade was small. Figures 13 and 14 demonstrate that the weighting variables for the approximation of control input gradually converge. Figures 4–10 present the variations of CP1, CF1, CP2, CF2, VLP, VLF, and pulp level, respectively. Figure 11 indicates the tailing flow rate and the air flow rate are within their limitations; see [955,975] and [230,250]. From these figures, it was found that the increase of recovery is mainly caused by the increase of CF1 (the concentration of the rich mineralogical class in the froth) and decrease of pulp level h, which is consistent with site experience.Figure 4
C P 1 trajectory in Task 1.Figure 5
C F 1 trajectory in Task 1.Figure 6
C P 2 trajectory in Task 1.Figure 7
C F 2 trajectory in Task 1.Figure 8
V L P trajectory in Task 1.Figure 9
V L F trajectory in Task 1.Figure 10
Pulp level trajectory in Task 1.Figure 11
Tailing flow rate and air flow rate trajectories in Task 1.Figure 12
Recovery and grade trajectories in Task 1.Figure 13
Iteration of the weighting variables for tailing flow rate in Task 1.Figure 14
Iteration of the weighting variables for air flow rate in Task 1.In Task 2, the initial admissible controller was selected as(63)k^11,k^12,k^13,k^14,k^15,k^16=-0.01,-0.01,-0.01,-0.01,0,0k^21,k^22,k^23,k^24,k^25,k^26=-0.01,-0.01,-0.01,-0.01,0,0The resulting controller is(64)QT=965+10tanh⁡-74.4174x1-12.8180x2+221.4781x3-5.0377x4-111.5485x5-45.4909x6QA=240+10tanh⁡-23.1681x1-6.5684x2-63.9545x3-4.6639x4+65.0465x5+28.1158x6The simulation results of Task 2 are shown in Figures15–25. Figures 15 and 16 show that the concentrations of the rich mineralogical species in the pulp and froth layer converge to their desired setting values with small steady-state bias caused by approximation error, while the recovery increased to 42% (Figure 23). This indicates that the desired recovery rate can be achieved by the setting and control of concentrations of each mineralogical species in the flotation cell. This is a useful rule in the optimal control of a flotation circuit composed of multiple cells. Figures 17, 18, 19, 20, and 21 demonstrate the variations of the concentration of poor mineralogical species in the pulp and froth layer, VLP, VLF, pulp level, respectively. Figure 22 indicates that the flow rate of tailing and air input are within their limitations; see [955,975] and [230,250]. Figures 24 and 25 present the convergence of the weighting variables for the approximation of control input, respectively.Figure 15
C P 1 trajectory in Task 2.Figure 16
C F 1 trajectory in Task 2.Figure 17
C P 2 trajectory in Task 2.Figure 18
C F 2 trajectory in Task 2.Figure 19
V L P trajectory in Task 2.Figure 20
V L F trajectory in Task 2.Figure 21
Pulp level trajectory in Task 2.Figure 22
Tailing flow rate and air flow rate trajectories in Task 2.Figure 23
Recovery and grade trajectories in Task 2.Figure 24
Iteration of the weighting variables for tailing flow rate in Task 2.Figure 25
Iteration of the weighting variables for air flow rate in Task 2.In this study, it was observed that the selection of basis function affects the approximation accuracy, which could in turn affect the algorithm convergence and controller performance. Theoretically, more basis functions for the value function and control input could decrease the approximation error. In the two tasks, different basis functions for the approximation of value function were used due to different optimization objectives. The selection of basis functions for the approximation of control input should have certain physical sense. The model structural information could provide more insights in the basis function selection. The magnitude of explosion noise is also an important factor. A small explosion cannot guarantee the rank condition, while an overlarge explosion noise could cause overexcitation.


## 5.1. Simulation Setup

In order to test the feasibility and performance of the proposed optimal control, a testing environment based on the model introduced in Section2 was constructed to simulate a real flotation cell. Consider the available instrumentation in the real flotation plant; the model (7a), (7b), (7c), (7d), (7e), (7f), and (7g) was reformulated to (46a), (46b), (46c), (46d), (46e), (46f), and (46g), which makes the simulation more close to practical. In the reformulated model, new system states M1P=VLPC1P, M1F=VLFC1F, M2P=VLPC2P, and M2F=VLFC2F are introduced. So C1P=M1P/VLP, C1F=M1F/VLF, C2P=M2P/VLP, and C2F=M2F/VLF.(46a)dM1Pdt=C1PQFeed-QT-r3QA+QFeedC1Feed-β1QA-QT1+α1PC1P(46b)dM1Fdt=C1Fr3QA-VLFVBFQA+VLFVBPVBFVLPQT+β1QA-VLFVBFQAC1F-α1P1+α1PQTC1P+VLFVBPVBFVLPQTC1F(46c)dM2Pdt=C2PQFeed-QT-r3QA+QFeedC2Feed-β2QA-QT1+α2PC2P(46d)dM2Fdt=C2Fr3QA-VLFVBFQA+VLFVBPVBFVLPQT+β2QA-VLFVBFQAC2F-α2P1+α2PQTC2P+VLFVBPVBFVLPQTC2F(46e)dVLPdt=QFeed-QT-r3QA(46f)dVLFdt=r3QA-VLFVBFQA+VLFVBPVBFVLPQT(46g)dhdt=VLPS2VLP-ShQFeed-QT-r3QAThe parameters of the model[r1,r2,α1P,α1F,k1R,α2P,α2F,k2R] were identified from industrial data collected from a real plant [10]. The initial value of system states is (47)M1P0,M1F0,M2P0,M2F0,VLP0,VLF0,h0=4.4531,2.1558,8.3847,0.6074,33.0105,0.8893,3.0The model parameter set is (48)r1,r2,r3,α1P,α2P,α1kF,α2kF=0.1667,0.0417,0.125,4.2060,0.2921,0.5335,4.6816the concentration and sulfur grade of the two mineralogical classes are:(49)C1Feed,C2Feed=0.22,0.27g1,g2=12%,0.5%In order to test two different ways of control, i.e., setting the recovery rate and grade and setting the concentrations of each mineralogical species in both pulp and froth layer, the experimental study includes two tasks: (i)
Task 1: the control objective is to obtain the desired recovery rate and grade:(50)γobj,ηobj=41%,0.09%(ii)
Task 2: the control objective is to obtain desired concentrations of each mineralogical species in both pulp and froth layer:(51)C∗=C1P∗,C1F∗,C2P∗,C2F∗=0.1290,2.5000,0.2545,0.6600In Task 1 and Task 2, the performance integrandQ(x) was chosen as (52)Qx=q1objγ-γobj2+q2objη-ηobj2and(53)Qx=C-C∗TQC-C∗respectively, where(54)q1obj=2×106q2obj=1Q=diag1,1,1,1C=C1P,C1F,C2P,C2FTo handle the input constraint,h(·) was selected as hyperbolic tangent function, and R=I2.The basis functions for the approximation of the control input were selected as(55)QT=QTmax+QTmin2+QTmax-QTmin2tanh⁡k^11x1+k^12x2+k^13x3+k^14x4+k^15x5+k^16x6+e1(56)QA=QAmax+QAmin2+QAmax-QAmin2tanh⁡k^21x1+k^22x2+k^23x3+k^24x4+k^25x5+k^26x6+e2where e1 and e2 are the explosion noises, and(57)e1=100rand11-0.5rand12-0.5sin⁡t+rand13-0.5sin⁡10t+rand14-0.5sin⁡100te2=100rand21-0.5rand22-0.5sin⁡t+rand23-0.5sin⁡10t+rand24-0.5sin⁡100t{randij} are random variables, i=1,2, j=1,2,3,4.The basis functions for the approximation of the value function were selected as:(58)V^=c^1x12+c^2x22+c^3x32+c^4x42+c^5x52+c^6x62+c^7x72+c^8x12x22+c^9x12x32+c^10x12x42+c^11x12x52+c^12x12x62+c^13x12x72+c^14x22x32+c^15x22x42+c^16x22x52+c^17x22x62+c^18x22x72+c^19x32x42+c^20x32x52+c^21x32x62+c^22x32x72+c^23x42x52+c^24x42x62+c^25x42x72+c^26x52x62+c^27x52x72+c^28x62x72for Task 1, and(59)V^=c^1z12+c^2z22+c^3z32+c^4z42+c^5z14+c^6z24+c^7z34+c^8z44+c^9z12z22+c^10z12z32+c^11z12z42+c^12z22z32+c^13z22z42+c^14z32z42+c^15z16+c^16z26+c^17z36+c^18z46+c^19z12z24+c^20z12z34+c^21z12z44+c^22z22z34+c^23z22z44+c^24z32z44+c^25z14z22+c^26z14z32+c^27z14z42+c^28z24z32+c^29z24z42+c^30z34z42for Task 2, respectively, where(60)z1=x1x5-x1objx5The learning phase is 30 minutes. After the operational data was collected, an offline policy iteration was conducted until the stop criterion was met.


## 5.2. Simulation Results and Discussion

In Task 1, the initial admissible controller was selected as:(61)k^11,k^12,k^13,k^14,k^15,k^16=-0.01,-0.01,-0.01,-0.01,0,0k^21,k^22,k^23,k^24,k^25,k^26=-0.01,-0.01,-0.01,-0.01,0,0The resulting controller was(62)QT=965+10tanh⁡133.8804x1-13.3021x2-21.5489x3+19.2795x4-58.2083x5-4.5132x6QA=240+10tanh⁡-283.1446x1+32.4985x2+51.4569x3-31.8602x4+96.6250x5+9.2052x6The simulation results of Task 1 are shown in Figures4–14. Figure 12 shows that the recovery raised to 41% after 1 hour and kept stable since then, while the variation of concentrate grade was small. Figures 13 and 14 demonstrate that the weighting variables for the approximation of control input gradually converge. Figures 4–10 present the variations of CP1, CF1, CP2, CF2, VLP, VLF, and pulp level, respectively. Figure 11 indicates the tailing flow rate and the air flow rate are within their limitations; see [955,975] and [230,250]. From these figures, it was found that the increase of recovery is mainly caused by the increase of CF1 (the concentration of the rich mineralogical class in the froth) and decrease of pulp level h, which is consistent with site experience.Figure 4
C P 1 trajectory in Task 1.Figure 5
C F 1 trajectory in Task 1.Figure 6
C P 2 trajectory in Task 1.Figure 7
C F 2 trajectory in Task 1.Figure 8
V L P trajectory in Task 1.Figure 9
V L F trajectory in Task 1.Figure 10
Pulp level trajectory in Task 1.Figure 11
Tailing flow rate and air flow rate trajectories in Task 1.Figure 12
Recovery and grade trajectories in Task 1.Figure 13
Iteration of the weighting variables for tailing flow rate in Task 1.Figure 14
Iteration of the weighting variables for air flow rate in Task 1.In Task 2, the initial admissible controller was selected as(63)k^11,k^12,k^13,k^14,k^15,k^16=-0.01,-0.01,-0.01,-0.01,0,0k^21,k^22,k^23,k^24,k^25,k^26=-0.01,-0.01,-0.01,-0.01,0,0The resulting controller is(64)QT=965+10tanh⁡-74.4174x1-12.8180x2+221.4781x3-5.0377x4-111.5485x5-45.4909x6QA=240+10tanh⁡-23.1681x1-6.5684x2-63.9545x3-4.6639x4+65.0465x5+28.1158x6The simulation results of Task 2 are shown in Figures15–25. Figures 15 and 16 show that the concentrations of the rich mineralogical species in the pulp and froth layer converge to their desired setting values with small steady-state bias caused by approximation error, while the recovery increased to 42% (Figure 23). This indicates that the desired recovery rate can be achieved by the setting and control of concentrations of each mineralogical species in the flotation cell. This is a useful rule in the optimal control of a flotation circuit composed of multiple cells. Figures 17, 18, 19, 20, and 21 demonstrate the variations of the concentration of poor mineralogical species in the pulp and froth layer, VLP, VLF, pulp level, respectively. Figure 22 indicates that the flow rate of tailing and air input are within their limitations; see [955,975] and [230,250]. Figures 24 and 25 present the convergence of the weighting variables for the approximation of control input, respectively.Figure 15
C P 1 trajectory in Task 2.Figure 16
C F 1 trajectory in Task 2.Figure 17
C P 2 trajectory in Task 2.Figure 18
C F 2 trajectory in Task 2.Figure 19
V L P trajectory in Task 2.Figure 20
V L F trajectory in Task 2.Figure 21
Pulp level trajectory in Task 2.Figure 22
Tailing flow rate and air flow rate trajectories in Task 2.Figure 23
Recovery and grade trajectories in Task 2.Figure 24
Iteration of the weighting variables for tailing flow rate in Task 2.Figure 25
Iteration of the weighting variables for air flow rate in Task 2.In this study, it was observed that the selection of basis function affects the approximation accuracy, which could in turn affect the algorithm convergence and controller performance. Theoretically, more basis functions for the value function and control input could decrease the approximation error. In the two tasks, different basis functions for the approximation of value function were used due to different optimization objectives. The selection of basis functions for the approximation of control input should have certain physical sense. The model structural information could provide more insights in the basis function selection. The magnitude of explosion noise is also an important factor. A small explosion cannot guarantee the rank condition, while an overlarge explosion noise could cause overexcitation.


## 6. Conclusions

An ADP-based optimal control approach for a sulfur flotation cell is proposed as an alternative when the parameters of the system model are unknown. The experimental study verified the feasibility and performance of the approach, which indicates the potential of applying bioinspired control methods in complex industrial processes. The advantage of ADP-based optimal control is that it does not rely on the full knowledge of the process model. The difficulties in the implementation of an ADP-based control are in the design of an initial admissible controller and the selection of basis functions. This is problem-specific and could benefit from the structural information of the process model and site experience. In addition, in this study, the system states are assumed available online which is not always the case, so the integration of state-observer [31] or machine-vision [32] and ADP-based control for flotation process should be studied in future. Other future research topics include ADP robust to approximation error caused by limited learning time [33], the optimal control of a single flotation cell in more complicated cases, e.g., with state constraints, the optimal control of a flotation circuit composed of multiple flotation cells, the integration of ADP and classic model-based optimal control (e.g., MPC) and its application in flotation processes.



---

*Source: 4754508-2019-07-16.xml*