# Variables Reduction in Sequential Resource Allocation Problems
## 2 Dynamic Principle for Optimal Switching and Reduction of Decision Variables
For \(t=0,\ldots,T-1\), introduce the _Bellman operator_  
\[\mathcal{T}_{t}v(x)=\sup_{a\in A}\left(r_{t}(x,a)+\mathcal{K}_{t}^{a}v(x) \right),\quad x\in E \tag{4}\]  
which acts on each measurable function \(v:E\to\mathbb{R}\) where the integrals \(\mathcal{K}_{t}^{a}v\) for all \(a\in A\) exist. Furthermore, consider the _Bellman recursion_  
\[v_{T}^{*}=r_{T},\quad v_{t}^{*}=\mathcal{T}_{t}v_{t+1}^{*}\qquad\text{for }t=T-1, \ldots,0. \tag{5}\]  
Under appropriate assumptions, there exists a recursive solution \((v_{t}^{*})_{t=0}^{T}\) to the Bellman recursion, which gives the so-called _value functions_ and determines an optimal policy \(\pi^{*}\) via  
\[\pi_{t}^{*}(x)=\operatorname{argmax}_{a\in A}\left(r_{t}(x,a)+\mathcal{K}_{t} ^{a}v_{t+1}^{*}(x)\right),\quad x\in E,\quad t=0,\ldots,T-1\]  
**Stochastic switching:** Consider now a Markov decision model whose state evolution consists of one controllable and one uncontrollable component. To be more specific, we assume that the state space \(E=P\times\mathcal{Z}\) is the product of a space \(P\) (operation modes) and a set \(\mathcal{Z}\) (states of environment) being a subset \(\mathcal{Z}\subset\mathbb{R}^{d}\) of the Euclidean space. We suppose that at each time \(t=0,\ldots T-1\) the mode component \(p\in P\) is driven by actions \(a\in A\) in terms of a deterministic function  
\[\alpha_{t}:P\times\mathcal{Z}\times A\to P,\quad(p,z,a)\to\alpha_{t}^{p,z}(a)\]  
where \(\alpha_{t}^{p,z}(a)\in P\) stands for the new mode from if the action \(a\in A\) was taken at time \(t=0,\ldots,T-1\) in the state \((p,z)\in E\). In this setting, the transition operators are given by  
\[\mathcal{K}_{t}^{a}v(p,z)=\mathbb{E}(v(\alpha_{t}^{p,z}(a),Z_{t+1})\,|\,Z_{t}= z),\quad z\in\mathcal{Z} \tag{6}\]  
for \(t=0,\ldots,T-1\), and \(a\in A\). **Variables reduction:** In the above context of stochastic switching, the application of Bellman operator  
\[\mathcal{T}_{t}v(p,z)=\sup_{a\in A}\left(r_{t}(p,z,a)+\mathbb{E}(v(\alpha_{t}^ {p,z}(a),Z_{t+1})\,|\,Z_{t}=z)\right),\quad(p,z)\in E \tag{7}\]  
may cause numerical difficulties, particularly if the action space \(A\) is high-dimensional and fragmented. Unfortunately, this situation appears frequently in practice, where high-dimensional vectors of decision variables usually subject to numerous feasibility inter-dependencies. In such framework, the problem may become unsolvable due to difficulty of maximization over a complex action set \(A\). The main contribution of our paper is to provide a significant reduction of set of actions, which are relevant for maximization. Under specific assumptions described in this paper, we show there exists a one-parameter family (curve) \(A_{t}^{p,z}\subset A\) which depends on the recent time \(t=0,\ldots,T=1\) and state \((p,z)\in E\) such that the domain of maximization reduces from \(A\) to \(A_{t}^{p,z}\) which yields instead of the infeasible maximization (7) a new problem  
\[\mathcal{T}_{t}v(p,z)=\sup_{a\in A_{t}^{p,z}}\left(r_{t}(p,z,a)+\mathbb{E}(v(a _{t}^{p,z}(a),Z_{t+1})\,|\,Z_{t}=z)\right),\quad(p,z)\in E \tag{8}\]  
which is significantly simpler and usually admits a (numerical) solution. To determine the curve \(A_{t}^{p,z}\subset A\) of relevant actions, a separate deterministic optimization problem must be solved. Its solution is usually obtained explicitly and provides interesting economic insights. **Contribution of this work:** Using a standard Bellman principle, we explore an abstract, but natural framework to reduce a potentially very large space of decision variables (actions) to a _single one-parameter_ family. Although our technique is entirely placed within the traditional [PERSON] principle of stochastic/classic dynamic programming and addresses a relatively narrow problem class, a wide range of application is covered, including mining operations, pension fund management, and emission control. This approach can serve efficient numerical algorithms for notoriously difficult and important problems from practice.