# Variables Reduction in Sequential Resource Allocation Problems
## 3 Optimal Control via Virtual Resource Price  
Let us introduce the required framework more precisely. Consider an agent who is confronted with the following problem. At the beginning \(t=0,1\ldots,T-1\) of each decision epoch, an activity plan (work schedule) \(\xi_{t}\) is to be determined. Therefore, all costs of this work plan must be optimally balanced against their resource consumption/generation. Suppose that the limited resources are described in terms of the state variable \(e\in I\) which stands for the current resource shortage and can vary within a certain interval \(I\subset\mathbb{R}\). For instance, if the resource under consideration is a commodity in the storage, then \(e\in I\) stands for the amount of commodity required to fill the storage to its maximal capacity. The other state variable \(z\in\mathbb{R}^{d}\) is supposed to represent the situation in the surrounding environment. Let us agree that this environmental state variable \(z\) is relevant for decisions, but cannot be influenced by agent's actions. For instance, for commodity storage, \(z\) may comprise the driving market factors of the commodity price evolution. We furtherly assume that the environment state occurs at any time \(t=0,\ldots,T\) as a realization \(z=Z_{t}\) of an \(\mathbb{R}^{d}\)-valued Markov process \((Z_{t})_{t=0}^{T}\) whose dynamics carries all information, relevant for decisions. Having observed at time \(t=0,\ldots,T-1\) the resource level \(e\in I\) and the realization of the environmental state variable \(z=Z_{t}\), the agent selects a plan \(\xi\in\Xi\) from the set \(\Xi\) of all feasible activity plans. This choice yields an immediate cost \(C_{t}^{e,z}(\xi)\) and causes an immediate resource consumption \(E_{t}^{e,z}(\xi)\) via pre-specified functions \(C_{t}^{e,z}\), \(E_{t}^{e,z}\) on \(\Xi\) which may depend on the recent state \((e,z)\in I\times\mathbb{R}^{d}\). While all costs are accumulated, the resource level is carried over to the next decision time \(t+1\) as \(e+E_{t}^{e,z}(\xi)\) and will influence the decision at this time. The availability of resources becomes crucial at the end \(t=T\) of the planning horizon, when a certain terminal costs \(C_{T}(e,z)\) must be paid, which depend on the total resource level \(e\in I\) and on the state \(z\in\mathbb{R}^{d}\) of the environment. Under additional assumptions, such control problems are solved in terms of the so-called value functions \((V_{t})_{t=0}^{T}\) which are obtained via Bellman recursions as  
\[V_{T}(e,z)=C_{T}(e,z) \tag{9}\] \[\tilde{V}_{t+1}(e,z)=\left\{ \begin{array}{ll}\mathbb{E}(V_{t+1 }(e,Z_{t+1})\,|\,Z_{t}=z)&e\in I\\ +\infty&e\
ot\in I\end{array} \right.,z\in\mathbb{R}^{d}\] (10) \[V_{t}(e,z)=\inf_{\xi\in\Xi}\left(C_{t}^{e,z}(\xi)+\tilde{V}_{t+ 1}(e+E_{t}^{e,z}(\xi),z)\right), \tag{11}\]for all arguments \((e,z)\in I\times\mathbb{R}^{d}\) representing the current situation at decision time. Please note that we implicitly agreed to penalize the violation of the admissible resource level in (10) by infinity, gaining the freedom to interpret the value function for all arguments \(e\in\mathbb{R}\) with possible values \(+\infty\) in order to avoid watching restriction \(e+E_{t}^{e,z}(\xi)\in I\) in (11). The idea underlying our variable reduction scheme is based on the realization that if the resources had a market price, then an agent would minimize all activity costs taking into account the monetary value of the consumed resources. To realize this concept, we suppose that some entity (a regulatory body) could charge a resources price \(A\in\mathbb{R}\) at any time \(t\), depending on the situation \((e,z)\).