## 2 Partially Observable Markov Decision Process (POMDP)
\[V^{*}(b)=\max_{a}\left(\sum_{s^{\prime}\in S}r\big{(}s^{\prime},a\big{)}b(s^{ \prime})+\gamma\sum_{o\in O}P(o|b,a)V^{*}(b)\right) \tag{2}\]where  
\[r\big{(}s^{\prime},a\big{)}=\sum_{s\in S}\sum_{o\in O}r\big{(}s^{\prime},a,s,o \big{)}P\big{(}s|s^{\prime},a\big{)}P(o|s,a)\]  
\[V^{*}(b)=\max_{a}\left(\sum_{s^{\prime}\in S}\alpha\big{(}s^{\prime},a\big{)}b \big{(}s^{\prime}\big{)}\right) \tag{3}\]  
Another variation for attaining an optimal solution (policy) is to link the belief state to actions (referred to as policy iteration). In this approach, the optimization searches for the optimal policy (\(\delta^{*}\)), i.e., the optimal state-action combination [10]. \[\delta^{*}(b)=\max_{a}\left(\sum_{s\in S}b(s)r(s,b)+\gamma\sum_{o\in O}P(o|b,a )V(b(a,o))\right) \tag{4}\]  
Different search algorithms can be found in the literature to attain optimal values or policies for a POMDP problem model. [PERSON], [PERSON], and [PERSON] [11] categorized these algorithms into two broad categories: exact and approximate algorithms. In exact algorithms, every \(\alpha\)-vector for the complete belief state is defined, including the actions. This approach becomes computationally complex as the number of iterations increases, making it unsuitable for large-scale optimization problems. Some of the algorithms categorized in this group include [PERSON]'s enumeration (identifies all possible \(\alpha\)-vectors and only keeps those that are relevant to the problem in each iteration), One Pass (generate an \(\alpha\)-vector and search the belief space where this vector is dominant for any action or outcome), Linear Support (modified version of One Pass where only the actions are searched), and Incremental Pruning (combines Monahan's enumeration with Witness). Value and policy functions are used to search for the best solutions in the sub-space believed to have contained these solutions. MDP-based solvers include Most Likely State (searches the state with the best action-using probability), QMDP (uses Q-function search, picking one vector at a time with all future states being observable to compute the value function), and Fast Informed Bound (searches the best action-state combination using the observation function). Grid-based solvers include a Fixed Grid Method (a fixed number of grids are used to represent an infinite number of states, which could be searched using some boundaries to calculate the value function) and a Variable Grid Method (continually varying the grid point, which would lead to generating a policy) [12]. Point-based solvers search the solution space by sampling the belief states. Some of the algorithms include Point-Based Value Iteration (starts from an initial state and gradually builds up by selecting the best points), Randomized Point-Based Value Iteration--Perseus (builds up by randomly selecting points), and Heuristic Search Value Iteration, HSVI- SARSOP (uses heuristics to select points for value functions bounded by upper and lower limits). Monte Carlo simulation-based models have also been developed to solve large POMDP models. For example, Monte Carlo Tree Search (MCTS) combined random sampling and tree search approaches to balance the search effort between exploitation and exploration [13]. These Monte Carlo methods have also been applied to the continuous POMDP models (refer to [PERSON] [14] and [PERSON] et al. [15]). The implementation of the POMDP to corrosion maintenance optimization requires the adoption of this general formulation model in order to apply the solution to attain the recommended maintenance action. The next section discusses this formulation of the model by defining each POMDP component, as presented above.