Before revision:  We formulate the liquidation problem as a discrete-time Markov Decision Processes  where the state process is a Piecewise Deterministic Markov Process (PDMP) , which is a member of right continuous Markov Process family. We study the dynamics of a limit order book and its influence on the price dynamics and develop a stochastic model to retain the main statistical characteristics of limit order books in illiquid markets .
After revision:  We formulate the liquidation problem as a discrete-time Markov Decision Processes where the state process is a Piecewise Deterministic Markov Process (PDMP) . The numerical results indicate that an optimal trading strategy is dependent on characteristics of the market microstructure. When no orders above certain value come the optimal solution takes offers in the lower levels of the limit order book and its influence on the price dynamics and develop a stochastic model to retain the main statistical characteristics of limit order books in illiquid markets .