  In the biopharmaceutical manufacturing, fermentation process plays a critical
role impacting on productivity and profit. Since biotherapeutics are
manufactured in living cells whose biological mechanisms are complex and have
highly variable outputs, in this paper, we introduce a model-based
reinforcement learning framework accounting for model risk to support
bioprocess online learning and guide the optimal and robust customized stopping
policy for fermentation process. Specifically, built on the dynamic mechanisms
of protein and impurity generation, we first construct a probabilistic model
characterizing the impact of underlying bioprocess stochastic uncertainty on
impurity and protein growth rates. Since biopharmaceutical manufacturing often
has very limited data during the development and early stage of production, we
derive the posterior distribution quantifying the process model risk, and
further develop the Bayesian rule based knowledge update to support the online
learning on underlying stochastic process. With the prediction risk accounting
for both bioprocess stochastic uncertainty and model risk, the proposed
reinforcement learning framework can proactively hedge all sources of
uncertainties and support the optimal and robust customized decision making. We
conduct the structural analysis of optimal policy and study the impact of model
risk on the policy selection. We can show that it asymptotically converges to
the optimal policy obtained under perfect information of underlying stochastic
process. Our case studies demonstrate that the proposed framework can greatly
improve the biomanufacturing industrial practice.
