hmdp is a library and tools for solving Markov Decision Processes (MDPs) with hybrid discrete and/or continuous state-spaces.