Inference for State Space Models like playing with duplo blocks