  Capturing scene dynamics and predicting the future scene state is challenging
but essential for robotic manipulation tasks, especially when the scene
contains both rigid and deformable objects. In this work, we contribute a
simulation environment and generate a novel dataset for task-specific
manipulation, involving interactions between rigid objects and a deformable
bag. The dataset incorporates a rich variety of scenarios including different
object sizes, object numbers and manipulation actions. We approach dynamics
learning by proposing an object-centric graph representation and two modules
which are Active Prediction Module (APM) and Position Prediction Module (PPM)
based on graph neural networks with an encode-process-decode architecture. At
the inference stage, we build a two-stage model based on the learned modules
for single time step prediction. We combine modules with different prediction
horizons into a mixed-horizon model which addresses long-term prediction. In an
ablation study, we show the benefits of the two-stage model for single time
step prediction and the effectiveness of the mixed-horizon model for long-term
prediction tasks. Supplementary material is available at
https://github.com/wengzehang/deformable_rigid_interaction_prediction
