  Robot manipulation in cluttered scenes often requires contact-rich
interactions with objects. It can be more economical to interact via
non-prehensile actions, for example, push through other objects to get to the
desired grasp pose, instead of deliberate prehensile rearrangement of the
scene. For each object in a scene, depending on its properties, the robot may
or may not be allowed to make contact with, tilt, or topple it. To ensure that
these constraints are satisfied during non-prehensile interactions, a planner
can query a physics-based simulator to evaluate the complex multi-body
interactions caused by robot actions. Unfortunately, it is infeasible to query
the simulator for thousands of actions that need to be evaluated in a typical
planning problem as each simulation is time-consuming. In this work, we show
that (i) manipulation tasks (specifically pick-and-place style tasks from a
tabletop or a refrigerator) can often be solved by restricting robot-object
interactions to adaptive motion primitives in a plan, (ii) these actions can be
incorporated as subgoals within a multi-heuristic search framework, and (iii)
limiting interactions to these actions can help reduce the time spent querying
the simulator during planning by up to 40x in comparison to baseline
algorithms. Our algorithm is evaluated in simulation and in the real-world on a
PR2 robot using PyBullet as our physics-based simulator. Supplementary video:
\url{https://youtu.be/ABQc7JbeJPM}.
