GRIM: Fix deleting material when it is still needed