  Removing objects from images is a challenging problem that is important for
many applications, including mixed reality. For believable results, the shadows
that the object casts should also be removed. Current inpainting-based methods
only remove the object itself, leaving shadows behind, or at best require
specifying shadow regions to inpaint. We introduce a deep learning pipeline for
removing a shadow along with its caster. We leverage rough scene models in
order to remove a wide variety of shadows (hard or soft, dark or subtle, large
or thin) from surfaces with a wide variety of textures. We train our pipeline
on synthetically rendered data, and show qualitative and quantitative results
on both synthetic and real scenes.
