Spaces:

noni27

/

Shift_and_Inpaint

like 0

Runtime error

Task : Given an image and a text prompt (specifying class of an object), shift the location of that object in image plane by x and y units.

My Solution:

1. Pass input image and text prompt through GroundingDINO model to get bounding box around the object specified in text prompt.
2. Use the bounding box to get segmentation mask using SegmentAnything model.
3. Remove the segmented object.
4. Use StableDiffusion Inpainting model to inpaint the hole created after removing object.
5. Shift the segmented object in image place using opencv affine transformation and combine it with the inpainted image.
   And wolla !!, you have shifted the object specified in text prompt.

Requirement : I need small GPU, to load groundingDINO, SAM, StableDiffusion model and run inference on single (image, text_prompt) at a time.

Regards,
Naman Jaswani