Parallax Images
Tags: Parallax, AI, Python, Opencv, 3d

Now, after we have loaded the depth map, we can create masks for different layers by thresholding the depth map at different Thresholds.

While making one layer we need two masks, one of this layer and the second of the previous layer to inpaint the missing parts. We will take the last layer outside the loop so we can extract all the remaining parts in this layer.



layers = []

prev_thres = 255

div=30



for thres in range(255 - div, 0, -div):

ret, mask = cv2.threshold(depth_map, thres, 255, cv2.THRESH_BINARY)



ret, prev_mask = cv2.threshold(depth_map, prev_thres, 255, cv2.THRESH_BINARY)



prev_thres = thres

inpaint_img = cv2.inpaint(img, prev_mask, 10, cv2.INPAINT_NS)

layer = cv2.bitwise_and(inpaint_img, inpaint_img, mask = mask) layers.append(conv_cv_alpha(layer, mask))



# adding last layer



mask = np.zeros(depth_map.shape, np.uint8)

mask[:,:] = 255



ret, prev_mask = cv2.threshold(depth_map, prev_thres, 255, cv2.THRESH_BINARY)



inpaint_img = cv2.inpaint(img, prev_mask, 10, cv2.INPAINT_NS) layer = cv2.bitwise_and(inpaint_img, inpaint_img, mask = mask) layers.append(conv_cv_alpha(layer, mask))



layers = layers[::-1]

We have reversed the layers so we can arrange them in order of last to the first layer. While we are adding the layer to the list, we are using a function ‘conv_cv_alpha’ this will add the alpha value (make RGB to RGBA) and make parts of the layer transparent using the mask.

def conv_cv_alpha(cv_image, mask):

b, g, r = cv2.split(cv_image)

rgba = [r, g, b, mask]

cv_image = cv2.merge(rgba,4)



return cv_image

Now comes the part of face detection and showing the image. For face detection, we will use haarcascade. download them from their official Github repository.

To download them, right-click “Raw” => “Save link as”. Make sure they are in your working directory.

Now we will load haar cascade for face detection and make a function that will return the face-rect from the image.

face_cascade = cv2.CascadeClassifier( 'haarcascade_frontalface_default.xml')



def get_face_rect(img):

gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

face_rects = face_cascade.detectMultiScale(gray_img, 1.3, 5)

if len(face_rects) == 0:

return () return face_rects[0]

Now we have to show the image that will shift according to the user’s head. We will use OpenCV to read cam and then Pygame to render each frame on top of each other. To calculate the shift for each layer, we will calculate the shift of head from the center of the frame and then scale it down to get a small shift value. After that, we will multiply the index value of each layer to get the shift value for the respective layer, you can also multiply some constant value in that for better results.

We will create a Pygame window slightly smaller than the original image and load the camera. We have used scale, so you change its value to make the final result bigger.

scale = 1

off_set = 20 width, height = layers[0].get_width(), layers[0].get_height() win = pg.display.set_mode((int((width - off_set)*scale), int((height - off_set)*scale)))

pg.display.set_caption('Parallax_image') scaled_layers = []

for layer in layers:

scaled_layers.append(pg.transform.scale(layer, (int(width*scale), int(height*scale)))) cap = cv2.VideoCapture(0, cv2.CAP_DSHOW)

We will set some constants. you can play with these constants to get different results.

x_transform = True # allow shift in x-axis

y_transform = False # allow shift in y-axis

sens = 50 # the amount of scale down of shift value

show_cam = False # show your face cam

shift_x = 0

shift_y = 0

run = True

Finally, the main loop to render all layers.

while run:

for event in pg.event.get():

if event.type==pg.QUIT:

run = False ret, frame = cap.read()

frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

initial_pos = (frame.shape[0]/2, frame.shape[1]/2)

face_rect = get_face_rect(frame) if len(face_rect) != 0:

x,y,w,h, = face_rect

face_rect_frame = cv2.rectangle(frame, (x, y), (x + w, y + h), (255,255,0), 3) shift_x = (initial_pos[0] - (x + w/2))/(sens*scale)

shift_y = (initial_pos[1] - (y + h/2))/(sens*scale) win.fill((255, 255, 255))



for i, layer in enumerate(scaled_layers):

new_x = -off_set/2

new_y = -off_set/2

if x_transform:

new_x = 0 + shift_x*i

if y_transform:

new_y = 0 + shift_y*i

win.blit(layer, (new_x, new_y))



face_rect_frame = cv2.resize(face_rect_frame, (100, 100))

if show_cam:

win.blit(conv_cv_pygame(face_rect_frame), (0, 0))

pg.display.update() cap.release()

cv2.destroyAllWindows()

pg.quit()

There it is, the final result.

Final Result

demo for different image

I have created a more advanced version of this tool where you can just choose the image and it will automatically create the parallax image, the depth map will be automatically generated.

You can check more on my GitHub repo.