Get all color differences along the boundary