A computing system comprising: a processing system including at least one processing unit, the processing system being configured to implement a method that includes: receiving a depth map with a first boundary of an object; receiving a color image that corresponds to the depth map, the color image including a second boundary of the object; extracting depth edge points of the first boundary from the depth map; identifying candidates for the depth edge points along two-dimensional normal line segments in which each 2D normal line segment is perpendicular to a respective smoothed position of the depth edge point using smoothed positions of two neighboring depth edge points; identifying target depth edge points on the depth map from among the corresponding candidates by minimizing an energy function based at least on a data term involving image gradient data from the color image and a smoothness term involving a smoothness constraint, and wherein the target depth edge points correspond to color edge points of the second boundary of the object in the color image; and snapping the depth edge points towards the target depth edge points, wherein the smoothness constraint for a pair of consecutive edge points p i and p j is defined by computing the cost of snapping p i onto c i,k and p j onto c j,l via following equation: E s i k j l = ∞, k − l ≥ d max c i, k − c j, l 2, otherwise, wherein the candidate set of a depth edge point p i is denoted as L i = {c i,k | k = 1,...,2r s +1}, where r s is a range in both the positive and negative direction of the 2D normal line segment of edge point p i and d max defines the maximal discrepancy allowed for two consecutive depth edge points. receiving a depth map with a first boundary of an object; receiving a color image that corresponds to the depth map, the color image including a second boundary of the object; extracting depth edge points of the first boundary from the depth map; identifying candidates for the depth edge points along two-dimensional normal line segments in which each 2D normal line segment is perpendicular to a respective smoothed position of the depth edge point using smoothed positions of two neighboring depth edge points; identifying target depth edge points on the depth map from among the corresponding candidates by minimizing an energy function based at least on a data term involving image gradient data from the color image and a smoothness term involving a smoothness constraint, and wherein the target depth edge points correspond to color edge points of the second boundary of the object in the color image; and snapping the depth edge points towards the target depth edge points, wherein the smoothness constraint for a pair of consecutive edge points p i and p j is defined by computing the cost of snapping p i onto c i,k and p j onto c j,l via following equation: E s i k j l = ∞, k − l ≥ d max c i, k − c j, l 2, otherwise, wherein the candidate set of a depth edge point p i is denoted as L i = {c i,k | k = 1,...,2r s +1}, where r s is a range in both the positive and negative direction of the 2D normal line segment of edge point p i and d max defines the maximal discrepancy allowed for two consecutive depth edge points.