Patent ID: 11875510
Assignee: ADOBE INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 16:
17. A non-transitory computer-readable medium storing instructions thereon that, when executed by at least one processor, cause a computing device to:
determine a digital visual media item depicting one or more objects;
extract, utilizing a shared encoder of a segmentation refinement neural network, encoded feature maps from the digital visual media item;
generate, utilizing a recursive decoder of the segmentation refinement neural network, an initial segmentation mask for the digital visual media item by:
providing the encoded feature maps generated by the shared encoder to a deconvolution branch and a plurality of hierarchical point-wise refining blocks from a hierarchy of hierarchical point-wise refining blocks of a refinement branch;
determining, utilizing one or more deconvolutional layers of the deconvolution branch, a first decoded feature map based on a first encoded feature map from the encoded feature maps;
generating, utilizing a first hierarchical point-wise refining block of the hierarchy of hierarchical point-wise refinement blocks, an uncertainty map based on the first decoded feature map, the uncertainty map indicating pixels having an associated uncertainty that the pixels correspond to the one or more objects or do not correspond to the one or more objects; and
determining, utilizing the first hierarchical point-wise refining block, the initial segmentation mask for the digital visual media item based on the uncertainty map, the first decoded feature map, and a second encoded feature map from the encoded feature maps; and

recursively refine the initial segmentation mask utilizing additional hierarchical point-wise refining blocks of the hierarchy of hierarchical point-wise refining blocks based on additional encoded feature maps from the encoded feature maps extracted by the shared encoder and additional decoded feature maps from the deconvolution branch,
wherein the recursively refining comprises refining the initial segmentation mask by each of the additional hierarchical point-wise refining blocks, and
wherein encodings of coarse-to-fine variations generated via the additional hierarchical point-wise refining blocks are provided back to the deconvolution branch such that at least some of the additional decoded feature maps are generated based on the generated encodings of coarse-to-fine variations.