Patent ID: 11908036
Assignee: ADOBE INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 13:
14. A supervised learning-based method of iteratively training a domain adaptive refinement agent, the method comprising:
feeding synthetic image acquisition data and real image acquisition data derived from a video to a cross-domain encoder to convert the synthetic image acquisition data and the real image acquisition data to a compact synthetic feature representation and a compact real feature representation, respectively, wherein the synthetic image acquisition data and the real image acquisition data comprises a frame of the video, the cross-domain encoder comprising at least four input channels, wherein three of the at least four input channels respectively comprise red (R), green (G), and blue (B) input channels for an RGB image corresponding to the frame of the video, and a fourth input channel comprises the real image acquisition data, wherein the real image acquisition data is a depth map corresponding to the RGB image;
feeding the compact synthetic feature representation from the cross-domain depth encoder and conditional synthetic image data to a synthetic conditional depth prediction branch network to generate a refined version of the synthetic image acquisition data conditioned on the conditional synthetic image data, the synthetic conditional depth prediction branch network comprising a synthetic encoder and a synthetic decoder, the synthetic encoder outputting a compact synthetic conditional depth feature representation, the compact synthetic conditional depth feature representation concatenated with the compact synthetic feature representation output from the cross-domain encoder for input to the synthetic decoder to generate the refined version of the synthetic image acquisition data;
feeding the compact real feature representation from the cross-domain encoder and conditional real image data to a real conditional depth prediction branch network to generate a refined version of the real image acquisition data conditioned on the conditional real image data the real conditional depth prediction branch network comprising a real encoder and a real decoder, the real encoder outputting a compact real conditional depth feature representation, the compact real conditional depth feature representation concatenated with the compact synthetic feature representation output from the cross-domain encoder for input to the real decoder to generate the refined version of the real image acquisition data;
comparing the refined version of the synthetic image acquisition data to ground truth synthetic image acquisition data to calculate a synthetic domain loss and the refined version of the real image acquisition data to the real image acquisition data to calculate a real domain loss; and
updating network parameters of the cross-domain depth encoder and the synthetic conditional depth prediction branch network based on the synthetic domain loss and network parameters of the cross-domain depth encoder and the real conditional depth prediction branch network based on the real domain loss to iteratively train the domain adaptive refinement agent.