  Creating virtual avatars with realistic rendering is one of the most
essential and challenging tasks to provide highly immersive virtual reality
(VR) experiences. It requires not only sophisticated deep neural network (DNN)
based codec avatar decoders to ensure high visual quality and precise motion
expression, but also efficient hardware accelerators to guarantee smooth
real-time rendering using lightweight edge devices, like untethered VR
headsets. Existing hardware accelerators, however, fail to deliver sufficient
performance and efficiency targeting such decoders which consist of
multi-branch DNNs and require demanding compute and memory resources. To
address these problems, we propose an automation framework, called F-CAD
(Facebook Codec avatar Accelerator Design), to explore and deliver optimized
hardware accelerators for codec avatar decoding. Novel technologies include 1)
a new accelerator architecture to efficiently handle multi-branch DNNs; 2) a
multi-branch dynamic design space to enable fine-grained architecture
configurations; and 3) an efficient architecture search for picking the
optimized hardware design based on both application-specific demands and
hardware resource constraints. To the best of our knowledge, F-CAD is the first
automation tool that supports the whole design flow of hardware acceleration of
codec avatar decoders, allowing joint optimization on decoder designs in
popular machine learning frameworks and corresponding customized accelerator
design with cycle-accurate evaluation. Results show that the accelerators
generated by F-CAD can deliver up to 122.1 frames per second (FPS) and 91.6%
hardware efficiency when running the latest codec avatar decoder. Compared to
the state-of-the-art designs, F-CAD achieves 4.0X and 2.8X higher throughput,
62.5% and 21.2% higher efficiency than DNNBuilder and HybridDNN by targeting
the same hardware device.
