  Advances in deep learning have led to promising progress in inferring
graphics programs by de-rendering computer-generated images. However, current
methods do not explore which decoding methods lead to better inductive bias for
inferring graphics programs. In our work, we first explore the effectiveness of
LSTM-RNN versus Transformer networks as decoders for order-independent graphics
programs. Since these are sequence models, we must choose an ordering of the
objects in the graphics programs for likelihood training. We found that the
LSTM performance was highly sensitive to the sequence ordering (random order
vs. pattern-based order), while Transformer performance was roughly independent
of the sequence ordering. Further, we present a policy gradient based
reinforcement learning approach for better inductive bias in the decoder via
multiple diverse rewards based both on the graphics program specification and
the rendered image. We also explore the combination of these complementary
rewards. We achieve state-of-the-art results on two graphics program generation
datasets.
