REGEN: Real-Time Photorealism Enhancement in Games via a Dual-Stage
  Generative Network Framework
Authors: name: Stefanos Pasios, name: Nikos Nikolaidis

Photorealism is an important aspect of modern video games since it can shape
the player experience and simultaneously impact the immersion, narrative
engagement, and visual fidelity. Although recent hardware technological
breakthroughs, along with state-of-the-art rendering technologies, have
significantly improved the visual realism of video games, achieving true
photorealism in dynamic environments at real-time frame rates still remains a
major challenge due to the tradeoff between visual quality and performance. In
this short paper, we present a novel approach for enhancing the photorealism of
rendered game frames using generative adversarial networks. To this end, we
propose Real-time photorealism Enhancement in Games via a dual-stage gEnerative
Network framework (REGEN), which employs a robust unpaired image-to-image
translation model to produce semantically consistent photorealistic frames that
transform the problem into a simpler paired image-to-image translation task.
This enables training with a lightweight method that can achieve real-time
inference time without compromising visual quality. We demonstrate the
effectiveness of our framework on Grand Theft Auto V, showing that the approach
achieves visual results comparable to the ones produced by the robust unpaired
Im2Im method while improving inference speed by 32.14 times. Our findings also
indicate that the results outperform the photorealism-enhanced frames produced
by directly training a lightweight unpaired Im2Im translation method to
translate the video game frames towards the visual characteristics of
real-world images. Code, pre-trained models, and demos for this work are
available at: https://github.com/stefanos50/REGEN.