Oh yeah, they're still alive but you won't see them getting published as often due to both most people switching to diffusion and the self fulfilling prophesy of considering things dead. But yeah, if you look at any diffusion platform like Automatic1111 you'll find that GANs are a popular choice of upscaler[0,1]. So you use them together to try to benefit from each of their advantages.

Also, if you look at the top scores for FFHQ at the 256[2] or 1024[3] resolutions, you see GANs winning, and by a good amount. The best diffusion model is #4, and LDM (Stable Diffusion) is #25. Most diffusion research has avoided this dataset due to scale, but this is changing. Probably worth noting that StyleSAN is about a method, not an architecture. Also the #2 on [2] looks to be a smaller lab and they complain about limited compute and spend time arguing about why they think if they scale they'd perform better. They do have some compelling evidence given their FFHQ success is beating much bigger models. But they don't seem to have as much success on LSUN. They also are less successful on 1024, but they again claim limited compute so hard to say. They don't appear to be published in a conference, so I guess they are in fact stuck.

> What did you mean by your last sentence, what is the limiting compute factor for GANs?

Sorry, I meant the limiting compute factor for diffusion. Why GANs are faster. I felt it was worth mentioning since I mentioned that Karras wrote custom cuda kernels for StyleGAN, and this does have a significant impact on speed. In the appendix of StyleGAN2 at the end of B under "Performance optimizations" they mention that their kernels result in an improvement of 30% in training time and 20% improvements on memory footprint.

But the limiting factor between diffusion models and (typical) GANs is that GANs are typically formulated with just a decoder. On the other hand, Diffusion has a full encoder decoder network. This is even true for Latent Diffusion models (i.e. Stable Diffusion), which specifically was designed to tackle the compute challenges of a standard diffusion model. The backbone is a UNet (almost a VAE + residual connections), which is an autoencoder and decoder (there are ViT based backbones, but these are still in the same parameter ballpark). So it is just a challenging architecture to reduce in size. There are clear benefits for doing so, but when it comes to practical applications you have to consider a wide variety of factors. I mean think about the computational costs of generating a 256x256 image with SD, that's a few gigs on your GPU. You need procumer hardware to get 1024 and I can tell you that on a 4080S that images are not instantaneously generated lol. So you're not going to use that in a compute constrained environment like gaming. But on the other hand, I can generate 60 imgs/s on a 2080Ti with StyleGAN2 (haven't checked on my 4080S). There are things like ArtSpew, that start getting closer but the image quality is crap (this is being improved FWIW). But also PGAN is crazy fast...

For more specifics I'm not sure how to accurately explain without getting into the math and a conversation about density estimation. But I don't think that is well suited for a HN conversation. This should be enough to point you in the right direction for that though.

[0] https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki...

[1] https://medium.com/rendernet/using-hires-fix-to-upscale-your...

[2] https://paperswithcode.com/sota/image-generation-on-ffhq-256...

[3] https://paperswithcode.com/sota/image-generation-on-ffhq-102...