Spaces:
Running
How do these numbers compare to the findings of Sasha Luccioni`s paper from last year?
I am writing a master's research paper on the topic of energy use and carbon emissions of text to image models during inference.
I have reviewed the data in this paper 'Power Hungry Processing: Watts Driving the Cost of AI Deployment?' (http://arxiv.org/abs/2311.16863) by the same author and the figures for energy consumption for the same models are vastly different when compared to the AI Energy Score.
For example for stabilityai/stable-diffusion-xl-base-1.0, the paper cites a consumption of 11.41kWh per 1000 inferences. In comparison, for the exact same model in this AI Energy score, it generates 1639.85 Wh per 1000 inferences (converted to kWh being 1.64kWh per 1000 inferences).
The processor for the first one was NVIDIA A100-SXM480GB GPU whereas for the second it was NVIDIA H100 GPUs. Can the processor change so vastly the energy use? A small variation could be accounted for, but almost 7 times more? Am I missing something? Which figures should I use or trust?
Hi!
For "Power Hungry Processing", we used the default settings of the model and so the images generated were a lot bigger, whereas in AI Energy Score we constrain the models to generate the same dimension of image. I think the part of the difference is from there, and the rest from the hardware and set up.
We are going to explore this question in more detail in the future (the impact of all these different factors on energy use) - reach out if you want to work together on this, that would be great!
About your question on "Can the processor change so vastly the energy use?". H100 GPUs are almost 2 times more Energy efficient in FLOPs/W than A100 https://epoch.ai/data/machine-learning-hardware, meaning they can do the same amount of calculations with 2 times less energy. So, yes to your question, the processor efficiency can effectively change the energy use. With brand new nvidia B100 it would be 4x more efficient than with A100... That's why it's important to compare model on the same hardware.
Adding to that the constraints in the generation, as mentioned, it reach x7 here.
@sasha I would love to contribute to this work even if a little bit. My research is focusing on trying to quantify the environmental impact of AI image generation in the context of the work of web design professionals, by analyzing their professional practices while using these tools. I hope to collect some significant information on inference from heavy users. What is the best way to contact you?
I'm at sasha.luccioni@hf.co :)