Automatic Parameter Control for Metropolis Light Transport | Two Minute Papers #30
Photorealistic rendering (also called global illumination) enables us to see how digital objects would look like in real life. It is an amazingly powerful tool in the hands of a professional artist, who can create breathtaking images or animations with. Metropolis light transport is an advanced photorealistic rendering technique that is remarkably effective at finding the brighter regions of a scene and building many light paths that target these regions. The resulting algorithm is more efficient than traditional random path building algorithms, such as path tracing. This algorithm endeavors to choose an optimal mixture between naive random path sampling techniques (such as path tracing and bidirectional path tracing) and Metropolis Light Transport.

___________________

The paper "Automatic Parameter Control for Metropolis Light Transport" is available here:
https://cg.tuwien.ac.at/~zsolnai/gfx/adaptive_metropolis/

We thank Kai Schwebke for providing LuxTime, Vlad Miller for the Spheres, Giulio Jiang for the Chess, Aaron Hill for the Cornell Box, Andreas Burmberger for the Cherry Splash and Glass Ball scenes. 

I held a course on photorealistic rendering at the Technical University of Vienna. Here you can learn how the physics of light works and to write programs like this:
https://www.youtube.com/playlist?list=PLujxSBD-JXgnGmsn7gEyN28P1DnRZG7qi


Subscribe if you would like to see more of these! - http://www.youtube.com/subscription_center?add_user=keeroyz

Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu

Károly Zsolnai-Fehér's links:
Patreon → https://www.patreon.com/TwoMinutePapers
Facebook → https://www.facebook.com/TwoMinutePapers/
Twitter → https://twitter.com/karoly_zsolnai
Web → https://cg.tuwien.ac.at/~zsolnai/
 Dear Fellow Scholars, this is Two Minute Papers with Karo Zsolnai-Fehair. We'll start with a quick recap on metropolis light transport and then discuss a cool technique that builds on top of it. If we would like to see how digitally modeled objects would look like in real life, we would create a 3D model of the desired scene, assign material models to the objects within, and use a photorealistic rendering algorithm to finish the job. It simulates rays of light that connect the camera to the light sources in the scene and compute the flow of energy between them. Initially, after a few rays will only have a rough idea on how the image should look like, therefore our initial results will contain a substantial amount of noise. We can get rid of this by simulating the path of millions and millions of rays that will eventually clean up our image. This process, where a noisy image gets clearer and clearer, we call convergence, and the problem is that this can take excruciatingly long, even up to hours, to get a perfectly clear image. With the simple algorithms out there, we generate these light paths randomly. This technique we call path tracing. However, in the scene that you see here, most random paths can't connect the camera and the light source because this wall is in the way obstructing many of them. Light paths like these don't contribute anything to our calculations and are ultimately a waste of time and resources. After generating hundreds of random light paths, we finally found a path that connects the camera with the light source without any obstructions. When generating the next path, it would be a crime not to use this knowledge to our advantage. A technique called metropolis light transport will make sure to use this valuable knowledge and upon finding a bright light path, it will explore other paths that are nearby to have the best shot at creating valid, unobstructed connections. If we have a difficult scene at hand, metropolis light transport gives us way better results than traditional, completely random paths sampling techniques such as path tracing. This scene is extremely difficult in a sense that the only source of light is coming from the upper left and after the light goes through multiple glass spheres, most of the light paths that we generate will be invalid. As you can see, this is a valiant effort with random path tracing that yields really dreadful results. Metropolis light transport is extremely useful in these cases and therefore should always be the weapon of choice. However, it is more expensive to compute than traditional random sampling. This means that if we have an easy scene on our hands, this smart metropolis sampling doesn't pay off and performs worse than a naive technique in the same amount of time. So, on easy scenes, traditional random sampling, difficult scenes, metropolis sampling, super simple, super intuitive, but the million dollar question is how to mathematically formulate and measure what an easy and what a difficult scene is. This problem is considered extremely difficult and was left open in the metropolis light transport paper in 2002. Even if we knew what to look for, we would likely get an answer by creating a converged image of the scene, which, without the knowledge of what algorithm to use, may take up to days to complete. But, if we have created the image, it's too late, we would need this information before we start this rendering process. This way we can choose the right algorithm on the first try. With this technique that came more than 10 years after the metropolis paper, it is possible to mathematically formalize and quickly decide whether a scene is easy or difficult. The key insight is that in a difficult scene, we often experience that a completely random ray is very likely to be invalid. This insight, with two other simple metrics, gives us all the knowledge we need to decide whether a scene is easy or difficult. And the algorithm tells us what mixture of the two sampling techniques we exactly need to use to get beautiful images quickly. The more complex light transport algorithms get, the more efficient they become, but at the same time, we are wallowing in parameters that we need to set up correctly to get adequate results quickly. This way we have an algorithm that doesn't take any parameters, you just fire it up and forget about it. Like a good employee, it knows when to work smart and when a dumb solution with a lot of firepower is better. And it was tested on a variety of scenes and found close to optimal settings. Implementing this technique is remarkably easy. Someone who is familiar with the basics of light transport can do it in less than half an hour. Thanks for watching and for your generous support and I'll see you next time.
[{"start": 0.0, "end": 4.32, "text": " Dear Fellow Scholars, this is Two Minute Papers with Karo Zsolnai-Fehair."}, {"start": 4.32, "end": 9.0, "text": " We'll start with a quick recap on metropolis light transport and then discuss a cool technique"}, {"start": 9.0, "end": 10.52, "text": " that builds on top of it."}, {"start": 10.52, "end": 14.56, "text": " If we would like to see how digitally modeled objects would look like in real life, we"}, {"start": 14.56, "end": 20.0, "text": " would create a 3D model of the desired scene, assign material models to the objects within,"}, {"start": 20.0, "end": 23.44, "text": " and use a photorealistic rendering algorithm to finish the job."}, {"start": 23.44, "end": 28.16, "text": " It simulates rays of light that connect the camera to the light sources in the scene and"}, {"start": 28.16, "end": 30.96, "text": " compute the flow of energy between them."}, {"start": 30.96, "end": 35.0, "text": " Initially, after a few rays will only have a rough idea on how the image should look"}, {"start": 35.0, "end": 39.76, "text": " like, therefore our initial results will contain a substantial amount of noise."}, {"start": 39.76, "end": 44.84, "text": " We can get rid of this by simulating the path of millions and millions of rays that will"}, {"start": 44.84, "end": 47.04, "text": " eventually clean up our image."}, {"start": 47.04, "end": 52.28, "text": " This process, where a noisy image gets clearer and clearer, we call convergence, and the problem"}, {"start": 52.28, "end": 57.879999999999995, "text": " is that this can take excruciatingly long, even up to hours, to get a perfectly clear"}, {"start": 57.88, "end": 59.04, "text": " image."}, {"start": 59.04, "end": 63.2, "text": " With the simple algorithms out there, we generate these light paths randomly."}, {"start": 63.2, "end": 65.36, "text": " This technique we call path tracing."}, {"start": 65.36, "end": 70.4, "text": " However, in the scene that you see here, most random paths can't connect the camera"}, {"start": 70.4, "end": 75.24000000000001, "text": " and the light source because this wall is in the way obstructing many of them."}, {"start": 75.24000000000001, "end": 80.44, "text": " Light paths like these don't contribute anything to our calculations and are ultimately a waste"}, {"start": 80.44, "end": 82.88, "text": " of time and resources."}, {"start": 82.88, "end": 88.16, "text": " After generating hundreds of random light paths, we finally found a path that connects the"}, {"start": 88.16, "end": 91.6, "text": " camera with the light source without any obstructions."}, {"start": 91.6, "end": 95.88, "text": " When generating the next path, it would be a crime not to use this knowledge to our"}, {"start": 95.88, "end": 96.88, "text": " advantage."}, {"start": 96.88, "end": 101.52, "text": " A technique called metropolis light transport will make sure to use this valuable knowledge"}, {"start": 101.52, "end": 106.44, "text": " and upon finding a bright light path, it will explore other paths that are nearby to"}, {"start": 106.44, "end": 111.08, "text": " have the best shot at creating valid, unobstructed connections."}, {"start": 111.08, "end": 116.16, "text": " If we have a difficult scene at hand, metropolis light transport gives us way better results"}, {"start": 116.16, "end": 121.4, "text": " than traditional, completely random paths sampling techniques such as path tracing."}, {"start": 121.4, "end": 126.44, "text": " This scene is extremely difficult in a sense that the only source of light is coming from"}, {"start": 126.44, "end": 131.16, "text": " the upper left and after the light goes through multiple glass spheres, most of the light"}, {"start": 131.16, "end": 135.4, "text": " paths that we generate will be invalid."}, {"start": 135.4, "end": 140.56, "text": " As you can see, this is a valiant effort with random path tracing that yields really dreadful"}, {"start": 140.56, "end": 141.92000000000002, "text": " results."}, {"start": 141.92000000000002, "end": 146.2, "text": " Metropolis light transport is extremely useful in these cases and therefore should always"}, {"start": 146.2, "end": 147.96, "text": " be the weapon of choice."}, {"start": 147.96, "end": 152.4, "text": " However, it is more expensive to compute than traditional random sampling."}, {"start": 152.4, "end": 156.96, "text": " This means that if we have an easy scene on our hands, this smart metropolis sampling"}, {"start": 156.96, "end": 162.56, "text": " doesn't pay off and performs worse than a naive technique in the same amount of time."}, {"start": 162.56, "end": 168.48000000000002, "text": " So, on easy scenes, traditional random sampling, difficult scenes, metropolis sampling, super"}, {"start": 168.48, "end": 174.2, "text": " simple, super intuitive, but the million dollar question is how to mathematically formulate"}, {"start": 174.2, "end": 178.51999999999998, "text": " and measure what an easy and what a difficult scene is."}, {"start": 178.51999999999998, "end": 183.28, "text": " This problem is considered extremely difficult and was left open in the metropolis light transport"}, {"start": 183.28, "end": 185.23999999999998, "text": " paper in 2002."}, {"start": 185.23999999999998, "end": 189.32, "text": " Even if we knew what to look for, we would likely get an answer by creating a converged"}, {"start": 189.32, "end": 195.0, "text": " image of the scene, which, without the knowledge of what algorithm to use, may take up to days"}, {"start": 195.0, "end": 196.0, "text": " to complete."}, {"start": 196.0, "end": 201.56, "text": " But, if we have created the image, it's too late, we would need this information before"}, {"start": 201.56, "end": 203.76, "text": " we start this rendering process."}, {"start": 203.76, "end": 207.32, "text": " This way we can choose the right algorithm on the first try."}, {"start": 207.32, "end": 211.92, "text": " With this technique that came more than 10 years after the metropolis paper, it is possible"}, {"start": 211.92, "end": 217.48, "text": " to mathematically formalize and quickly decide whether a scene is easy or difficult."}, {"start": 217.48, "end": 222.36, "text": " The key insight is that in a difficult scene, we often experience that a completely random"}, {"start": 222.36, "end": 225.04, "text": " ray is very likely to be invalid."}, {"start": 225.04, "end": 229.67999999999998, "text": " This insight, with two other simple metrics, gives us all the knowledge we need to decide"}, {"start": 229.67999999999998, "end": 232.2, "text": " whether a scene is easy or difficult."}, {"start": 232.2, "end": 237.04, "text": " And the algorithm tells us what mixture of the two sampling techniques we exactly need"}, {"start": 237.04, "end": 240.2, "text": " to use to get beautiful images quickly."}, {"start": 240.2, "end": 244.84, "text": " The more complex light transport algorithms get, the more efficient they become, but at"}, {"start": 244.84, "end": 250.6, "text": " the same time, we are wallowing in parameters that we need to set up correctly to get adequate"}, {"start": 250.6, "end": 252.16, "text": " results quickly."}, {"start": 252.16, "end": 256.96, "text": " This way we have an algorithm that doesn't take any parameters, you just fire it up and"}, {"start": 256.96, "end": 258.2, "text": " forget about it."}, {"start": 258.2, "end": 262.84, "text": " Like a good employee, it knows when to work smart and when a dumb solution with a lot of"}, {"start": 262.84, "end": 265.04, "text": " firepower is better."}, {"start": 265.04, "end": 270.64, "text": " And it was tested on a variety of scenes and found close to optimal settings."}, {"start": 270.64, "end": 273.2, "text": " Implementing this technique is remarkably easy."}, {"start": 273.2, "end": 276.8, "text": " Someone who is familiar with the basics of light transport can do it in less than half"}, {"start": 276.8, "end": 277.8, "text": " an hour."}, {"start": 277.8, "end": 281.32, "text": " Thanks for watching and for your generous support and I'll see you next time."}]