Neural Two-Level Monte Carlo Real-Time Rendering

Karlsruhe Institute of Technology
Eurographics, 2025 - Computer Graphics Forum

Received ⭐"Best Paper Honorable Mention"⭐

Abstract

We introduce an efficient Two-Level Monte Carlo (subset of Multi-Level Monte Carlo, MLMC) estimator for real-time renderingof scenes with global illumination. Using MLMC we split the shading integral into two parts: the radiance cache integral andthe residual error integral that compensates for the bias of the first one. For the first part, we developed the Neural Incident Radiance Cache (NIRC) leveraging the power of tiny neural networks [MRNK21] as a building block, which is trained on thefly. The cache is designed to provide a fast and reasonable approximation of the incident radiance: an evaluation takes 2-25× less compute time than a path tracing sample. This enables us to estimate the radiance cache integral with a high number ofsamples and by this achieve faster convergence. For the residual error integral, we compute the difference between the NIRCpredictions and the unbiased path tracing simulation. Our method makes no assumptions about the geometry, materials, orlighting of a scene and has only few intuitive hyper-parameters. We provide a comprehensive comparative analysis in differentexperimental scenarios. Since the algorithm is trained in an on-line fashion, it demonstrates significant noise level reductioneven for dynamic scenes and can easily be combined with other noise reduction techniques

We've already got NRC... why NIRC?

Our Neural Incident Radiance Cache (NIRC) learns incoming light from all directions in real-time, unlike NRC [MRNK21] which predicts outgoing light. This key difference enables several advantages:

⚡Immediate cache invokation at primary surface w/o additional rays, but with BSDF properties preservation 🎨

⚡Multiresolution neural hash-grid sampling costs amortized across all directional samples (~70% time of original NRC)

🚀With inlined invokations performance scales directly with tensor-core compute rather than bandwidth, benefiting from the recent GPU-trends and low-precision TFLOPs

So, if many samples invoked, a NIRC's one is cheaper in 5-10x times compared to NRC's one on average.

Performance Comparison — Performance study with the Bistro Exterior scene rendered on an RTX 3080 at 1080p. Path-Tracing vs NRC vs NIRC (Ours) with a few low-level optimizations.

On-the-fly NIRC visualization for an arbitrary point in the Sponza scene

Scalability from tensor-core compute 💻

Reference

For more details and full context, see Figure 15 in the paper, as it doesn't always scale well for noisy scenes.

2 Layers
MSE: 0.56, 0.84ms

4 Layers
MSE: 0.27, 1.24ms

6 Layers
MSE: 0.14, 1.64ms

Reference

2 Layers
MSE: 0.23, 0.84ms

4 Layers
MSE: 0.21, 1.24ms

6 Layers
MSE: 0.13, 1.64ms

Reference

All timings on RTX 3080

🤔 Two-Level Monte Carlo?

BibTeX

@article{https://doi.org/10.1111/cgf.70050, author = {Dereviannykh, Mikhail and Klepikov, Dmitrii and Hanika, Johannes and Dachsbacher, Carsten}, title = {Neural Two-Level Monte Carlo Real-Time Rendering}, journal = {Computer Graphics Forum}, volume = {44}, number = {2}, pages = {e70050}, keywords = {CCS Concepts, • Computing methodologies → Ray tracing, Neural networks}, doi = {https://doi.org/10.1111/cgf.70050}, url = {https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.70050}, eprint = {https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.70050}, abstract = {Abstract We introduce an efficient Two-Level Monte Carlo (subset of Multi-Level Monte Carlo, MLMC) estimator for real-time rendering of scenes with global illumination. Using MLMC we split the shading integral into two parts: the radiance cache integral and the residual error integral that compensates for the bias of the first one. For the first part, we developed the Neural Incident Radiance Cache (NIRC) leveraging the power of tiny neural networks [MRNK21] as a building block, which is trained on the fly. The cache is designed to provide a fast and reasonable approximation of the incident radiance: an evaluation takes 2–25 × less compute time than a path tracing sample. This enables us to estimate the radiance cache integral with a high number of samples and by this achieve faster convergence. For the residual error integral, we compute the difference between the NIRC predictions and the unbiased path tracing simulation. Our method makes no assumptions about the geometry, materials, or lighting of a scene and has only few intuitive hyper-parameters. We provide a comprehensive comparative analysis in different experimental scenarios. Since the algorithm is trained in an on-line fashion, it demonstrates significant noise level reduction even for dynamic scenes and can easily be combined with other noise reduction techniques.} }