SIGGRAPH Asia 2013


Johannes Kopf     Fabian Langguth     Daniel Scharstein     Richard Szeliski     Michael Goesele
Microsoft Research     TU Darmstadt     Middlebury College     Microsoft Research     TU Darmstadt

Standard image-based rendering synthesizes novel views of a scene by reprojecting the input image (a) using a coarse estimated depth map (b). The technique cannot handle scenes with reflections and has problems in untextured regions; holes show up in the reprojection (c). We perform image-based rendering in the gradient domain. Our technique synthesizes horizontal and vertical gradient fields Fx and Fy, as well as an approximate solution S (d). The final image is obtained through Poisson integration (e). Our technique naturally handles reflections and untextured regions.




Abstract

We propose a novel image-based rendering algorithm for handling complex scenes that may include reflective surfaces. Our key contribution lies in treating the problem in the gradient domain. We use a standard technique to estimate scene depth, but assign depths to image gradients rather than pixels. A novel view is obtained by rendering the horizontal and vertical gradients, from which the final result is reconstructed through Poisson integration using an approximate solution as a data term. Our algorithm is able to handle general scenes including reflections and similar effects without explicitly separating the scene into reflective and transmissive parts, as required by previous work. Our prototype is fully implemented on the GPU and runs in real time on commodity hardware. 
@article{Kopf2013,
    author  = {Johannes Kopf and Fabian Langguth and Daniel Scharstein
               and Richard Szeliski and Michael Goesele},
    title   = {Image-Based Rendering in the Gradient Domain},
    journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2013)},
    year    = {2013},
    volume  = {32},
    number  = {6},
    pages   = {to appear}
}
   
Paper
Click here (23.3 MB)
Supplementary Video
Click here
Music courtesy of Nadjana Gaffron
https://soundcloud.com/nadjana/sycamore
Supplementary Material
Click here
Implementation Details
Click here