SIGGRAPH Asia 2012

Quality Prediction for Image Completion

Johannes Kopf     Wolf Kienzle     Steven Drucker     Sing Bing Kang
Microsoft Research     Microsoft Research     Microsoft Research     Microsoft Research

Input panorama and quality prediction (brighter is higher quality) Full completion
Best crop using only “known” pixels (“conservative crop”) Our optimized crop based on quality prediction
Our data-driven technique is capable of predicting image completion quality (top left) before the completion is actually computed (top right). Based on our prediction, we compute an optimal crop rectangle that tries to include as many known pixels as possible while avoiding low-quality regions (bottom right). Compared to previous cropping approaches that do not fill in (bottom left) we can usually include a larger amount of the input image in our crop. Our algorithm only completes the cropped region, thus saving a significant amount of computation compared to full completion.


We present a data-driven method to predict the performance of an image completion method. Our image completion method is based on the state-of-the-art non-parametric framework of Wexler et al. [2007]. It uses automatically derived search space constraints for patch source regions, which lead to improved texture synthesis and semantically more plausible results. These constraints also facilitate performance prediction by allowing us to correlate output quality against features of possible regions used for synthesis. We use our algorithm to first crop and then complete stitched panoramas. Our predictive ability is used to find an optimal crop shape before the completion is computed, potentially saving significant amounts of computation. Our optimized crop includes as much of the original panorama as possible while avoiding regions that can be less successfully filled in. Our predictor can also be applied for hole filling in the interior of images. In addition to extensive comparative results, we ran several user studies validating our predictive feature, good relative quality of our results against those of other state-of-the-art algorithms, and our automatic cropping algorithm.
    author  = {Johannes Kopf and Wolf Kienzle and Steven Drucker and Sing Bing Kang},
    title   = {Quality Prediction for Image Completion},
    journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2012)},
    year    = {2012},
    volume  = {31},
    number  = {6},
    pages   = {to appear}
PDF (15.4 MB)
Supplementary Document
PDF (0.16 MB)
Supplementary Material
Click here