Project Description
We propose a near-light illumination model for image relighting and 3D shape recovery. We correct the near-light distance effect to provide much more uniformly lit images that yield more appealing image for relighting applications. Furthermore, we use our near-light model for more accurate photometric stereo calculations of surface normals, eliminating the “potato-chip” shaped surface reconstruction error that results from violating the far-light assumption. We verify our model with both free-form capture using hand-held flash as the illumination source, and capture using LED lights mounted on a dome shaped surface.
Publications
"Near Light Correction for Image Relighting and 3D Shape Recovery"
Xiang Huang, Marc Walton, Greg Bearman and Oliver Cossairt
Digital Heritage Granada Conference, September 2015
[PDF]
Downloads
Data set and Matlab Source Code (Uploaded Apr 08, 2016, still need clean up).
Images
Our Capture Devices:
We used two types of capture. Left shows free-form capture using hand-held flash as the illumination source. Right image shows capture using LED lights mounted on a dome shaped surface.
Comparision of Image Relighting without and with Near Light Correction:
Relighting comparisons for two works by Paul Gauguin housed at the Art Institute of Chicago, a woodblock (top), and a transfer print (bottom). The concession number for the woodblock
is 1940-91, and 2002-237 for the print. A comparison is shown between the raw captured images (left) and after the near-light correction technique introduced in this paper(right). We use the calibrated light position to compute the light attenuation due to the distance squared fall-off. The inverse of this attenuation mask (middle) is used to produce relit images with even illumination (right). The corrected images look uniformly lit and more visually pleasing.
"Potato-chip" Error of Reconstructed Shape Due to Wrong Light Model:
This figure shows the typical "Potato-chip" shaped error resulted from modeling near lights as far lights, which typically occur in the Reflectance Transfer Image (RTI). In this example the object is 50cm wide, smooth and uniformly flat, and the light is 150cm above the object. The center of the object has an incident illumination angle of 90
degrees, but the left and right border of the central scan line have incident illumination angles of 80.5 degrees and 99.5 degrees respectively, with an error range of 19=2atan(25/150)
degrees from left to right. The erroneous lighting angles cause mis-estimated surface normals in RTI, which causes the ``potato-chip'' shaped surface.
Comparision of Surface Reconstruction without and with Near Light Correction I:
Experimental comparison between far-light and near-light models for surface normal and 3d shape reconstruction. We use a flat matte paper as a ground truth test object. The first column shows the captured and corrected images. The position of the red dot in the circle approximates lighting direction. The second column shows the surface normal. The ground truth should be uniform, but the normal from the distant light model has a large error especially at the border. The third column shows the reconstructed 3d surface. The last column shows a horizontal scan line of the recovered depth map. The near-light model results are close to ground truth, while the far-light model results in a large potato chipped distortion.
Comparision of Surface Reconstruction without and with Near Light Correction II:
Comparison of far-light and near-light models for 3d shape reconstruction. The work of art shown is a woodblock produced by artist Paul Gauguin, housed at the Art Institute of Chicago. The concession number of the artwork is 2238-56. (a) One of the captured images showing the near-light illumination effect. (b) A 3D reconstruction without correcting for the near-light effect. The woodblock appears to be bent like a ``potato chip'' so surface details are difficult to resolve. (c) A 3D reconstruction using our method to correct for the near-light effect. The woodblock is now flat and details of the carving can be discerned from the reconstruction.
Acknowledgements
This project was funded in part through a NU-ACCESS exploratory grant and NSF CAREER grant IIS-1453192.