Overview of Shape from Shifting: Our shape-from-shifting technique uses a mobile device camera to capture images around the object with the built-in flash used as a source of illumination. SIFT-based image registration renders the images to the same viewpoint but each with a unique illumination direction. The synthesized images are further processed by uncalibrated photometric stereo to acquire dense surface normal vector maps.
Project Description
Surface shape scanning techniques, such as laser scanning and photometric stereo, are widespread analytical tools used in the field of cultural heritage. Compared to regular 2D RGB photos, 3D surface scans provide higher fidelity of an object’s surface shape which assist conservators, art historians, and archaeologists in understanding how these artworks and artifacts are made and to digitally document them for purposes of conservation. However, current state-of-the-art 3D surface scanning tools used in art conservation are often expensive and bulky- such as light dome structures that are often over 1 m in diameter. In this paper, we introduce mobile shape-from-shifting (SfS): a simple, low-cost and streamlined photometric stereo framework for scanning planar surfaces with a consumer mobile device coupled to a low-cost add-on component. Our free-form mobile SfS framework relaxes the rigorous hardware and other complex requirements inherent to conventional 3D scanning tools. This is achieved by taking a sequence of photos with the on-board camera and flash of a mobile device. The sequence of captures are used to reconstruct high quality normal maps using near- light photometric stereo algorithms, which are of comparable quality to conventional photometric stereo. We demonstrate 3D surface reconstructions with SfS on different materials and scales. Moreover, the mobile SfS technique can be used ”in the wild” so that 3D scans may be performed in their natural environment, eliminating the need for transport to a laboratory setting. With the elegant design and low cost, we believe our Mobile SfS can greatly benefit the conservation community by providing a user-friendly and cost-effective solution for 3D surface scanning.
Publications
"Shape-from-Shifting: Uncalibrated Photometric Stereo with a Mobile Device"
Chia-Kai Yeh, Fengqiang Li, Gianluca Pastorelli, Marc Walton, Aggelos K. Katsaggelos and Oliver Cossairt
IEEE International Conference on eScience Workshop on High Throughput Digitization for Natural History Collections (BigDig) 2017
[PDF]
Images
Mobile SfS hardware
(a). two polarizers with opposite polarization
are attached to the camera and the flash light of the mobile device, respectively;
(b). photo of our prototype Mobile SfS with an iPhone 6 and a custom
3D printed widget.
Samples for evaluation with Mobile Sf
A duplicate (a) of Aztec calendar stone (b); a portion of the wall of Bah´a’´ı Temple (c); and a page of an
old manuscript (d).
Acquisition procedure with Mobile SfS
Hand-held Mobile SfS faces
to the object. Slightly shift the phone and take one image at each position.
Nine images are taken in this paper for the surface normal reconstruction.
Image pre-processing
Step 1: nine images are captured; Step 2:
images are registered with the SIFT function; Step 3: images are cropped to
display the same region.
Surface normal for the plaster replica of an Aztec calendar stone
(a). the photo of the imaged sample; (b). the surface normal generated with
near-light PS using light dome; (c). the surface normal generated using mobile SfS but without polarizers; (d). the surface normal reconstructed with the
proposed mobile SfS. Close-up images show the ”nose” on the plate. Scale bar: 5 cm.
Surface normal for the architectural elements
(a). photo of the region of Bah´a’´ı Temple which is been scanned; (b). the surface normal generated
with mobile SfS; (c). the rendering with surface normal generated with mobile SfS. Close-up shows the detailed feature on the stone. Scale bar: 10 cm.
Surface normal for a page of the manuscript page
(a). the RGB photo of portion of the page of the manuscript; (b). the surface normal
reconstruction with mobile SfS for that region. Scale bar: 5 mm.
Acknowledgements
This project was supported by National Science Foundation (NSF) CAREER grant IIS-1453192, and a generous grant from the Andrew W. Mellon Foundation to NU-ACCESS. The authors would like to thank Alec Reinke for the support of the hardware prototype and US. Bah́a’́ı National Center for their permission to image the wall of the temple.
Shape-from-Shifting: Uncalibrated Photometric Stereo with a MobileDevice
