Cover story: enhancing temporal resolution for lens-free on-chip video with subsampled phase retrieval (SPR).
"Subsampled phase retrieval for temporal resolution enhancement in lensless on-chip holographic video"
Donghun Ryu, Zihao Wang, Kuan He, Guoan Zheng, Roarke Horstmeyer, and Oliver Cossairt
OSA Biomedical Optics Express, 8 (3) 1981-1995 (2017)
"4D Tracking of Biological Samples using Lens-free On-chip In-line Holography"
Zihao Wang, Donghun Ryu, Kuan He, Oliver Cossairt, Aggelos K Katsaggelos
OSA Digital Holography and Three-Dimensional Imaging, May 2017
Phase retrieval from lens-free on-chip hologram
Simulation of standard in-line holography for a set of 5 microspheres. (a)-(b) Simu- lated sample’s original amplitude and phase, leading to (c) detected hologram intensities at sensor plane (d = 1 mm). (d)-(e) Recovered amplitude and phase using the ER phase retrieval algorithm (updating all pixel amplitudes). (f) The final sample support.
Proposed subsampled phase retrieval (SPR) algorithm
Phase retrieval algorithm for on-chip holography. Each step is detailed in the text. Subsampling (SPR) only modifies the constraint in step 3. This modification results in over an order of magnitude potential speedup for lensless holographic video.
Performance comparison for microspheres
Experimental results, on-chip imaging of polystyrene microspheres. (a) Raw detected hologram with one region of interest highlighted. (b) The recovered sample phase from the region of interest using the SPR algorithm (top and middle) and standard ER phase retrieval with interpolation (bottom). (c) Line traces through the center of the recovered microsphere phase (dashed lines) reveal quantitative agreement with the expected phase shift, even after reducing the number of pixels in factor of 9, 16 and 25.
Example: subsampled holographic reconstruction of in vivo Peranema in motion (subsampling factor vs. time). Horizontal axis depicts time and vertical axis represents subsampling factor. Reconstructions of both amplitude and phase using all pixels on the detector are shown at top, while reconstructions from subsampled pixel array data, using a factor of 4 and 9, are in middle and bottom, respectively. Consecutive frames show Peranema motion from left to right. Frame rate: first row – 4.4FPS, second row – 13.6FPS, third row – 24.8FPS. Scale bar is 22 μm. See Visualization 1, 2 and 3 for the full videos.
As we demonstrated both in simulation and experiment, SPR can dramatically reduce the number of measurements per frame in on-chip holography while still maintaining suitable reconstruction quality. Using a sensor that achieves a higher frame rate via pixel sub-sampling, we demonstrated a factor of 5.5× speedup in holographic video of moving microorganisms. Our experimental work demonstrates SPR is quite resilient to unknown sensor and shot noise. Placed in the context of alternative compressive holography schemes, SPR is simple, computationally efficient and accurate.
Several steps may help further improve the accuracy of subsampling. The primary challenge faced with live biological specimen imaging was correctly determining its support, using a somewhat arbitrary starting point. A modified shrink-wrap algorithm, which could incorporate prior knowledge of e.g. the Peranema body shape, would certainly help improve performance. In addition, the challenge of support identification becomes increasingly difficult when measuring from fewer pixels (i.e., with larger subsampling). Thus, a practical implementation might employ a bootstrapped approach, where the imaging process begins with a larger number of measured pixels per image and then forms a model of the expected sample support to use in later recon-
structions with fewer measured pixels. Despite limitations in our experimental hardware, we successfully demonstrated our Sub-sampled Phase Retrieval technique applied to real experimen- tal data, with a clear improvement in the fidelity of captured motion. SPR would ideally benefit from a fully addressable pixel readout scheme. In addition, we believe SPR and compressed sensing have similarity and would like to compare these two techniques side by side in future.
We believe SPR offers a useful conceptual starting point for more advanced procedures. First, SPR currently does not consider the redundant nature of the video signal over time. Adopting the insights gained by SPR into a more general approach to optimize phase retrieval over both space and time will likely lead to additional video speedup. Methods such as optical flow may provide a good path forward in this regard. Second, SPR is capable of removing objects that are not in focus, which offers a means to simultaneously achieve optical sectioning. Third, the effectiveness of SPR indicates that it might also be useful for X-ray imaging and coherent diffraction imaging, as well as related techniques for ptychography.
Z.W., K.H. and O.C. acknowledge the following funding:
NSF CAREER award IIS-1453192, ONR award 1(GG010550)//N00014-14-1-0741, ONR award N00014-15-1-2735, and DARPA award (G001534-7510)//HR0011-16-C-0028.
G.Z. acknowledges funding in part by NSF 1555986, NIH R21EB022378, and NIH R03EB022144
R.H. acknowledges financial support from the Einstein Foundation Berlin.
We are thankful for the discussions and generous help from Dr. Xiaoze Ou, Dr. Mooseok Jang and Jaebum Chung. We also thank Prof. Changhuei Yang at Caltech and Prof. Do Young Noh at GIST for their insights and the kind use of his equipment for experimental development. D.R. acknowledges Caltech Electrical Engineering department Fellowship.