Figure 1: Our sequential optimal design algorithm vs. other commonly used design algorithms: Unlike PCA and Hadamard based designs, our design approach takes both the signal and noise statistics into account and sequentially selects projection directions that are oriented along the posterior distribution after accounting for previous measurements and noise statistics.
Project Description
Computational Imaging (CI) systems that exploit optical multiplexing and algorithmic demultiplexing have been shown to improve imaging performance in tasks such as motion deblurring, extended depth of field, light field and hyper-spectral imaging. Design and performance analysis of many of these approaches tend to ignore the role of image priors. It is well known that utilizing statistical image priors significantly improves demultiplexing performance. In this paper, we extend the Gaussian Mixture Model as a data-driven image prior to under-determined linear systems and study compressive CI methods such as light-field and hyper-spectral imaging. Further, we derive a novel algorithm for optimizing multiplexing matrices that simultaneously accounts for (a) sensor noise (b) image priors and (c) CI design constraints. We use our algorithm to design data-optimal multiplexing matrices for a variety of existing CI designs, and we use these matrices to analyze the performance of CI systems as a function of noise level. Our analysis gives new insight into the optimal performance of CI systems, and how this relates to the performance of classical multiplexing designs such as Hadamard matrices.
A database of light field images used in this project is available here.
Publications
"Can we beat Hadamard? Data-driven design and Analysis of Computational Imaging"
K. Mitra, O. Cossairt, A. Veeraraghavan
International Conference on Computational Photography (ICCP), May 2014
[PDF]
Images
Original vs. optimized compressive hyper-spectral systems:
(Left) performance of CASSI-Dual Dispersion, multimodal LF camera and GAP camera. The SNR gain (with multimodal LF as reference) for different values of photon-to-read noise ratio. CASSI-DD, which uses Hadamard multiplexing, performs better in low light. In high light, CASSI-DD and GAP have similar performance. (Right) Data-driven optimization of the masks/filters results in improved performance for all systems. The Optimized GAP camera achieves the best
A comparison between original and optimized hyper-spectral systems:
Example reconstructions for a sample hyperspectral dataset are shown for Generalized Assorted Pixels (GAP), CASSI-Dual Dispersion, and multimodal LF cameras. Performance is increased for each camera after optimization. These simulations correspond to moderate light level (the photon to read noise ratio is 50).
Optimized spectral filters for GAP:
The optimized 16 spectral filters each of length 16 (arranged as 16 x 16 matrices with each column being a spectral filter) for different light levels. Our sequential optimization algorithm takes noise statistics into account and hence at low light levels we obtain repeated all-one spectral filters whereas at high light we obtain diverse spectral filters.
lassical vs. optimized compressive light field systems:
For snapshot light field capture, data-optimized multiplexing performs similar to classical multiplexing. The performance of multi-frame capture is plotted as the dotted black line (2-frames) and the solid black line (5-frames). For multi-frame capture, data-optimized multiplexing performs much better in high light.
A comparison between original and optimized light field cameras:
Comparison of original and optimized light-field systems for a sample dataset shows improved performance. These simulations corresponding to a very low light level (photon to read noise variance is 1/5).
Acknowledgements
Kaushik Mitra and Ashok Veeraraghavan acknowledge support through NSF Grants NSF-IIS: 1116718, NSF-CCF:1117939 and a Samsung GRO grant. Oliver Cossairt acknowledges support through a Samsung GRO grant.
