Comparison of depth images between regular LR ToF measurements and HR reconstruction with our proposed CS-ToF framework with different compression ratios.
Project Description
Three-dimensional imaging using Time-of-flight (ToF) sensors is rapidly gaining widespread adoption in many applications due to their cost effectiveness, simplicity, and compact size. However, the current generation of ToF cameras suffers from low spatial resolution due to physical fabrication limitations. In this project, we propose CS-ToF, an imaging architecture to achieve high spatial resolution ToF imaging via optical multiplexing and compressive sensing. Our approach is based on the observation that, while depth is non-linearly related to ToF pixel measurements, a phasor representation of captured images results in a linear image formation model. We utilize this property to develop a CS-based technique that is used to recover high resolution 3D images. Based on the proposed architecture, we developed a prototype 1-megapixel compressive ToF camera that achieves as much as 4 times improvement in spatial resolution and 3 times improvement for natural scenes. We believe that our proposed CS-ToF architecture provides a simple and low-cost solution to improve the spatial resolution of ToF and related sensors.
Publications
“CS-ToF: High-resolution compressive time-of-flight imaging”
Fengqiang Li, Huaijin Chen, Adithya Pediredla, Chiakai Yeh, Kuan He, Ashok Veeraraghavan, and Oliver Cossairt
Optics Express, 25(25) 31096-31110, 2017
News
Images
Continuous-wave ToF imaging
In regular continuous-wave ToF camera, the controller outputs two RF signals: m(t) to modulate the amplitude of the laser diode and r(t) as reference signal to ToF camera. The reflection from object goes to ToF camera. The depth information is estimated by comparing the phase delay between the reflection signal am(t-ϕ) and the reference signal r(t).
Proposed CS-ToF framework
In our proposed CS-ToF framework, the light from the laser diode hits the object and is reflected and imaged on the high resolutional DMD (small pixel size and more pixel numbers compared to ToF pixel). Then, the scene on DMD is projected on the low resolutional ToF camera via a relay lens. The key idea is to use the low resolutional ToF measurements to reconstruct the high resolution scene on DMD plane.
CS-ToF prototype
This shows the components for CS-ToF prototype system. DMD: Texas Instrument DLP 4500. ToF camera: Texas Instrument OPT 8241 320×240 pixels.
Pixel scanning of the USAF resolution target
This is to demonstrate the upper bound improvement with our proposed CS-ToF framework. We compare the original low-resolution ToF measurement of the resolution chart target (a) and the pixel-wise scanning for the resolution target with the proposed CS-ToF framework. Fine details are also shown marked with the color boxes. As we can see from the comparison, the improvement is about 4 X.
Phase (or depth) images for a natural scene
Comparisons between ToF raw measurements and high resolution reconstruction with our proposed CS-ToF framework. We compare the phase images from LR ToF measurement (a), HR CS-ToF reconstruction with no compression (b), HR reconstruction with compression ratio of 0.6 (c) and 0.25 (d). We also shown the zoom-in details of the depth images where more fine details can be visualized from the reconstruction results with the proposed framework. For example, leaves and their branches on ”toy tree" are clearly seen from the reconstruction with high quality, which, however, totally missed in the low resolutional ToF measurements.
Intensity images for a natural scene
We show the intensity images from LR ToF measurement (a), HR CS-ToF reconstruction intensity image with no compression (b), HR reconstruction with compression ratio of 0.6 (c), and 0.25 (d). Fine patterns can be visualized from the reconstruction results with the proposed CS-ToF framework, such as the the screw cap and the tips on the metal strike.
Acknowledgements
This project was partially supported by NSF CAREER IIS-1453192, CAREER IIS-1652633, and CCF-1527501, ONR N00014-15-1-2735, DARPA REVEAL HR0011-16-C-0028, and Texas Instruments (TI) Graduate Student Fellowship.
