Computational 3D Imaging with High Temporal, Lateral, and Depth Resolution

Li, Fengqiang

doi:https://doi.org/10.21985/n2-yk5p-xg18

Work

Computational 3D Imaging with High Temporal, Lateral, and Depth Resolution

Public

Download PDF

Download All Files (.zip)

Three-dimensional (3D) imaging has been widely used in academic research and industrial applications. Compared to 2D representations, 3D imaging can yield more information about geometric structures of an object such as small surface variations that are difficult to perceive otherwise. 3D image contents provide additional information that is complementary to computer vision (CV) techniques that rely on 2D images alone. Unfortunately, the CV community still relies on 3D sensors with poor lateral and depth resolutions, due to the technical limitations of commercially available 3D sensors. In this thesis, we introduce novel computational 3D sensors that deliver scanning at high temporal, lateral, and depth resolution for a wide range of material properties (e.g., transparent, translucent, and opaque objects). Recently, continuous-wave amplitude-modulated time-of-flight (ToF) cameras (a.k.a ToF cameras) have become a popular 3D imaging technique due to their compactivity and low consumption of computation power. However, current commercially available ToF cameras suffer from low spatial and depth resolutions due to physical limitations. To break these limitations, we leverage compressed sensing with optical multiplexing to improve the spatial resolution of a commercial ToF camera. Moreover, we utilize multi-frame super-resolution technique to improve spatial resolution of a ToF camera without adding any extra hardware components. For the depth resolution of a ToF camera, it is inversely proportional to the modulation frequency. Current ToF cameras usually use tens of MHz modulation frequencies resulting in a centimeter-level depth resolution. To break the limitation and improve the depth resolution, a higher modulation frequency (e.g., GHz) is required. However, the silicon imager used in ToF cameras can not respond to very higher modulation frequencies (e.g., GHz), which fundamentally limits the depth resolution of current ToF cameras. In this thesis, we break this limitation and propose a ToF camera by leveraging optical interferometry and wavelength diversity to produce GHz to THz modulation frequencies, resulting in a micro depth resolution. Specifically, we exploit superheterodyne interferometry with two closely spaced wavelengths to generate a tunable modulation frequency from GHz to THz. Simulations and real-world experiments are performed to verify the proposed sensor. In the experimental prototype, tunable light sources are used to provide the flexibility of imaging range and depth resolution. We have implemented the proposed principle with four different sensor architectures including an avalanche photodiode (APD), a lock-in camera, a flutter shutter camera, and a regular CMOS camera, where we exploit different lock-in frequencies, data capture speeds, and spatial data scan densities. To the best of our knowledge, the APD prototype represents the first documented demonstration of full-field 3D scanning on rough-surface objects using superheterodyne interferometry. Focal-plane-array (FPA) sensors of the lock-in camera, the flutter shutter camera, and the regular CMOS camera avoid mechanical scanning of 3D objects. Performances and trade-off with implementations using different FPA sensors are analyzed and we suggest an optimal selection for users in different applications. Although the proposed ToF 3D sensor with optical interferometry helps improve the depth resolution, it requires a reference beam which can reduce illumination powers to objects and can be sensitive to environmental vibrations. To remove this limitation, we propose and prototype a computational wavefront sensor by leveraging phase retrieval algorithms with wavelength diversity to provide mega-pixel spatial and micro depth resolutions. The proposed 3D sensor with wavefront sensing is very compact which can open doors for microscopy applications where a miniature device is preferred. Lastly, we propose methods to couple our 3D sensor together with a high-resolution RGB camera to produce a 3D scan with high spatial resolution, micro depth resolution, and full color.

Creator