Imaging with light & computing
@ Zhejiang Lab
Research Summary
Our main interest is to sample high-dimensional optical information with innovative optics and computational algorithms to push vision foward.
Compact light field Photography (CLIP)
Modern cameras can attain three-dimensional vision via multi-view geometry like compound eyes in flies, or time-of-flight sensing like echolocation in bats. However, high-speed, accurate three-dimensional sensing capable of scaling over an extensive distance range and coping well with severe occlusions remains challenging. Compact light field photography is developed to acquire large-scale light fields with simple optics and a small number of sensors in arbitrary formats (from 2D to single-point detectors), thereby facilitating dense multi-view time-of-flight measurement with orders of magnitude lower dataload.
CLIP hence enables snapshot three-dimensional imaging with an extended depth range and through severe scene occlusions, which further allows curved and disconnected scattering surfaces to be exploited for non-line-of-sight 3D vision.
Fig 1. CLIP only sample a partial (but nonlocal) information in each angular view. All different views contains complementary measurements of the 3D scene for final reconstruction.
Light field tomography (LIFT)
Cameras with extreme speeds are enabling technologies in both fundamental and applied sciences. However, existing ultrafast cameras are incapable of coping with extended three-dimensional scenes and remain limited in frame depths and resolution. To address this long-standing challenge, we developed light field tomography (LIFT) for efficient sampling of light fields and snapshot acquisition of large-scale 2D time-resolved data.
LIFT transforms a one-dimensional (1D) sensor to a 2D light field camera, exploiting the fact conventional light field acquisition is highly redundant. The vastly faster frame rate of 1D sensors benefits LIFT for high speed imaging. Coupled with a streak camera, LIFT can capture the complete four-dimensional spatiotemporal space in a single snapshot and provide an image resolution over 120 × 120 with a sequence depth beyond 1000, enabling unprecedented ultrafast 3D imaging capabilities
Fig 2. a, LIFT uses cylindrical lens for image formation; b, its mathmatical model; c, Ultrafast LIFT camera.
Time-of-flight Imaging with scattering medium
Looking through and inside scattering medium is important in many areas of fundamental and applied science. When passing through a scattering medium, light is scattered multiple times, and due to the random nature of this process, a photon lost its original direction, and the object light field are spatiotemporally broadened on a macroscopic level, making image retrieval challenging. Conventional imaging like diffuse optical tomography (DOT) requires an decent prior knowlede of medium’s scattering and absorption coefficients to retrieve the image of interest, or even a reference measurement with the same imaging geometry on a homogenous medium, which is much less practical.
We are seeking to imaging inside or through scattering medium without prior knowledge by mimicing imaging in the clear medium, which are typically qualitative rather than quantitative, and relied on less scattered photons, whose scarcity and separation may be addressed by time-of-flight single photon detection.
Fig 3. a, Single pixel imaging is more robust to scattering effects; b, With time-of-flight single photon detection, we developed algorithms that need no prior knowledge for 3D image retrieval inside scattering medium.