News

7th PhD of Tian lab: Hao Wang. Congratulations!

September 25th, 2024in Graduation, People

Wednesday, September 25, 2024

11:00 a.m. - 1:00 p.m

Advancing Optical Metrology Through Computational Imaging Techniques

Committee: Professor Lei Tian, Professor Vivek Goyal, Professor Janusz Konrad, Professor Michelle Sander

Abstract

Optical metrology, which uses light to precisely measure and characterize objects, is essential in industrial manufacturing, scientific research, and engineering design. Computational optical imaging techniques have leveraged novel optical systems, advanced algorithms, and increased computing power to accurately reconstruct and analyze the physical properties of objects, surpassing the capabilities of traditional optical metrology methods. This thesis explores the advancement of optical metrology through computational imaging techniques, focusing on the following three primary research areas.

The first research focuses on semiconductor chip surface topography measurement, which plays a critical role in various semiconductor industrial applications. Conventional optical metrology techniques often face challenges in balancing field-of-view (FOV) and spatial resolution. To overcome these limitations, I propose a novel technique termed Fourier Ptychographic Topography (FPT). FPT provides wide FOV, high resolution, nanoscale height reconstruction accuracy. Experimental validation on a sample semiconductor chip also demonstrates the robustness and superior performance of FPT compared to current standard optical profilometry methods.

In the second study, I extend my research from surface metrology to large-scale 3D particle detection from a single hologram. Traditional 3D particle localization methods often rely on single-scattering models, which are limited in their ability to accurately capture dense particle distributions at deep imaging depths. To address this limitation, I use a multiple-scattering beam propagation method (BPM) to model multiple scattering effect of 3D particle fields. I achieve robust detection of dense 3D particles and experimental results demonstrate that my approach provides significantly higher accuracy compared to conventional single-scattering models, particularly in scenarios with high particle densities and deep imaging depths.

Finally, I explore the application of deep learning methods for phase retrieval, a fundamental problem in computational imaging-based optical metrology. My previous research utilized physical models to simulate the optical imaging process. While these traditional model-based methods achieved state-of-the-art results in many ap-plications, the development of deep learning methods introduces new possibilities for solving highly ill-posed problems that were previously unsolvable. In my third study, I introduce the Neural Phase Retrieval (NeuPh) network, which enables flexible object representations and resolution-enhanced phase reconstruction from multiplexed Fourier ptychographic microscopy (FPM). Experimental results demonstrate the NeuPh networks scalability, robustness, accuracy, and generalizability in solving the highly ill-posed phase retrieval problem, outperforming existing methods and offering potential wide applications in optical metrology techniques.

Through these investigations, this thesis advances optical metrology by integrating computational imaging techniques, offering improved measurement accuracy, scalability, and robustness and paving the way for enhanced optical metrology methodologies in various applications.

 

 

6th PhD of Tian lab: Joseph Greene!

June 20th, 2024in People

Computational Extended Depth of Field Fluorescence Microscopy in Miniaturized and Tabletop Platforms

Advisor: Professor Lei Tian

Committee: Professor Jerome Mertz, Professor Janusz Konrad, Professor Abdoulaye Ndao, Professor Xiaojun Cheng, Professor Tianyu Wang

Thursday, June 20, 2024

12:00 p.m. - 2:00 p.m.

Abstract

Fluorescence microscopy has emerged as a powerful solution to enable the direct study of biological compounds by labelling key structures and dynamics with optically responsive fluorophores to push fields ranging from medicine to biology to neuroscience. Due to its low-cost, flexible architecture and widefield imaging capabilities, one-photon epi-fluorescence microscopes have emerged as a standard platform for interrogating samples both in miniaturized and tabletop applications in real time. However, these systems are plagued by the low power of fluorescence samples, a lack of optical sectioning, susceptibility to scattering and shallow optical depth-of-field (DoF). As a result, collected signals exhibit low signal-to-noise and signal-to-background, high aberration and emerge from a constrained volume near the surface of the sample. To overcome these challenges, this thesis introduces several flexible and generalizable computational imaging frameworks to co-optimize custom optics and algorithms to encode target signals over a significantly enlarged extended depth of field (EDoF) then computational extract those signals from the high perturbations. This design paradigm fundamentally relies on concepts of pupil engineering, which uses a Fourier optics description of light propagation to design custom flat phase masks on the often-vacant pupil plane of a standard microscope to perform optical encoding.

This thesis begins by introducing a novel miniaturized EDoF-enabled miniscope, entitled EDoF-Miniscope, to motivate the utility of EDoF fluorescence imaging in a miniaturized form factor and on the complex environment of the brain. This project utilizes an innovative genetic algorithm to optimize a miniaturized and lightweight binary diffractive optical element (DOE) on the pupil plane to extend the DoF over 2.8x utilizing twin imaging foci. This project represents the first successful integration of diffractive optics into a miniscope to enable unprecedented control of the optical field inside the brain. To keep EDoF-Miniscope broadly accessible, I then utilize a simple off-the-shelf post-processing filter which enables the recovery of neuronal sources down to an SBR of 1.08. Next, I seek to improve upon the proposed framework in several capacities by designing a flexible 1-photon widefield tabletop testbed that exhibits comparable field-of-view (FoV), NA and aberrations to a miniscope entitled EDoF-Tabletop. This platform utilizes a spatial light modulator (SLM) on the pupil plane to rapidly deploy optimized phases without the need of manufacturing and aligning miniaturized optics. To improve both the optimization of novel pupil phases as well as the reconstruction algorithm, EDoF-Tabletop incorporates a novel end-to-end deep learning pipeline. Innovative physical modeling, optimization and initialization strategies are employed to make the framework computationally efficient, stable and reliably converge to the desired EDoF without the need for explicit initialization. By incorporating rigorous physical modeling, this thesis is able to perform cutting edge encoding and recovery of sources up to 140 microns (1.4 scattering lengths) deep in scattering media or over 400 microns deep in non-scattering samples without sacrificing the NA (NA=0.5), speed nor form factor.

Sponsoring a flexible optimization pipeline with a co-optimized reconstruction network and highly accurate physical simulator, the EDoF pipeline presented in this thesis offers a generalized solution for developing pushing fluorescence microscopy. By demonstrating this technology across miniaturized and tabletop platforms as well as across a broad range of fluorescence and biological samples, I believe that this framework will continue to push the practicality of 1-photon fluorescence imaging across a wide range of fluorescence imaging applications.

Jeffrey and Qianwan in CVPR CCD workshop

June 18th, 2024in Conference

Computational multi-aperture camera for wide-field high-resolution imaging Qianwan Yang

Behind the Blurry Background: Practical Synthetic Features To Enable Robust Imaging Through Scattering Jeffrey Alido

CISL at Optica Imaging Congress 2023

August 20th, 2023in Conference

Hao Wang
Wide-field, high-resolution reflection-mode Fourier ptychographic microscopy
* best student paper award 1st place

Jiabei Zhu
3D Phase Imaging from Intensity Measurements with Non-Paraxial Multiple Scattering Model
* best student paper award 1st place
* featured in Optica news.

 

Qianwan Yang
Advancing Computational Miniature Mesoscope With Simulator-Based Deep Learning Reconstruction
* featured in Optica news.

 

Guorong Hu
Caustic Illumination-based HiLo Microscopy

Lei Tian
Computational 3D Phase Imaging by Intensity Diffraction Tomography * invited

Lei Tian
Computational Miniature Mesoscope: augmenting miniature optics with algorithms for large-scale 3D fluorescence imaging * invited

Shiyi defended PhD!

July 10th, 2023in People

Congratulations, Dr. Cheng!

=============================

Title:                    Augmenting Label-free Imaging Modalities with Deep Learning based Digital Staining
Presenter:          Shiyi Cheng
Date:                    Monday, July 10th, 2023
Time:                   11:00 am to 1:00 pm
Location:             8 Saint Mary's Street, Room 339
Advisor:              Professor Lei Tian
Chair:                   Professor Ari Trachtenberg
Committee:        Professor Lei Tian, Professor Eshed Ohn-Bar, Professor David A. Boas, Professor Irving Bigio, Professor Ji-Xin Cheng.

Abstract:

Label-free imaging techniques provide valuable insights into biological samples and processes in their native states, eliminating the need for labor-intensive and potentially disruptive processes of physical staining. However, these methods often lack structural and molecular specific information. To overcome this limitation, recent advances in deep learning based digital staining techniques have shown the ability to virtually introduce digital labels or stains into label-free images, which enables extracting rich information that would typically require physical staining. The integration of label-free imaging and digital staining holds great potential for significantly expanding the toolkit for biomedical imaging, facilitating improved analysis, and enhancing our understanding of biomedical sciences at both the cellular and tissue level. In this thesis, I explore supervised and semi-supervised methodologies for digital staining and their applications in augmenting label-free imaging, with a focus on imaging cytometry and human brain imaging.

In the first part of the thesis, I present a novel integration of multi-contrast dark-field reflectance microscopy and digital staining by supervised deep learning. This method enables multiplexed immunofluorescence labeling of subcellular features and single cell cytometry. By leveraging the rich structural information and sensitivity of reflectance microscopy, the digital staining method accurately predicts subcellular features and achieves up to 3 times improvement in prediction accuracy over the state-of-the-art techniques. Additionally, the method accurately reproduces single-cell level structural phenotypes related to cell cycles. The multiplexed digital labeling enables multi-parametric single-cell profiling across a large cell population.

In the second part, I developed a novel semi-supervised digital staining technique for serial sectioning OCT (S-OCT) for 3D histological imaging of human brain tissue. The deep learning model integrates unpaired image translation, a biophysical model, and unsupervised cross-modality image registration. The digital staining model enables the translation of S-OCT images to Gallyas silver staining, provides consistent staining quality across different samples, and enhances contrast across cortical layer boundaries, enabling reliable layer differentiation. Importantly, the integration of S-OCT and digital staining allows volumetric histological imaging while preserving complex 3D geometry on centimeter-scale brain tissue blocks. In addition, our pilot study demonstrates promising results on other anatomical regions acquired from different S-OCT systems.

In summary, I investigated deep-learning-based digital staining techniques for augmenting two types of label-free imaging modalities. I showcased two important applications in the field of single-cell immunofluorescence microscopy and mesoscale 3D histological human brain imaging. I expect two major potential impacts from my thesis work. First, the integration of digital staining techniques with multi-contrast microscopy can potentially enhance the throughput of single-cell imaging cytometry and phenotyping. Second, the integration of digital staining techniques with S-OCT can potentially enable high-throughput human brain imaging, facilitating comprehensive studies on the brain's structure and function. Through this exploration, this thesis advances the digital staining technique and its applications for various biomedical disciplines.