Computational Microscopy

Structured illumination microscopy with unknown patterns and a statistical prior
Li-Hao Yeh, Lei Tian, and Laura Waller
Biomed. Opt. Express 8, 695-711 (2017).

Structured illumination microscopy (SIM) improves resolution by down-modulating high-frequency information of an object to fit within the passband of the optical system. Generally, the reconstruction process requires prior knowledge of the illumination patterns, which implies a well-calibrated and aberration-free system. Here, we propose a new algorithmic self-calibration strategy for SIM that does not need to know the exact patterns a priori, but only their covariance. The algorithm, termed PE-SIMS, includes a pattern-estimation (PE) step requiring the uniformity of the sum of the illumination patterns and a SIM reconstruction procedure using a statistical prior (SIMS). Additionally, we perform a pixel reassignment process (SIMS-PR) to enhance the reconstruction quality. We achieve 2× better resolution than a conventional widefield microscope, while remaining insensitive to aberration-induced pattern distortion and robust against parameter tuning.

sim

 


3D differential phase contrast microscopy

Michael Chen, Lei Tian, Laura Waller
Biomed. Opt. Express 7, 3940-3950 (2016).

We demonstrate 3D phase and absorption recovery from partially coherent intensity images captured with a programmable LED array source. Images are captured through-focus with four different illumination patterns. Using first Born and weak object approximations (WOA), a linear 3D differential phase contrast (DPC) model is derived. The partially coherent transfer functions relate the sample’s complex refractive index distribution to intensity measurements at varying defocus. Volumetric reconstruction is achieved by a global FFT-based method, without an intermediate 2D phase retrieval step. Because the illumination is spatially partially coherent, the transverse resolution of the reconstructed field achieves twice the NA of coherent systems and improved axial resolution.

3D qDPC

Computational illumination for high-speed in vitro Fourier ptychographic microscopy
L. Tian, Z. Liu, L. Yeh, M. Chen, J. Zhong, L. Waller
Optica 2(10), 904-911 (2015).

We demonstrate a new computational illumination technique that achieves a large space-bandwidth-time product, for quantitative phase imaging of unstained live samples in vitro. Microscope lenses can have either a large field of view (FOV) or high resolution, and not both. Fourier ptychographic microscopy (FPM) is a new computational imaging technique that circumvents this limit by fusing information from multiple images taken with different illumination angles. The result is a gigapixel-scale image having both a wide FOV and high resolution, i.e., a large space-bandwidth product. FPM has enormous potential for revolutionizing microscopy and has already found application in digital pathology. However, it suffers from long acquisition times (of the order of minutes), limiting throughput. Faster capture times would not only improve the imaging speed, but also allow studies of live samples, where motion artifacts degrade results. In contrast to fixed (e.g., pathology) slides, live samples are continuously evolving at various spatial and temporal scales. Here, we present a new source coding scheme, along with real-time hardware control, to achieve 0.8 NA resolution across a 4x FOV with subsecond capture times. We propose an improved algorithm and a new initialization scheme, which allow robust phase reconstruction over long time-lapse experiments. We present the first FPM results for both growing and confluent in vitro cell cultures, capturing videos of subcellular dynamical phenomena in popular cell lines undergoing division and migration. Our method opens up FPM to applications with live samples, for observing rare events in both space and time.

InvitroFPM

3D intensity and phase imaging from light field measurements in an LED array microscope
Lei Tian, L. Waller
Optica 2, 104-111 (2015).

Realizing high resolution across large volumes is challenging for 3D imaging techniques with high-speed acquisition. Here, we describe a new method for 3D intensity and phase recovery from 4D light field measurements, achieving enhanced resolution via Fourier Ptychography. Starting from geometric optics light field refocusing, we incorporate phase retrieval and correct diffraction artifacts. Further, we incorporate dark-field images to achieve lateral resolution beyond the diffraction limit of the objective (5x larger NA) and axial resolution better than the depth of field, using a low magnification objective with a large field of view. Our iterative reconstruction algorithm uses a multi-slice coherent model to estimate the 3D complex transmittance function of the sample at multiple depths, without any weak or single-scattering approximations. Data is captured by an LED array microscope with computational illumination, which enables rapid scanning of angles for fast acquisition. We demonstrate the method with thick biological samples in a modified commercial microscope, indicating the technique’s versatility for a wide range of applications.

3DFourierPtychography

Multiplexed coded illumination for Fourier Ptychography with an LED array microscope
Lei Tian, X. Li, K. Ramchandran, L. Waller
Biomedical Optics Express 5, 2376-2389 (2014).

Fourier Ptychography is a new computational microscopy technique that achieves gigapixel images with both wide field of view and high resolution in both phase and amplitude. The hardware setup involves a simple replacement of the microscope’s illumination unit with a programmable LED array, allowing one to flexibly pattern illumination angles without any moving parts. In previous work, a series of low-resolution images was taken by sequentially turning on each single LED in the array, and the data were then combined to recover a bandwidth much higher than the one allowed by the original imaging system. Here, we demonstrate a multiplexed illumination strategy in which multiple randomly selected LEDs are turned on for each image. Since each LED corresponds to a different area of Fourier space, the total number of images can be significantly reduced, without sacrificing image quality. We demonstrate this method experimentally in a modified commercial microscope. Compared to sequential scanning, our multiplexed strategy achieves similar results with approximately an order of magnitude reduction in both acquisition time and data capture requirements.

MultiplexFP

Quantitative differential phase contrast imaging in an LED array microscope
L. Tian, L. Waller
Opt. Express 23, 11394-11403 (2015).

Illumination-based differential phase contrast (DPC) is a phase imaging method that uses a pair of images with asymmetric illumination patterns. Distinct from coherent techniques, DPC relies on spatially partially coherent light, providing 2× better lateral resolution, better optical sectioning and immunity to speckle noise. In this paper, we derive the 2D weak object transfer function (WOTF) and develop a quantitative phase reconstruction method that is robust to noise. The effect of spatial coherence is studied experimentally, and multiple-angle DPC is shown to provide improved frequency coverage for more stable phase recovery. Our method uses an LED array microscope to achieve real-time (10 Hz) quantitative phase imaging with in vitro live cell samples.

DPC

Motion deblurring with temporally coded illumination in an LED array microscope
C. Ma, Z. Liu, L. Tian, Q. Dai, L. Waller
Opt. Lett. 40, 2281-2284 (2015).

Motion blur, which results from time-averaging an image over the camera’s exposure time, is a common problem in microscopy of moving samples. Here, we demonstrate linear motion deblurring using temporally coded illumination in an LED array microscope. By illuminating moving objects with a well-designed temporal coded sequence that varies during each single camera exposure, the resulting motion blur is invertible and can be computationally removed. This scheme is implemented in an existing LED array microscope, providing benefits of being grayscale, fast, and adaptive, which leads to high-quality deblur performance and a flexible implementation with no moving parts. The proposed method is demonstrated experimentally for fast moving targets in a microfluidic environment.

motiondeblur LED

 

Real-time brightfield, darkfield and phase contrast imaging in an LED array microscope
Z. Liu, Lei Tian, S. Liu, L. Waller
Journal of Biomedical Optics, 19(10), 106002 (2014).

We demonstrate a single-camera imaging system that can simultaneously acquire brightfield, darkfield and phase contrast images in real-time. Our method uses computational illumination via a programmable LED array at the source plane, providing flexible patterning of illumination angles. Brightfield, darkfield and differential phase contrast (DPC) images are obtained by changing the LED patterns, without any moving parts. Previous work with LED array illumination was only valid for static samples because the hardware speed was not fast enough to meet real-time acquisition and processing requirements. Here, we time multiplex patterns for each of the three contrast modes in order to image dynamic biological processes in all three contrast modes simultaneously. We demonstrate multi-contrast operation at the maximum frame rate of our camera (50 Hz with 2160×2560 pixels).

MultiContrast

3D differential phase contrast microscopy with computational illumination using an LED array
Lei Tian, J. Wang, L. Waller
Optics Letters 39, 1326 – 1329 (2014).

We demonstrate 3D differential phase-contrast (DPC) microscopy, based on computational illumination with a programmable LED array. By capturing intensity images with various illumination angles generated by sequentially patterning an LED array source, we digitally refocus images through various depths via light field processing. The intensity differences from images taken at complementary illumination angles are then used to generate DPC images, which are related to the gradient of phase. The proposed method achieves 3D DPC with simple, inexpensive optics and no moving parts. We experimentally demonstrate our method by imaging a camel hair sample in 3D.

3DDPC


Experimental robustness of Fourier Ptychography phase retrieval algorithms
L. Yeh, J. Dong, J. Zhong, L. Tian, M. Chen, G. Tang, M. Soltanolkotabi, L. Waller
Opt. Express 23(26) 33212-33238 (2015).

Fourier ptychography is a new computational microscopy technique that provides gigapixel-scale intensity and phase images with both wide field-of-view and high resolution. By capturing a stack of low-resolution images under different illumination angles, an inverse algorithm can be used to computationally reconstruct the high-resolution complex field. Here, we compare and classify multiple proposed inverse algorithms in terms of experimental robustness. We find that the main sources of error are noise, aberrations and mis-calibration (i.e. model mis-match). Using simulations and experiments, we demonstrate that the choice of cost function plays a critical role, with amplitude-based cost functions performing better than intensity-based ones. The reason for this is that Fourier ptychography datasets consist of images from both brightfield and darkfield illumination, representing a large range of measured intensities. Both noise (e.g. Poisson noise) and model mis-match errors are shown to scale with intensity. Hence, algorithms that use an appropriate cost function will be more tolerant to both noise and model mis-match. Given these insights, we propose a global Newton’s method algorithm which is robust and accurate. Finally, we discuss the impact of procedures for algorithmic correction of aberrations and mis-calibration.

FPM_algorithm

Self-learning based Fourier ptychographic microscopy
Y. Zhang, W. Jiang, L. Tian, L. Waller, Q. Dai
Opt. Express 23, 18471-18486 (2015).

Fourier Ptychographic Microscopy (FPM) is a newly proposed computational imaging method aimed at reconstructing a high-resolution wide-field image from a sequence of low-resolution images. These low-resolution images are captured under varied illumination angles and the FPM recovery routine then stitches them together in the Fourier domain iteratively. Although FPM has achieved success with static sample reconstructions, the long acquisition time inhibits real-time application. To address this problem, we propose here a self-learning based FPM which accelerates the acquisition and reconstruction procedure. We first capture a single image under normally incident illumination, and then use it to simulate the corresponding low-resolution images under other illumination angles. The simulation is based on the relationship between the illumination angles and the shift of the sample’s spectrum. We analyze the importance of the simulated low-resolution images in order to devise a selection scheme which only collects the ones with higher importance. The measurements are then captured with the selection scheme and employed to perform the FPM reconstruction. Since only measurements of high importance are captured, the time requirements of data collection as well as image reconstruction can be greatly reduced. We validate the effectiveness of the proposed method with simulation and experimental results showing that the reduction ratio of data size requirements can reach over 70%, without sacrificing image reconstruction quality.

Multi-contrast imaging and digital refocusing on a mobile microscope with a domed LED array
Z. Phillips, M. D’Ambrosio, L. Tian, J. Rulison, H. Patel, N. Sadras, A. Gande, N. Switz, D. Fletcher, L. Waller
PLoS ONE 10, e0124938 (2015).

We demonstrate the design and application of an add-on device for improving the diagnostic and research capabilities of CellScope—a low-cost, smartphone-based point-of-care microscope. We replace the single LED illumination of the original CellScope with a programmable domed LED array. By leveraging recent advances in computational illumination, this new device enables simultaneous multi-contrast imaging with brightfield, darkfield, and phase imaging modes. Further, we scan through illumination angles to capture lightfield datasets, which can be used to recover 3D intensity and phase images without any hardware changes. This digital refocusing procedure can be used for either 3D imaging or software-only focus correction, reducing the need for precise mechanical focusing during field experiments. All acquisition and processing is performed on the mobile phone and controlled through a smartphone application, making the computational microscope compact and portable. Using multiple samples and different objective magnifications, we demonstrate that the performance of our device is comparable to that of a commercial microscope. This unique device platform extends the field imaging capabilities of CellScope, opening up new clinical and research possibilities.

CellScope