{"id":954,"date":"2018-10-21T12:19:41","date_gmt":"2018-10-21T17:19:41","guid":{"rendered":"https:\/\/sites.bu.edu\/tianlab\/?page_id=954"},"modified":"2025-11-16T15:29:49","modified_gmt":"2025-11-16T20:29:49","slug":"physics-embedded-deep-learning","status":"publish","type":"page","link":"https:\/\/sites.bu.edu\/tianlab\/publications\/physics-embedded-deep-learning\/","title":{"rendered":"Deep learning for physics-based imaging"},"content":{"rendered":"

Whitened Score Diffusion: A Structured Prior for Imaging Inverse Problems
\n<\/strong><\/a>J Alido, T Li, Y Sun, L Tian
\nNeurIPS<\/strong><\/em> 2025.
\n\u2b51\u00a0<\/span>Github Project<\/a><\/strong><\/p>\n

\"\"<\/p>\n

Wide-field, high-resolution reconstruction in computational multi-aperture miniscope using a Fourier neural network<\/strong><\/a>
\nQianwan Yang, Ruipeng Guo, Guorong Hu, Yujia Xue, Yunzhe Li, and Lei Tian
\nOptica<\/strong><\/em> Vol. 11, Issue 6, pp. 860-871 (2024).
\n\u2b51<\/span>\u00a0<\/em><\/span><\/span>Github Project<\/a><\/strong><\/p>\n

\"\"<\/p>\n

Multiple-scattering simulator-trained neural network for intensity diffraction tomography<\/strong><\/a>
\nA. Matlock, J. Zhu, L. Tian
\nOptics Express<\/em><\/strong> 31, 4094-4107 (2023)<\/p>\n

Recovering 3D phase features of complex biological samples traditionally sacrifices computational efficiency and processing time for physical model accuracy and reconstruction quality. Here, we overcome this challenge using an approximant-guided deep learning framework in a high-speed intensity diffraction tomography system. Applying a physics model simulator-based learning strategy trained entirely on natural image datasets, we show our network can robustly reconstruct complex 3D biological samples. To achieve highly efficient training and prediction, we implement a lightweight 2D network structure that utilizes a multi-channel input for encoding the axial information. We demonstrate this framework on experimental measurements of weakly scattering epithelial buccal cells and strongly scattering\u00a0<\/span>C. elegans<\/i>\u00a0<\/span>worms. We benchmark the network\u2019s performance against a state-of-the-art multiple-scattering model-based iterative reconstruction algorithm. We highlight the network\u2019s robustness by reconstructing dynamic samples from a living worm video. We further emphasize the network\u2019s generalization capabilities by recovering algae samples imaged from different experimental setups. To assess the prediction quality, we develop a quantitative evaluation metric to show that our predictions are consistent with both multiple-scattering physics and experimental measurements.<\/p>\n

\"\"<\/p>\n

Recovery of Continuous 3D Refractive Index Maps from Discrete Intensity-Only Measurements using Neural Fields
\n<\/strong><\/a>Renhao Liu, Yu Sun, Jiabei Zhu, Lei Tian, Ulugbek Kamilov
\nNature Machine Intelligence<\/strong><\/em> 4, <\/span>781\u2013791 <\/span>(2022).<\/p>\n

Intensity diffraction tomography (IDT) refers to a class of optical microscopy techniques for imaging the three-dimensional refractive index (RI) distribution of a sample from a set of two-dimensional intensity-only measurements. The reconstruction of artefact-free RI maps is a fundamental challenge in IDT due to the loss of phase information and the missing-cone problem. Neural fields has recently emerged as a new deep learning approach for learning continuous representations of physical fields. The technique uses a coordinate-based neural network to represent the field by mapping the spatial coordinates to the corresponding physical quantities, in our case the complex-valued refractive index values. We present Deep Continuous Artefact-free RI Field (DeCAF) as a neural-fields-based IDT method that can learn a high-quality continuous representation of a RI volume from its intensity-only and limited-angle measurements. The representation in DeCAF is learned directly from the measurements of the test sample by using the IDT forward model without any ground-truth RI maps. We qualitatively and quantitatively evaluate DeCAF on the simulated and experimental biological samples. Our results show that DeCAF can generate high-contrast and artefact-free RI maps and lead to an up to 2.1-fold reduction in the mean squared error over existing methods.<\/p>\n

\n
\n
\n

\"\"<\/p>\n<\/div>\n<\/div>\n<\/section>\n

Deep learning-augmented Computational Miniature Mesoscope<\/strong><\/a>
\nYujia Xue, Qianwan Yang, Guorong Hu, Kehan Guo, Lei Tian
\nOptica<\/strong><\/em>\u00a0<\/span>9, 1009-1021 (2022)<\/span><\/p>\n

\u2b51<\/span>\u00a0<\/em><\/span><\/span>Github Project<\/a><\/strong><\/p>\n

Fluorescence microscopy is essential to study biological structures and dynamics. However, existing systems suffer from a tradeoff between field-of-view (FOV), resolution, and complexity, and thus cannot fulfill the emerging need of miniaturized platforms providing micron-scale resolution across centimeter-scale FOVs. To overcome this challenge, we developed Computational Miniature Mesoscope (CM<\/span><\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>) that exploits a computational imaging strategy to enable single-shot 3D high-resolution imaging across a wide FOV in a miniaturized platform. Here, we present CM<\/span><\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>\u00a0V2 that significantly advances both the hardware and computation. We complement the 3<\/span>\u00d7<\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>3 microlens array with a new hybrid emission filter that improves the imaging contrast by 5<\/span>\u00d7<\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>, and design a 3D-printed freeform collimator for the LED illuminator that improves the excitation efficiency by 3<\/span>\u00d7<\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>. To enable high-resolution reconstruction across the large imaging volume, we develop an accurate and efficient 3D linear shift-variant (LSV) model that characterizes the spatially varying aberrations. We then train a multi-module deep learning model, CM<\/span><\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>Net, using only the 3D-LSV simulator. We show that CM<\/span><\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>Net generalizes well to experiments and achieves accurate 3D reconstruction across a\u00a0<\/span>\u223c<\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>7-mm FOV and 800-<\/span>\u03bc<\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>m depth, and provides\u00a0<\/span>\u223c<\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>6-<\/span>\u03bc<\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>m lateral and\u00a0<\/span>\u223c<\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>25-<\/span>\u03bc<\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>m axial resolution. This provides\u00a0<\/span>\u223c<\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>8<\/span>\u00d7<\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>\u00a0better axial localization and\u00a0<\/span>\u223c<\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>1400<\/span>\u00d7<\/span><\/span><\/span><\/span><\/span><\/nobr><\/span>\u00a0faster speed as compared to the previous model-based algorithm. We anticipate this simple and low-cost computational miniature imaging system will be impactful to many large-scale 3D fluorescence imaging applications.<\/span><\/p>\n

\"\"<\/p>\n

Adaptive 3D descattering with a dynamic synthesis network<\/strong><\/a>
\nWaleed Tahir, Hao Wang, Lei Tian
\nLight: Science & Applications<\/strong><\/em>\u00a011, 42, 2022<\/p>\n

\u2b51<\/span>\u00a0<\/em><\/span><\/span>Github Project<\/a><\/strong><\/p>\n

Deep learning has been broadly applied to imaging in scattering applications. A common framework is to train a “descattering” neural network for image recovery by removing scattering artifacts. To achieve the best results on a broad spectrum of scattering conditions, individual “expert” networks have to be trained for each condition. However, the performance of the expert sharply degrades when the scattering level at the testing time differs from the training. An alternative approach is to train a “generalist” network using data from a variety of scattering conditions. However, the generalist generally suffers from worse performance as compared to the expert trained for each scattering condition. Here, we develop a drastically different approach, termed dynamic synthesis network (DSN), that can dynamically adjust the model weights and adapt to different scattering conditions. The adaptability is achieved by a novel architecture that enables dynamically synthesizing a network by blending multiple experts using a gating network. Notably, our DSN adaptively removes scattering artifacts across a continuum of scattering conditions regardless of whether the condition has been used for the training, and consistently outperforms the generalist. By training the DSN entirely on a multiple-scattering simulator, we experimentally demonstrate the network’s adaptability and robustness for 3D descattering in holographic 3D particle imaging. We expect the same concept can be adapted to many other imaging applications, such as denoising, and imaging through scattering media. Broadly, our dynamic synthesis framework opens up a new paradigm for designing highly adaptive deep learning and computational imaging techniques.<\/p>\n

\"\"<\/p>\n

 <\/p>\n

Displacement-agnostic coherent imaging through scatter with an interpretable deep neural network<\/strong><\/a>
\nY. Li, S. Cheng, Y. Xue, L. Tian
\nOptics Express<\/strong><\/em> Vol. 29, Issue 2, pp. 2244-2257 (2021).<\/p>\n

Coherent imaging through scatter is a challenging task. Both model-based and data-driven approaches have been explored to solve the inverse scattering problem. In our previous work, we have shown that a deep learning approach can make high-quality and highly generalizable predictions through unseen diffusers. Here, we propose a new deep neural network model that is agnostic to a broader class of perturbations including scatterer change, displacements, and system defocus up to 10\u00d7 depth of field. In addition, we develop a new analysis framework for interpreting the mechanism of our deep learning model and visualizing its generalizability based on an unsupervised dimension reduction technique. We show that our model can unmix the scattering-specific information and extract the object-specific information and achieve generalization under different scattering conditions. Our work paves the way to a\u00a0<\/span>robust<\/i>\u00a0<\/span>and\u00a0<\/span>interpretable<\/i>\u00a0<\/span>deep learning approach to imaging through scattering media.<\/p>\n

\"\"<\/p>\n

SIMBA: Scalable Inversion in Optical Tomography using Deep Denoising Priors<\/strong><\/a>
\nZihui Wu, Yu Sun, Alex Matlock, Jiaming Liu, Lei Tian, Ulugbek S. Kamilov
\nIEEE Journal of Selected Topics in Signal Processing<\/strong><\/em> 14(6), 2020.<\/p>\n

\n
\n
\n
Two features desired in a three-dimensional (3D) optical tomographic image reconstruction algorithm are the ability to reduce imaging artifacts and to do fast processing of large data volumes. Traditional iterative inversion algorithms are impractical in this context due to their heavy computational and memory requirements. We propose and experimentally validate a novel scalable iterative minibatch algorithm (SIMBA) for fast and high-quality optical tomographic imaging. SIMBA enables high-quality imaging by combining two complementary information sources: the physics of the imaging system characterized by its forward model and the imaging prior characterized by a denoising deep neural net. SIMBA easily scales to very large 3D tomographic datasets by processing only a small subset of measurements at each iteration. We establish the theoretical fixed-point convergence of SIMBA\u00a0under nonexpansive denoisers for convex data-fidelity terms. We validate SIMBA\u00a0on both simulated and experimentally collected intensity diffraction tomography (IDT) datasets. Our results show that SIMBA can significantly reduce the computational burden of 3D image formation without sacrificing the imaging quality.<\/div>\n<\/div>\n

\"\"<\/p>\n

Reliable deep learning-based phase imaging with uncertainty quantification<\/strong><\/a>
\nYujia Xue, Shiyi Cheng, Yunzhe Li, Lei Tian
\nOptica<\/em><\/strong>\u00a0<\/span>6, 618-629 (2019)<\/span>.<\/p>\n

\u2b51<\/span>\u00a0<\/em><\/span><\/span>Github Project<\/a><\/strong><\/p>\n

Emerging deep-learning (DL)-based techniques have significant potential to revolutionize biomedical imaging. However, one outstanding challenge is the lack of reliability assessment in the DL predictions, whose errors are commonly revealed only in hindsight. Here, we propose a new Bayesian convolutional neural network (BNN)-based framework that overcomes this issue by quantifying the uncertainty of DL predictions. Foremost, we show that BNN-predicted uncertainty maps provide surrogate estimates of the true error from the network model and measurement itself. The uncertainty maps characterize imperfections often unknown in real-world applications, such as noise, model error, incomplete training data, and out-of-distribution testing data. Quantifying this uncertainty provides a per-pixel estimate of the confidence level of the DL prediction as well as the quality of the model and data set. We demonstrate this framework in the application of large space\u2013bandwidth product phase imaging using a physics-guided coded illumination scheme. From only five multiplexed illumination measurements, our BNN predicts gigapixel phase images in both static and dynamic biological samples with quantitative credibility assessment. Furthermore, we show that low-certainty regions can identify spatially and temporally rare biological phenomena. We believe our uncertainty learning framework is widely applicable to many DL-based biomedical imaging techniques for assessing the reliability of DL predictions.<\/span><\/p>\n

\n
\n
\n

\"\"<\/p>\n

 <\/p>\n

Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media
\n<\/strong><\/a>Yunzhe Li, Yujia Xue,\u00a0Lei Tian
\nOptica<\/em><\/strong> 5, 1181-1190 (2018).
\n\u2b51<\/strong> Top 5 most cited articles in Optica published in 2018 (Source: Google Scholar)<\/em><\/strong><\/span><\/p>\n

\u2b51<\/span>\u00a0<\/em><\/span><\/span>Github Project<\/a><\/strong><\/p>\n

Imaging through scattering is an important yet challenging problem. Tremendous progress has been made by exploiting the deterministic input\u2013output \u201ctransmission matrix\u201d for a fixed medium. However, this \u201cone-to-one\u201d mapping is highly susceptible to speckle decorrelations – small perturbations to the scattering medium lead to model errors and severe degradation of the imaging performance. Our goal here is to develop a new framework that is highly scalable to both medium perturbations and measurement requirement. To do so, we propose a statistical \u201cone-to-all\u201d deep learning (DL) technique that encapsulates a wide range of statistical variations for the model to be resilient to speckle decorrelations. Specifically, we develop a convolutional neural network (CNN) that is able to learn the statistical information contained in the speckle intensity patterns captured on a set of diffusers having the same macroscopic parameter. We then show for the first time, to the best of our knowledge, that the trained CNN is able to generalize and make high-quality object predictions through an entirely different set of diffusers of the same class. Our work paves the way to a highly scalable DL approach for imaging through scattering media.<\/p>\n

\"intro\"<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Whitened Score Diffusion: A Structured Prior for Imaging Inverse Problems J Alido, T Li, Y Sun, L Tian NeurIPS 2025. \u2b51\u00a0Github Project Wide-field, high-resolution reconstruction in computational multi-aperture miniscope using a Fourier neural network Qianwan Yang, Ruipeng Guo, Guorong Hu, Yujia Xue, Yunzhe Li, and Lei Tian Optica Vol. 11, Issue 6, pp. 860-871 (2024). […]<\/p>\n","protected":false},"author":12228,"featured_media":854,"parent":133,"menu_order":4,"comment_status":"closed","ping_status":"closed","template":"page-templates\/no-sidebars.php","meta":[],"_links":{"self":[{"href":"https:\/\/sites.bu.edu\/tianlab\/wp-json\/wp\/v2\/pages\/954"}],"collection":[{"href":"https:\/\/sites.bu.edu\/tianlab\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/sites.bu.edu\/tianlab\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/sites.bu.edu\/tianlab\/wp-json\/wp\/v2\/users\/12228"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.bu.edu\/tianlab\/wp-json\/wp\/v2\/comments?post=954"}],"version-history":[{"count":29,"href":"https:\/\/sites.bu.edu\/tianlab\/wp-json\/wp\/v2\/pages\/954\/revisions"}],"predecessor-version":[{"id":2504,"href":"https:\/\/sites.bu.edu\/tianlab\/wp-json\/wp\/v2\/pages\/954\/revisions\/2504"}],"up":[{"embeddable":true,"href":"https:\/\/sites.bu.edu\/tianlab\/wp-json\/wp\/v2\/pages\/133"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sites.bu.edu\/tianlab\/wp-json\/wp\/v2\/media\/854"}],"wp:attachment":[{"href":"https:\/\/sites.bu.edu\/tianlab\/wp-json\/wp\/v2\/media?parent=954"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}