New Computational Model Significantly Improves Quality of Microscopy Images
Posted on 25 Dec 2024
The computational processing of images allows for the detailed examination of samples using various light microscopes. While there has been significant progress in this field, there remains potential to enhance aspects such as image contrast and resolution. Now, a new computational model, based on an advanced deep learning architecture, offers faster processing times while achieving or surpassing the image quality of traditional methods.
The model, called Multi-Stage Residual-BCR Net (m-rBCR), was specifically developed for microscopy images by researchers at Helmholtz-Zentrum Dresden-Rossendorf (HZDR, Dresden, German) and the Max Delbrück Center for Molecular Medicine (Berlin, Germany). It introduces a novel approach to image processing using deconvolution, a method aimed at improving the contrast and resolution of digital images captured by optical microscopes, including widefield, confocal, or transmission types. Deconvolution addresses image blur, which is a type of distortion caused by the optical system, and it can be performed in two main ways: explicit deconvolution and deep learning-based deconvolution.
Explicit deconvolution techniques rely on the concept of the point spread function (PSF), which describes how light from a point source is scattered by the optical system, creating a three-dimensional diffraction pattern. This spreading causes out-of-focus light to contribute to blur in a recorded image. By knowing the PSF of a system, the blur can be mathematically removed, producing a clearer representation of the original image. However, PSF-based deconvolution is limited by the difficulty in obtaining accurate or precise PSF data for certain systems. Blind deconvolution methods, where the PSF is estimated from the image itself, have been developed but still present significant challenges and have made limited progress.
To address this issue, the research team has applied "inverse problem-solving" techniques, which have proven effective in microscopy. Inverse problems involve determining the underlying factors that lead to certain observed results. Solving such problems typically requires large amounts of data and advanced deep learning algorithms. Like explicit deconvolution, the goal is to achieve higher-resolution or better-quality images. For their approach, presented at the ECCV, the team used a physics-informed neural network called m-rBCR. In image processing, there are two basic approaches: working with the spatial representation of an image or its frequency representation, the latter of which requires transforming the spatial data. Each method has its advantages, and the majority of deep learning models operate on the spatial domain, which works well for general photographs. However, microscopy images, particularly those from fluorescence microscopy, are often monochromatic and typically feature specific light sources against a dark background.
To address the unique challenges of microscopy images, m-rBCR begins with the frequency representation. This approach enables more meaningful optical data representations and allows the model to solve the deconvolution task with far fewer parameters compared to other deep learning models. The team validated the m-rBCR model across four different datasets—two simulated and two real microscopy datasets. It demonstrated high performance with significantly fewer training parameters and faster processing times than current deep learning models, while also outperforming explicit deconvolution methods.
“This new architecture is leveraging a neglected way to learn representations beyond the classic convolutional neural network approaches,” said co-author Prof. Misha Kudryashev, Leader of the “In situ Structural Biology” group of the Max Delbrück Center for Molecular Medicine. “Our model significantly reduces potentially redundant parameters. As the results show, this is not accompanied by a loss of performance. The model is explicitly suitable for microscopy images and, having a lightweight architecture, it is challenging the trend of ever-bigger models that require ever more computing power.”
