Background

Capturing highly dynamic physiological processes happening on milli-second time scales across large areas in living organisms requires imaging methods capable of such resolution. An attractive candidate for high-speed 3D image acquisition is light-field microscopy (LFM), which has already opened new avenues in the fields of neurobiology and cardiovascular dynamics. While some technical hindrances of this tool have been overcome since its conception, the widespread use of LFM has been hampered by a computationally demanding, iterative image reconstruction process that requires a complex computational infrastructure and adequate data management. Multiple algorithms derived from deep learning and convolutional neural networks have recently been proposed to replace iterative deconvolution procedures, and offer new methods for deblurring, denoising and super-resolution. While many of these methods have excellent performance in various biologically relevant settings, many are not optimal for dynamic imaging with LFM given the complexity of dynamic processes in small animals. In their work, Wagner and Beuttenmueller et al (1) present a novel framework consisting of a hybrid light-field light-sheet microscope (HyLFM) and deep-learning-based volume reconstruction. In it, single light-sheet acquisitions continuously serve as training data and validation for the convolutional neural network (termed HyLFM-Net) reconstructing the LFM volume.

Key findings and developments

A simultaneous selective-plane illumination microscopy (SPIM) modality was added into a standard LFM microscope, allowing the generation of high-resolution ground truth images of single planes for validation, training, and refinement for the convolutional neural network. Training can be done on static sample volumes, or dynamically from a single SPIM plane going through the volume during 3D image acquisition. An automated image processing pipeline ensures that LFM and SPIM volumes are co-registered in a common reference volume and coordinate system with high precision. This is important for convolutional neural network training and validation, and the systems’ ability to acquire 2D and 3D training data is key for reliable convolutional neural network reconstructions, including data never seen in training. Altogether, the HyLFM-Net is trained on pairs of SPIM-LFM images.

To evaluate the performance of the HyLFM system, the authors imaged sub-diffraction sized fluorescent beads suspended in agarose, and quantified the improvement in spatial resolution and image quality compared to standard iterative light-field deconvolution. They concluded that HyLFM-Net correctly inferred the 3D imaging volume from the raw light-field data, with better resolution than that which could be obtained by light field deconvolution, and without artifacts commonly found in light field deconvolution. The authors point out the importance of training on diverse datasets, to avoid biases in performance.

As proof of principle, the authors explore the capabilities of the HyLFM system by imaging the dynamics of a hatchling medaka fish heart in vivo, to demonstrate the capability of the system to correctly capture dynamic cellular movements in 3D. The HyLFM-Net allowed acquiring high image quality metrics compared to SPIM, and allowed 3D volume inference at up to 18Hz, with at least 1000-fold reconstruction speed compared to light field deconvolution. The authors note that the network trained on dynamically acquired SPIM single planes performed equally well or better than the network trained on fully static volumes.

The authors also tested the HyLFM system on transgenic larval zebrafish brains expressing calcium indicators, to monitor neural activity. The ground truth data enabled the HyLFM system to faithfully learn and infer structural, as well as intensity-based information. The authors conclude HyLFM is thus an attractive method for visualizing neural activity.

Altogether, the new system allows reconstructing light-field volumes at sub-second rates, eliminating the main computational hindrances for light field imaging. Moreover, the system enables acquiring appropriate training data simultaneously and on-the-fly, while allowing continuous validation and fine-tuning. The network over time learns on the actual experimental data, rather than requiring pre-acquisition of training images in separate microscopes, solving the hindrance of transferability.

What I like about this preprint

I like that the authors address a hugely important technical gap in a fast-advancing microscopy area. It has not been uncommon over the past few decades that major advances in microscopy tools occur, and the methods for image analysis stay behind. This at some point becomes a limiting factor in itself. I think that tools addressing this gap to overcome those limitations are key, and ultimately allow the different microscopy tools to be used to their full potential, and become widespread.

 

References

1. Wagner N, Beuttenmueller F, et al, Deep learning-enganced light-field imaging with continuous validation, bioRxiv, 2020.

 

doi: https://doi.org/10.1242/prelights.25619

Read preprint (No Ratings Yet)