Background

Deep learning holds great potential for biological microscopy data, and offers exciting opportunities for fluorescent feature reconstruction. Fluorescence reconstruction microscopy (FRM) takes in a transmitted light image of a biological sample and outputs a series of reconstructed fluorescence images that predict what the sample would look like had it been labeled with a given series of dyes or fluorescently tagged proteins. FRM works by first training a convolutional neural network to relate a large set of transmitted light data to corresponding real fluorescence images (the ground truth) for given markers. The network learns by comparing its fluorescence predictions to the ground truth fluorescence data and iterating until it reaches a cut off. Once trained, FRM can be performed on transmitted light data without requiring additional fluorescence imaging. Advantages to this include the possibility of reducing phototoxicity, freeing up fluorescence channels for more complex markers, and re-processing transmitted light data to extract new information. However, current FRM benchmarks are abstractions that are difficult to relate to how valuable or trustworthy an FRM prediction is. In their work, LeChance and Cohen aim to provide a standardized implementation of FRM, and demonstrate its practical performance and limitations for various conventionally performed cell biology analyses (1).

Key findings and developments

Development

U-Nets, and other deep learning approaches, have found broad applications to live-cell imaging tasks such as cell phenotype classification, feature segmentation, and histological stain analysis. The workflow presented in this work consists on collecting multi-channel training images of cultured cells whereby each image comprises a transmitted light channel and associated fluorescence channels. The images were broken into sub-images in ImageJ and input into the U-network for pattern recognition. The transmitted light images serve as input to the network and this process can be extended to full time-lapse microscopy fluorescence reconstruction. Pearson’s Correlation Coefficient (PCC) was selected as the conventional performance metric; however due to skewed results in images primarily with background, the authors report a corrected accuracy score (P) representing the PCC of a large subset of images in a given dataset containing positive examples of the feature of interest (eg nuclei) based on an intensity threshold. This approach aims to improve network performance for datasets containing large amounts of background signal.

 

Proof of concept

The authors explored FRM in the context of high-content imaging applications. For this, they captured transmitted light images using 4x, 10x, and 20x air objectives using either Phase Contrast or Differential Interference Contrast (DIC), and collected data across 3 different cell types. Moreover, the authors demonstrated the utility of low-magnification reconstruction and nuclear tracking in a 24h time-lapse of the growth dynamics of large epithelia, with images acquired every 10 minutes. FRM proved highly effective as an alternative to nuclear labeling approach for large-scale, long-term imaging, avoiding some shortcomings associated with the use of nuclear dyes.

The authors then went on to explore FRM in the context of reconstructing epithelial cell-cell junctions for segmentation and morphological studies. In the absence of specific markers, cell-cell junctions are relatively difficult to segment, especially from DIC images. In their work the authors show that the U-Net was able to reconstruct E-cadherin junctions with high visual accuracy, and determined that the FRM network is able to capture subtle 3D information from 2D input images.

The work then explores the possibility of finer reconstruction, by using a 20X/0.8NA objective, and HUVEC cells with multiple structures labeled. FRM had different levels of success for different structures, and the authors conclude that the value of FRM depends on the specific question and context. The conclude also that ultimately the decision on whether the detection of finer structures is good enough rests in the end user.

To facilitate information for such decision, the authors then provided several examples of how the size of the training affects the score P, and the accuracy of the resulting FRM predictions. In general, they show that FRM quality varies directly with the size of the training set, however, the rate of change in P vs. training set size is neither linear nor uniform across different biomarkers. They explored also the possibility of training a deeper network, but observed no significant improvement, leading to the conclusion that their minimal U-Net implementation performs well as a foundation for various types of analyses without the need for significant fine-tuning.

 

What I like about this preprint

I enjoyed this work a lot, and I think the tool the authors designed has a lot of potential for multiple fields.

Open questions

1.While in your work you aimed at proving the usefulness of the tool for high-throughput, how useful is its performance upon using a 63x or 100x objective? As you mention along your work, one of the advantages would be to overcome issues of photobleaching, and the possibility of freeing channels for visualizing other structures.

2.You mention the need of correction for PCC – (P), and that the skew you observe occurs in images with a lot of background. Can you expand further on this point?

3.Are there important artifacts you noticed, or further limitations to the use of your tool, beyond those you discuss?

4.You used various cell lines to test FRM. Following from the question above, are specific cell lines more complicated for FRM processing than others and if so, is it possible to generate training sets for specific cell lines a priori (ie already incorporated in the tool, and selectable by the user)?

5.Is your tool useful for more complex tissues such as histology sections?

References

1. LeChance J and Cohen DJ, Practical fluorescence reconstruction microscopy for large samples and low-magnification imaging, bioRxiv, 2020.

 

doi: https://doi.org/10.1242/prelights.22546

Read preprint (No Ratings Yet)