Close

Practical Fluorescence Reconstruction Microscopy for Large Samples and Low-Magnification Imaging

Julienne LaChance, Daniel J. Cohen

Preprint posted on 12 May 2020 https://www.biorxiv.org/content/10.1101/2020.03.05.979419v3

Solving the puzzle using bright field.

Selected by Mariana De Niz

Categories: cell biology

Background

Deep learning holds great potential for biological microscopy data, and offers exciting opportunities for fluorescent feature reconstruction. Fluorescence reconstruction microscopy (FRM) takes in a transmitted light image of a biological sample and outputs a series of reconstructed fluorescence images that predict what the sample would look like had it been labeled with a given series of dyes or fluorescently tagged proteins. FRM works by first training a convolutional neural network to relate a large set of transmitted light data to corresponding real fluorescence images (the ground truth) for given markers. The network learns by comparing its fluorescence predictions to the ground truth fluorescence data and iterating until it reaches a cut off. Once trained, FRM can be performed on transmitted light data without requiring additional fluorescence imaging. Advantages to this include the possibility of reducing phototoxicity, freeing up fluorescence channels for more complex markers, and re-processing transmitted light data to extract new information. However, current FRM benchmarks are abstractions that are difficult to relate to how valuable or trustworthy an FRM prediction is. In their work, LeChance and Cohen aim to provide a standardized implementation of FRM, and demonstrate its practical performance and limitations for various conventionally performed cell biology analyses (1).

Figure 1. High-content, high- throughput labeling of fluorescent features (From reference 1).

Key findings and developments

Development

U-Nets, and other deep learning approaches, have found broad applications to live-cell imaging tasks such as cell phenotype classification, feature segmentation, and histological stain analysis. The workflow presented in this work consists on collecting multi-channel training images of cultured cells whereby each image comprises a transmitted light channel and associated fluorescence channels. The images were broken into sub-images in ImageJ and input into the U-network for pattern recognition. The transmitted light images serve as input to the network and this process can be extended to full time-lapse microscopy fluorescence reconstruction. Pearson’s Correlation Coefficient (PCC) was selected as the conventional performance metric; however due to skewed results in images primarily with background, the authors report a corrected accuracy score (P) representing the PCC of a large subset of images in a given dataset containing positive examples of the feature of interest (eg nuclei) based on an intensity threshold. This approach aims to improve network performance for datasets containing large amounts of background signal.

 

Proof of concept

The authors explored FRM in the context of high-content imaging applications. For this, they captured transmitted light images using 4x, 10x, and 20x air objectives using either Phase Contrast or Differential Interference Contrast (DIC), and collected data across 3 different cell types. Moreover, the authors demonstrated the utility of low-magnification reconstruction and nuclear tracking in a 24h time-lapse of the growth dynamics of large epithelia, with images acquired every 10 minutes. FRM proved highly effective as an alternative to nuclear labeling approach for large-scale, long-term imaging, avoiding some shortcomings associated with the use of nuclear dyes.

The authors then went on to explore FRM in the context of reconstructing epithelial cell-cell junctions for segmentation and morphological studies. In the absence of specific markers, cell-cell junctions are relatively difficult to segment, especially from DIC images. In their work the authors show that the U-Net was able to reconstruct E-cadherin junctions with high visual accuracy, and determined that the FRM network is able to capture subtle 3D information from 2D input images.

The work then explores the possibility of finer reconstruction, by using a 20X/0.8NA objective, and HUVEC cells with multiple structures labeled. FRM had different levels of success for different structures, and the authors conclude that the value of FRM depends on the specific question and context. The conclude also that ultimately the decision on whether the detection of finer structures is good enough rests in the end user.

To facilitate information for such decision, the authors then provided several examples of how the size of the training affects the score P, and the accuracy of the resulting FRM predictions. In general, they show that FRM quality varies directly with the size of the training set, however, the rate of change in P vs. training set size is neither linear nor uniform across different biomarkers. They explored also the possibility of training a deeper network, but observed no significant improvement, leading to the conclusion that their minimal U-Net implementation performs well as a foundation for various types of analyses without the need for significant fine-tuning.

 

What I like about this preprint

I enjoyed this work a lot, and I think the tool the authors designed has a lot of potential for multiple fields.

Open questions

1.While in your work you aimed at proving the usefulness of the tool for high-throughput, how useful is its performance upon using a 63x or 100x objective? As you mention along your work, one of the advantages would be to overcome issues of photobleaching, and the possibility of freeing channels for visualizing other structures.

2.You mention the need of correction for PCC – (P), and that the skew you observe occurs in images with a lot of background. Can you expand further on this point?

3.Are there important artifacts you noticed, or further limitations to the use of your tool, beyond those you discuss?

4.You used various cell lines to test FRM. Following from the question above, are specific cell lines more complicated for FRM processing than others and if so, is it possible to generate training sets for specific cell lines a priori (ie already incorporated in the tool, and selectable by the user)?

5.Is your tool useful for more complex tissues such as histology sections?

References

  1. LeChance J and Cohen DJ, Practical fluorescence reconstruction microscopy for large samples and low-magnification imaging, bioRxiv, 2020.

 

Posted on: 2 July 2020

doi: https://doi.org/10.1242/prelights.22546

Read preprint (No Ratings Yet)

1 comment

2 years

Julienne LaChance

Hi, this is Julie LaChance! Let me try to address your questions:

1.While in your work you aimed at proving the usefulness of the tool for high-throughput, how useful is its performance upon using a 63x or 100x objective? As you mention along your work, one of the advantages would be to overcome issues of photobleaching, and the possibility of freeing channels for visualizing other structures.

– Generally speaking, the higher the magnification, the better the prediction quality- you just have to make sure you have enough data! If you consider similar papers (one example: https://pubmed.ncbi.nlm.nih.gov/29656897/), you’ll see them tackling higher magnification images, where you can pick out fine details. In this paper we’re examining the low-mag story, to demonstrate how people can use these tools for large tissue analysis!

2.You mention the need of correction for PCC – (P), and that the skew you observe occurs in images with a lot of background. Can you expand further on this point?

– Yes! Since the PCC is a correlation coefficient, it’s a measure of how well the intensities in the ground truth image correlate with the intensities in the predicted image. So if you have two images with features in them (meaning: high intensities where there are features, and low intensities where it’s just background), and the prediction is working well, you’ll get a high PCC result. But, if the ground truth only contains background, and your predicted image is trying (and failing) to reproduce only background noise, you’re probably going to get a low PCC value. So for a fair comparison of feature reconstruction scores, we only consider images with features in them. Hope that makes sense!

3.Are there important artifacts you noticed, or further limitations to the use of your tool, beyond those you discuss?

– The usefulness of this tool is very much problem dependent! For example, if I want to get a sense for where nuclei are for tracking, this is likely a much easier problem than accurately reconstructing very fine features, like VE-cadherin fingers. Users should think carefully about downstream processing before applying the method, and whether they think there is rich enough information in the input images to produce the reconstructions they need. For example, very blurry imaging conditions probably won’t be useful for predicting fine structures.

I would love to see the field of FRM grow, to enable researchers to examine practical metrics for different applications of the method!

4.You used various cell lines to test FRM. Following from the question above, are specific cell lines more complicated for FRM processing than others and if so, is it possible to generate training sets for specific cell lines a priori (ie already incorporated in the tool, and selectable by the user)?

– Some cell lines are more complicated in the sense that they look “messier”: for example, our MDCK cells visually appear more uniform than our keratinocyte cells, which makes prediction slightly more challenging in the keratinocyte case. Nothing changes in the code, but you may need more data for messier cell lines, or may get slightly lower prediction scores, depending on the task.

And yes, you can absolutely generate trained model weights for different cell lines a priori. Our lab holds onto trained weights so that any time we need nuclei data (e.g. for density estimations or tracking), we can simply capture DIC or phase contrast timelapses and predict the nuclei later. Just make sure your imaging conditions are as close as possible to the conditions used in the training data!

5.Is your tool useful for more complex tissues such as histology sections?

– It could be! There exist a few other papers which do the same thing for histology: for example, https://www.nature.com/articles/s41377-019-0129-y .

Thanks for highlighting our work!

Have your say

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Sign up to customise the site to your preferences and to receive alerts

Register here

preLists in the cell biology category:

Fibroblasts

The advances in fibroblast biology preList explores the recent discoveries and preprints of the fibroblast world. Get ready to immerse yourself with this list created for fibroblasts aficionados and lovers, and beyond. Here, my goal is to include preprints of fibroblast biology, heterogeneity, fate, extracellular matrix, behavior, topography, single-cell atlases, spatial transcriptomics, and their matrix!

 



List by Osvaldo Contreras

EMBL Synthetic Morphogenesis: From Gene Circuits to Tissue Architecture (2021)

A list of preprints mentioned at the #EESmorphoG virtual meeting in 2021.

 



List by Alex Eve

FENS 2020

A collection of preprints presented during the virtual meeting of the Federation of European Neuroscience Societies (FENS) in 2020

 



List by Ana Dorrego-Rivas

Planar Cell Polarity – PCP

This preList contains preprints about the latest findings on Planar Cell Polarity (PCP) in various model organisms at the molecular, cellular and tissue levels.

 



List by Ana Dorrego-Rivas

BioMalPar XVI: Biology and Pathology of the Malaria Parasite

[under construction] Preprints presented at the (fully virtual) EMBL BioMalPar XVI, 17-18 May 2020 #emblmalaria

 



List by Dey Lab, Samantha Seah

1

Cell Polarity

Recent research from the field of cell polarity is summarized in this list of preprints. It comprises of studies focusing on various forms of cell polarity ranging from epithelial polarity, planar cell polarity to front-to-rear polarity.

 



List by Yamini Ravichandran

TAGC 2020

Preprints recently presented at the virtual Allied Genetics Conference, April 22-26, 2020. #TAGC20

 



List by Maiko Kitaoka et al.

3D Gastruloids

A curated list of preprints related to Gastruloids (in vitro models of early development obtained by 3D aggregation of embryonic cells). Updated until July 2021.

 



List by Paul Gerald L. Sanchez and Stefano Vianello

ECFG15 – Fungal biology

Preprints presented at 15th European Conference on Fungal Genetics 17-20 February 2020 Rome

 



List by Hiral Shah

ASCB EMBO Annual Meeting 2019

A collection of preprints presented at the 2019 ASCB EMBO Meeting in Washington, DC (December 7-11)

 



List by Madhuja Samaddar et al.

EMBL Seeing is Believing – Imaging the Molecular Processes of Life

Preprints discussed at the 2019 edition of Seeing is Believing, at EMBL Heidelberg from the 9th-12th October 2019

 



List by Dey Lab

Autophagy

Preprints on autophagy and lysosomal degradation and its role in neurodegeneration and disease. Includes molecular mechanisms, upstream signalling and regulation as well as studies on pharmaceutical interventions to upregulate the process.

 



List by Sandra Malmgren Hill

Lung Disease and Regeneration

This preprint list compiles highlights from the field of lung biology.

 



List by Rob Hynds

Cellular metabolism

A curated list of preprints related to cellular metabolism at Biorxiv by Pablo Ranea Robles from the Prelights community. Special interest on lipid metabolism, peroxisomes and mitochondria.

 



List by Pablo Ranea Robles

BSCB/BSDB Annual Meeting 2019

Preprints presented at the BSCB/BSDB Annual Meeting 2019

 



List by Dey Lab

MitoList

This list of preprints is focused on work expanding our knowledge on mitochondria in any organism, tissue or cell type, from the normal biology to the pathology.

 



List by Sandra Franco Iborra

Biophysical Society Annual Meeting 2019

Few of the preprints that were discussed in the recent BPS annual meeting at Baltimore, USA

 



List by Joseph Jose Thottacherry

ASCB/EMBO Annual Meeting 2018

This list relates to preprints that were discussed at the recent ASCB conference.

 



List by Dey Lab, Amanda Haage
Close