Menu

Close

Applications, Promises, and Pitfalls of Deep Learning for Fluorescence Image Reconstruction

Chinmay Belthangady , Loic A. Royer

Preprint posted on December 11, 2018 https://www.preprints.org/manuscript/201812.0137/v1

Belthangady & Royer present a comprehensive and accessible perspective on the current and future impacts of Artificial Intelligence on microscopy. Their vision for the field is inspiring and sets the ground for an AI revolution.

Selected by Romain F. Laine

Background of the preprint

The development of artificial intelligence (AI) is driving fundamental changes in the way we travel (self-driving cars), we work (automation) and we interact with the world. It is also being exploited to improve diagnosis and detection of cancer. But there is no reason to stop here, fundamental research can also benefit from these advances and this has triggered the start of a revolution in the field of microscopy. The past couple of years have seen a growing literature on application of artificial intelligence to all kind of imaging techniques for biomedical research. Here, Chinmay Belthangady & Loic Royer from the Chan Zuckerberg Biohub, San Francisco, USA review the recent and promising developments in the field. This review aims at clearly presenting the wide range of applications of AI and its pitfalls and gives perspective to Loic Royer’s seminal work on fluorescence microscopy image restoration using AI [1].

Key points of the preprint

The paper first introduces a number of fundamental definitions about AI in the context of fluorescence microscopy. AI can generate an accurate mathematical transform that is able to take a corrupted microscopy image (e.g. with a lot of noise, or a low resolution) and correct for this image corruption. The way it does this is by learning how the data corruption occurs through looking at a lot of examples of data where the same type of corruption is present (the so-called ‘training dataset’). The authors describe how the typical neural networks used to carry this out can be thought of, e.g. a sequence of mathematical functions that progressively extract the important features in the image. The notion that AI acts as a black box is commonly founded on the fact that these features can be very abstract and therefore cannot be easily understood by the researchers.

The recent applications of AI to microscopy have showed that it can significantly improve noise or out-of-focus light removal and even provide super-resolution in unprecedented ways. The authors highlight that one major advantage of this is to be able to significantly reduce illumination on the sample because the images can be corrected post-acquisition, therefore enabling non-invasive long-term live sample imaging.

The authors then present a number of exciting directions where AI could be useful and have a major impact, e.g. massive multi-color imaging, image registration and alignment, intelligent and AI-guided image acquisition, or even providing a higher level of data mining: “In future, we expect that Deep Learning models trained end-to-end will blur the frontier between image reconstruction and image analysis.”

Potential applications of Deep Learning in fluorescence microscopy and key concepts. (a) Learning to reduce scattered light ’haze’ in light-sheet microscopy. (b) Learning spectral unmixing of simultaneous multi-color acquisitions. (c) Learning to reconstruct super-resolved images from structured illumination acquisitions. (d) Learning to straighten live dynamic samples with weak supervision against a template shape. (e) Fluorescence optical flow with Deep Learning. Forward model simulations can produce time-lapse data for given vector fields, and a network can be trained to compute the inverse: vector fields from fluorescence spatio-temporal fluctuations.

This manuscript also elegantly explains what the limitations of the approach are. When used with the wrong type of training dataset or when too few examples are available for it to learn, it can create hallucinations and therefore make up very realistic features that were not actually present in the underlying sample. The authors make a convincing case about looking beyond the scepticism sometimes found around applying AI to microscopy and focussing on where it is appropriate to use it and its performance.

What I like about this preprint

This preprint provides the accessible overview about AI applied to microscopy, that I have been waiting for, for a couple of years now. It is very well written and presents complex concepts with well-chosen examples. The authors make a balanced case of the pros and cons of AI, but highlight the areas where its potential remains untapped.

Future directions

It is clear to me that AI will change fundamentally the way we acquire and analyse our microscopy data. But in what ways? While this still remains to be established, this preprint sets the ground for an exciting path full of potential. For AI to become mainstream in microscopy, the bottleneck in my opinion still is usability and accessibility of these complex mathematical tools. Will every research department or institute soon need their own in-house AI team? Will these be run like some bioinformatics facilities are today?

[1] Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, (2018).

Tags: artificial intelligence, deep learning, fluorescence microscopy

Posted on: 9th January 2019

doi: https://doi.org/10.1242/prelights.7148

Read preprint (4 votes)




Have your say

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Sign up to customise the site to your preferences and to receive alerts

Register here

Also in the cell biology category:

FENS 2020

A collection of preprints presented during the virtual meeting of the Federation of European Neuroscience Societies (FENS) in 2020

 



List by Ana Dorrego-Rivas

Planar Cell Polarity – PCP

This preList contains preprints about the latest findings on Planar Cell Polarity (PCP) in various model organisms at the molecular, cellular and tissue levels.

 



List by Ana Dorrego-Rivas

BioMalPar XVI: Biology and Pathology of the Malaria Parasite

[under construction] Preprints presented at the (fully virtual) EMBL BioMalPar XVI, 17-18 May 2020 #emblmalaria

 



List by Gautam Dey, Samantha Seah

1

Cell Polarity

Recent research from the field of cell polarity is summarized in this list of preprints. It comprises of studies focusing on various forms of cell polarity ranging from epithelial polarity, planar cell polarity to front-to-rear polarity.

 



List by Yamini Ravichandran

TAGC 2020

Preprints recently presented at the virtual Allied Genetics Conference, April 22-26, 2020. #TAGC20

 



List by Maiko Kitaoka, Madhuja Samaddar, Miguel V. Almeida, Sejal Davla, Jennifer Ann Black, Gautam Dey

3D Gastruloids

A curated list of preprints related to Gastruloids (in vitro models of early development obtained by 3D aggregation of embryonic cells). Preprint missing? Don't hesitate to let us know.

 



List by Paul Gerald L. Sanchez and Stefano Vianello

ECFG15 – Fungal biology

Preprints presented at 15th European Conference on Fungal Genetics 17-20 February 2020 Rome

 



List by Hiral Shah

ASCB EMBO Annual Meeting 2019

A collection of preprints presented at the 2019 ASCB EMBO Meeting in Washington, DC (December 7-11)

 



List by Madhuja Samaddar, Ramona Jühlen, Amanda Haage, Laura McCormick, Maiko Kitaoka

EMBL Seeing is Believing – Imaging the Molecular Processes of Life

Preprints discussed at the 2019 edition of Seeing is Believing, at EMBL Heidelberg from the 9th-12th October 2019

 



List by Gautam Dey

Autophagy

Preprints on autophagy and lysosomal degradation and its role in neurodegeneration and disease. Includes molecular mechanisms, upstream signalling and regulation as well as studies on pharmaceutical interventions to upregulate the process.

 



List by Sandra Malmgren Hill

Lung Disease and Regeneration

This preprint list compiles highlights from the field of lung biology.

 



List by Rob Hynds

Cellular metabolism

A curated list of preprints related to cellular metabolism at Biorxiv by Pablo Ranea Robles from the Prelights community. Special interest on lipid metabolism, peroxisomes and mitochondria.

 



List by Pablo Ranea Robles

BSCB/BSDB Annual Meeting 2019

Preprints presented at the BSCB/BSDB Annual Meeting 2019

 



List by Gautam Dey

MitoList

This list of preprints is focused on work expanding our knowledge on mitochondria in any organism, tissue or cell type, from the normal biology to the pathology.

 



List by Sandra Franco Iborra

Biophysical Society Annual Meeting 2019

Few of the preprints that were discussed in the recent BPS annual meeting at Baltimore, USA

 



List by Joseph Jose Thottacherry

ASCB/EMBO Annual Meeting 2018

This list relates to preprints that were discussed at the recent ASCB conference.

 



List by Gautam Dey, Amanda Haage

Also in the developmental biology category:

EMBL Conference: From functional genomics to systems biology

Preprints presented at the virtual EMBL conference "from functional genomics and systems biology", 16-19 November 2020

 



List by Jesus Victorino

Single Cell Biology 2020

A list of preprints mentioned at the Wellcome Genome Campus Single Cell Biology 2020 meeting.

 



List by Alex Eve

Society for Developmental Biology 79th Annual Meeting

Preprints at SDB 2020

 



List by Irepan Salvador-Martinez, Martin Estermann

FENS 2020

A collection of preprints presented during the virtual meeting of the Federation of European Neuroscience Societies (FENS) in 2020

 



List by Ana Dorrego-Rivas

Planar Cell Polarity – PCP

This preList contains preprints about the latest findings on Planar Cell Polarity (PCP) in various model organisms at the molecular, cellular and tissue levels.

 



List by Ana Dorrego-Rivas

Cell Polarity

Recent research from the field of cell polarity is summarized in this list of preprints. It comprises of studies focusing on various forms of cell polarity ranging from epithelial polarity, planar cell polarity to front-to-rear polarity.

 



List by Yamini Ravichandran

TAGC 2020

Preprints recently presented at the virtual Allied Genetics Conference, April 22-26, 2020. #TAGC20

 



List by Maiko Kitaoka, Madhuja Samaddar, Miguel V. Almeida, Sejal Davla, Jennifer Ann Black, Gautam Dey

3D Gastruloids

A curated list of preprints related to Gastruloids (in vitro models of early development obtained by 3D aggregation of embryonic cells). Preprint missing? Don't hesitate to let us know.

 



List by Paul Gerald L. Sanchez and Stefano Vianello

ASCB EMBO Annual Meeting 2019

A collection of preprints presented at the 2019 ASCB EMBO Meeting in Washington, DC (December 7-11)

 



List by Madhuja Samaddar, Ramona Jühlen, Amanda Haage, Laura McCormick, Maiko Kitaoka

EDBC Alicante 2019

Preprints presented at the European Developmental Biology Congress (EDBC) in Alicante, October 23-26 2019.

 



List by Sergio Menchero, Jesus Victorino, Teresa Rayon, Irepan Salvador-Martinez

EMBL Seeing is Believing – Imaging the Molecular Processes of Life

Preprints discussed at the 2019 edition of Seeing is Believing, at EMBL Heidelberg from the 9th-12th October 2019

 



List by Gautam Dey

SDB 78th Annual Meeting 2019

A curation of the preprints presented at the SDB meeting in Boston, July 26-30 2019. The preList will be updated throughout the duration of the meeting.

 



List by Alex Eve

Lung Disease and Regeneration

This preprint list compiles highlights from the field of lung biology.

 



List by Rob Hynds

Young Embryologist Network Conference 2019

Preprints presented at the Young Embryologist Network 2019 conference, 13 May, The Francis Crick Institute, London

 



List by Alex Eve

Pattern formation during development

The aim of this preList is to integrate results about the mechanisms that govern patterning during development, from genes implicated in the processes to theoritical models of pattern formation in nature.

 



List by Alexa Sadier

BSCB/BSDB Annual Meeting 2019

Preprints presented at the BSCB/BSDB Annual Meeting 2019

 



List by Gautam Dey

Zebrafish immunology

A compilation of cutting-edge research that uses the zebrafish as a model system to elucidate novel immunological mechanisms in health and disease.

 



List by Shikha Nayar
Close