Real-Time Multi-Angle Projection Imaging of Biological Dynamics

Bo-Jui Chang, Etai Sapoznik, Theresa Pohlkamp, Tamara S. Terrones, Erik S. Welf, James D. Manton, Philippe Roudot, Kayley Hake, Lachlan Whitehead, Andrew G. York, Kevin M. Dean, Reto Fiolka

Preprint posted on October 29, 2020

Here, there and everywhere: projection imaging of biological dynamics

Selected by Mariana De Niz


Quantitative biological imaging requires Nyquist sampling in space and time. However, due to the limited acquisition rate of laser scanning and camera-based microscopes, many biological processes occur too rapidly to be observed in 3D. This is partly because volumes are usually acquired sequentially with tens-to-hundreds of focal planes. However, imaging rates could be several orders of magnitude faster if information spanning multiple focal planes could be integrated into a single raster scan or camera exposure. While multiple projection imaging methods exist, they have common hindrances, including that they typically deteriorate lateral resolution; that they require specialized setups, and that they provide only projections along the optical axis. In their work, Chang et al (1) introduce a cost-effective and easy-to-implement scan unit which enables any camera-based microscope to perform projection imaging from diverse angles. By imaging the sample from one or multiple perspectives, their novel method enables visualization of rapid biological processes, real-time stereoscopic imaging and 3D particle localization throughout a cellular volume from just two images.

Figure 1. Principle of Multi-angle projection imaging. Top panels: conventional LLS or OPM microscope: the sample is scanned on a diagonal trajectory relative to the detection axis while being illuminated with a light sheet. Bottom panels: for projection imaging under different viewing angles, a lateral shearing unit consisting of 2 galvanometric mirrors is added in front of the camera. When the sample is scanned, the two mirrors are rotated in synchrony causing the image to be displaced laterally on the camera. (From Ref. 1).


Key findings and developments

In their work, Chang et al introduce a simple scan unit that converts any camera-based microscope into a projection imaging system which can integrate information from various viewing angles. Instead of requiring sample rotation, the method uses optical shearing to provide various viewing directions of the sample. Moreover, the method can be combined with Lattice Light-sheet Microscopy (LLSM) and Oblique Plane Microscopy (OPM).  Importantly, while both of these imaging methods require computational shearing of the image data for visualization and quantification, the projection method developed by Chang and co-authors performs optical shearing during the acquisition of a single camera frame, resulting in a more intuitive instantaneous 3D rendering of the sample. The method involves the development of a simple galvanometer-based module that attaches to the microscope camera and shears the data optically instead of computationally. The scan unit is swept synchronously with the acquisition of a z-stack, resulting in the projection of high-contrast, high-resolution volumetric data onto a single camera frame. The magnitude of the scan sweep can be changed in order to obtain projections from different viewing perspectives in a way that is analogous to a shear-warp transform.

As proof of principle, the authors imaged mammalian cells using LLSM-Pro (i.e. LLSM combined with the projection mode here developed), and OPM-Pro. In both cases, the authors were able to switch between projection and volumetric modes. Computationally projected data acquired with a LLSM was indistinguishable from data acquired with a LLSM-Pro, with the difference that LLSM-Pro acquired the data in a single image frame, and thus reduced the imaging time and data overhead significantly. The same was true using OPM-Pro for both simple and complex samples. Another feature explored, was the speed of image acquisition. The authors were able to image rapid cellular dynamics such as the formation and retraction of pressure-based blebs with LLSM-Pro or calcium waves in cultured neurons with OPM-Pro. They note that in LLSM-Pro, the imaging speed is limited by the speed of the piezo stage where the sample is mounted on. In OPM-Pro, which uses much faster laser scanning optics than LLSM, the imaging speed is limited only by the camera frame rate or the available fluorescence signal. However, the authors also state that because fluorophores are only illuminated for a fraction of the camera exposure, the sensitivity of the method decreases as the imaged volume grows larger.

Besides the acquisition speed, the authors note that the possibility of acquiring data from multiple perspectives opens the door to a number of exciting opportunities, including rendering images stereoscopically. Equally, images acquired from different projection perspectives encode volumetric information otherwise difficult to acquire in a timely manner, like the distribution of nanoparticles throughout a 3D volume. Also, by adding a second camera to their microscope, they were able to simultaneously visualize the rapidly beating heart of a 3-day post-fertilization zebrafish embryo from orthogonal imaging perspectives with high spatial resolution and no apparent distortions. The authors further emphasize that unlike other methods, their method does not interfere with normal microscope operation. They also point out that their projection technique can cover large volumes, and this is only limited by the axial scan range while maintaining an invariant PSF. Lastly, because LLSM and OPM require computational shearing (a.k.a. deskewing) of their raw data before it can be intuitively visualized, exploring biological samples can be particularly challenging.  In contrast, the projection method developed here allows viewing of deskewed and projected volumes in real-time and as such greatly simplifies user interaction with such microscopes.

The authors conclude that their novel method exploits the shear-warp transform, which obtains rotated projections at intermediate angles without physically rotating the sample or imaging it through multiple detection objectives. Moreover, they explore multiple applications where this exciting approach would be very advantageous and propose that this tool will allow the transformation of cutting-edge imaging technologies into user-friendly and interactive data-acquisition and hypothesis-testing machines.


What I like about this preprint

 I like novel technologies that address methodological gaps. I think addressing those gaps will be key for the advancement of scientific exploration. Having consistently used intravital microscopy, and the hurdles that in this tool we must overcome, it seems this tool hereby proposed will be of great use for many scientific fields requiring fast imaging and/or volumetric imaging.



  1. Chang et al, Real-time multi-angle projection imaging of biological dynamics, bioRxiv, 2020


Posted on: 17th December 2020


Read preprint (No Ratings Yet)

Author's response

Bo-Jui Chang, Kevin M. Dean, Reto Fiolka shared

Open questions 

1. This is a wonderful development. What are current limitations to be aware of, for instance in terms of resolution, imaging depth, and speed? You gave as examples, 3D imaging of nanoparticles and imaging the beating heart of a zebrafish embryo.

Currently, the sample brightness and scanning speed can put limits on the acquisition speed. As mentioned in the paper, one can imagine one projection as the sum of “virtual” images, which can have hundredfold shorter exposure time than what can be normally achieved with the camera.

In terms of depth, we think that more mesoscopic light-sheet systems could leverage this technology to provide live projections of entire embryos – combined with the ability to rapidly change the viewing angle could lead to much quicker exploration of such samples.

The spatial resolution remains the same as in a lateral single view. This is different to other projection methods, which deteriorate the PSF, especially when increasing the depth range.

2. You describe in detail the implementation of the projection method into LLSM and OPM. Is it possible to implement it on other systems?

Yes, we see practically no limitation to implement this on other light-sheet systems or for example a spinning disk. The microscope should have optical sectioning capability and a reasonably fast z-scanning capability.

3. In terms of image analysis, are there already existing pipelines that can be used on data acquired using LLSM-Pro or OPM-Pro? If not, are you or others developing some?

3D reconstruction from projection data sets is still a work in progress. However, we believe that this problem has been extensively investigated in the biomedical imaging field, for example for Computed Tomography. We hope that methods to speed up, or improve, CT scans can be applicable here. One area where we see immediate potential is also 3D tracking. This requires adaptation of current tracking algorithm to include constraints between the detected particles in each view. This still has not been finished, and we think there will be opportunities for the development of new 3D tracking strategies.

4. You mentioned in your work the issue of fluorescence signals. Are there specific fluorescence proteins or dyes that you recommend are better suited for the use of this method, or some to be avoided?

We have not carefully explored which dyes and fluorophores are best for this method.  However, it is always best to use bright fluorescent proteins (e.g., mNeonGreen), biosensors (e.g., GCaMP), or small molecule fluorophores (e.g., JF549) when imaging since these allow you to minimize the illumination burden on the specimen, while maintaining high signal-to-noise ratios. 

5. With the current momentum of organoids, have you tested your system for imaging organs on chip or organoids, and do you envisage this as an application?

Our current microscope systems were designed to achieve the highest resolution possible in single cells. However, we are building an OPM system that will be able to rapidly image larger volumes, including organoids, and we are planning to augment it with our projection unit.

6. Is your method compatible with other advanced imaging methods, for instance FRAP, FRET, etc.?

We believe that our method can be combined with photoactivation, FRAP, FRET, and other advanced imaging methods, so long it biologically makes sense to observe the outcome of such experiments in a 2D projection format. 

Have your say

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Sign up to customise the site to your preferences and to receive alerts

Register here

Also in the cell biology category:

FENS 2020

A collection of preprints presented during the virtual meeting of the Federation of European Neuroscience Societies (FENS) in 2020


List by Ana Dorrego-Rivas

Planar Cell Polarity – PCP

This preList contains preprints about the latest findings on Planar Cell Polarity (PCP) in various model organisms at the molecular, cellular and tissue levels.


List by Ana Dorrego-Rivas

BioMalPar XVI: Biology and Pathology of the Malaria Parasite

[under construction] Preprints presented at the (fully virtual) EMBL BioMalPar XVI, 17-18 May 2020 #emblmalaria


List by Dey Lab, Samantha Seah


Cell Polarity

Recent research from the field of cell polarity is summarized in this list of preprints. It comprises of studies focusing on various forms of cell polarity ranging from epithelial polarity, planar cell polarity to front-to-rear polarity.


List by Yamini Ravichandran

TAGC 2020

Preprints recently presented at the virtual Allied Genetics Conference, April 22-26, 2020. #TAGC20


List by Maiko Kitaoka, Madhuja Samaddar, Miguel V. Almeida, Sejal Davla, Jennifer Ann Black, Dey Lab

3D Gastruloids

A curated list of preprints related to Gastruloids (in vitro models of early development obtained by 3D aggregation of embryonic cells). Preprint missing? Don't hesitate to let us know.


List by Paul Gerald L. Sanchez and Stefano Vianello

ECFG15 – Fungal biology

Preprints presented at 15th European Conference on Fungal Genetics 17-20 February 2020 Rome


List by Hiral Shah

ASCB EMBO Annual Meeting 2019

A collection of preprints presented at the 2019 ASCB EMBO Meeting in Washington, DC (December 7-11)


List by Madhuja Samaddar, Ramona Jühlen, Amanda Haage, Laura McCormick, Maiko Kitaoka

EMBL Seeing is Believing – Imaging the Molecular Processes of Life

Preprints discussed at the 2019 edition of Seeing is Believing, at EMBL Heidelberg from the 9th-12th October 2019


List by Dey Lab


Preprints on autophagy and lysosomal degradation and its role in neurodegeneration and disease. Includes molecular mechanisms, upstream signalling and regulation as well as studies on pharmaceutical interventions to upregulate the process.


List by Sandra Malmgren Hill

Lung Disease and Regeneration

This preprint list compiles highlights from the field of lung biology.


List by Rob Hynds

Cellular metabolism

A curated list of preprints related to cellular metabolism at Biorxiv by Pablo Ranea Robles from the Prelights community. Special interest on lipid metabolism, peroxisomes and mitochondria.


List by Pablo Ranea Robles

BSCB/BSDB Annual Meeting 2019

Preprints presented at the BSCB/BSDB Annual Meeting 2019


List by Dey Lab


This list of preprints is focused on work expanding our knowledge on mitochondria in any organism, tissue or cell type, from the normal biology to the pathology.


List by Sandra Franco Iborra

Biophysical Society Annual Meeting 2019

Few of the preprints that were discussed in the recent BPS annual meeting at Baltimore, USA


List by Joseph Jose Thottacherry

ASCB/EMBO Annual Meeting 2018

This list relates to preprints that were discussed at the recent ASCB conference.


List by Dey Lab, Amanda Haage