Ultrafast phasor-based hyperspectral snapshot microscopy for biomedical imaging

Per Niklas Hedde, Rachel Cinco, Leonel Malacrida, Andrés Kamaid, Enrico Gratton

Posted on: 5 November 2020

Preprint posted on 15 October 2020

An intriguing new concept in imaging many colors simultaneously: Ultrafast hyperspectral snapshot microscopy by Per Niklas Hedde, Enrico Gratton and colleagues

Selected by Stephan Daetwyler

Categories: bioengineering, biophysics, cell biology, developmental biology

Context

Fluorescence imaging aims at visualizing the spatial context, interactions and dynamic properties of molecules, cells and tissues. For this purpose, many dyes and fluorescent tags have been developed. However, only a very limited number of fluorophores can be imaged with conventional microscopy approaches simultaneously as they are not able to discriminate between overlapping emission spectra of different dyes and tags (Fig. 1A). To overcome this limitation, hyperspectral imaging methods have been proposed (Qingli Li et al., 2013). These methods aim to acquire the full spectral information for every spatial location. The acquired hyperspectral imaging data can thereby be thought of as data cube with spatial and wavelength coordinates (Fig. 1B).

**Figure 1: Challenges of multispectral imaging. A** Available dyes and protein tags such as ECFP, EGFP, mNeonGreen, EYFP, mBanana, mCherry, dtTomato or dsRed often overlap in their emission spectra. Therefore, traditional imaging method require a careful selection of fluorophores to successfully discriminate the contributions of each fluorophore to an image. B In contrast, hyperspectral imaging methods acquire a contiguous set of all wavelengths across the spectrum. The acquired data thereby can be represented as data cube of spatial (x,y) and spectral (wavelength) information. Using post-processing, the concentrations of the relevant molecules is determined from the data cube.

Different techniques have been proposed to acquire such data. Most widely used implementations leverage sequential acquisition of spectral information with changing filters, or rely on point- or line-wise scanning techniques that disperse the spectra onto multi-channel detectors and cameras. As a consequence, available hyperspectral imaging methods are inherently slow.

Regardless of the acquisition method, the goal of every hyperspectral imaging approach is to determine the concentrations of the relevant molecules from the acquired data cube. Many different algorithms have been designed to perform this task, including linear unmixing algorithms (R. Heylen et al., 2014). However, most of these algorithms rely on computationally intensive and iterative solutions. Interestingly, Fereidouni and colleagues have introduced phasor analysis for spectral imaging as an alternative way to obtain the relative concentrations of up to three relevant molecules (Fereidouni et al., 2012). Phasor analysis has later been expanded for segmentation and validated for up to 7 components, including defining the error in the phasor distributions (Cutrale et al., 2017). Interestingly, phasor analysis can be applied in situations where conventional unmixing and deconvolution does not work due to a lack of informatioin on the expected spectra maxima or shapes.

Phasor analysis thereby relies on calculating for each pixel in the data set (spatial location), two representative numbers (G(n) and S(n)) that characterize the spectrum measured at this location. Therefore, phasor analysis is an effective way to reduce the dimensionality of the problem at hand. To calculate these two numbers, a discrete Fourier transform is applied (Figure 2) :

**Figure 2: Fourier Transform equations to calculate Phasor representation of the spectrum** (from Cutrale et al., 2017).

Key findings and why I chose this preprint

In their preprint, Hedde and colleagues introduce a new paradigm for hyperspectral imaging. Instead of acquiring the complete hyperspectral data cube and calculating its phasor representation with post-processing algorithms, their light sheet microscope only acquires the phasor representation (Figure 3). This considerably enhances hyperspectral imaging by (1) reducing the acquisition time, especially for 3D imaging, and (2) removing the need for computational post-processing to determine the phasor representation.

**Figure 3: Comparison of old and new paradigms for hyperspectral imaging.**

To directly obtain the Phasor representation by imaging, Hedde and colleagues designed two filters – a cosine and a sine filter with changing transmission levels over the observed imaging range (400 – 700 nm). Those two filters optically solve the computational task of applying the Fourier equations to the data cube (Fig. 2) as the measured intensity on the camera is the integral of the signal multiplied with its (cosine/sine) transmission. This simple, yet brilliant idea, dramatically speeds up imaging as only three images have to be acquired – the image of the sine and cosine filter, and image of the total intensity for normalization. Moreover, in addition to application to light sheet microscopy, this strategy is easily implementable on many available microscopes, paving the way for a new area of hyperspectral imaging.

Hedde and colleagues validate their approach with fluorescent beads and demonstrate the approach on various imaging tasks – from subcellular organelle imaging, to imaging of the zebrafish retina, to metabolic imaging in mouse tissue.

Follow-up questions

What is the maximal number of fluorophores that can be separated with the phasor imaging method? How effectively is autofluorescence detected and does it influence the accuracy of detection?
What information is lost by the dimensionality reduction through a Phasor approach? Are there limitations of only obtaining the Phasor representation?
What were the equations that governed the design of the filter and how do they relate to the equations used in Cutrale et al., 2017? What are the rationales for choosing a specific harmonic number?
The authors mention that the filters were designed to reach near-zero transmissions at phases of 270°/180° resulting in a lower accuracy of the spectral information obtained near those minima. How much does this affect detection and quantification of fluorophores at these minima? Are there specific recommendations for the choice of fluorophores, e.g. avoid those with emission around 550– 650 nm?
To further improve the imaging speed, the authors suggest using several cameras or splitting the images on several parts of a chip. This will reduce the available light further. Given that the transmission is already low between 550-600 nm, do the authors think that this might be an issue?
The authors mention interesting applications for ultraviolet and infrared What new biology could be discovered in this range and what excitation wavelengths would be a good starting point for such follow-up experiment?

References

Cutrale, F., Trivedi, V., Trinh, L. A., Chiu, C.-L., Choi, J. M., Artiga, M. S. and Fraser, S. E. (2017). Hyperspectral phasor analysis enables multiplexed 5D in vivo imaging. Nature Methods 14, 149–152.

Fereidouni, F., Bader, A. N. and Gerritsen, H. C. (2012). Spectral phasor analysis allows rapid and reliable unmixing of fluorescence microscopy spectral images. Opt. Express 20, 12729–12741.

Qingli Li, Xiaofu He, Yiting Wang, Hongying Liu, Dongrong Xu and Fangmin Guo (2013). Review of spectral imaging technology in biomedical engineering: achievements and challenges. Journal of Biomedical Optics 18, 1–29.

Heylen, M. Parente and P. Gader (2014). A Review of Nonlinear Hyperspectral Unmixing Methods. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 7, 1844–1868.

Tags: cells, hyperspectral imaging, microscopy, phasor, zebrafish

doi: https://doi.org/10.1242/prelights.25550

Read preprint

(1 votes)

Author's response

Per Niklas Hedde and colleagues shared

What is the maximal number of fluorophores that can be separated with the phasor imaging method? How effectively is autofluorescence detected and does it influence the accuracy of detection?

This will depend mainly on the signal-to-noise and the spectral properties of the dyes (ultimate depends on 1/sqrt(number of photons)). In the paper we show that, given enough photon statistics, 11 dye solutions and 7 mixtures of two dyes can be easily separated, even minimal spectral differences such as between Rhodamine 110 and Fluorescein can be resolved. This can be more difficult in complex biological samples where photon yield is limited and autofluorescence can contribute as a spectrally broad background. However, the background can be accounted as a component and included in the phasor analysis and data interpretation. Generally, given the same signal-to-noise, dyes with more narrow emissions are easier to separate, regardless of the spectral imaging method applied. One important aspect is that phasor analysis can work when there is no a priori information available for the fluorophore characteristics, difficult to accomplish by general unmixing strategies. This model-free strategy is crucial for fluorophores with solvatochromic properties or endogenous fluorophores such as cell metabolites NADH and FAD.

What information is lost by the dimensionality reduction through a Phasor approach? Are there limitations of only obtaining the Phasor representation?

We do not measure the spectrum of the fluorophore, however, the phasor information provides details with regards to spectral shifts and shapes. The phasor approach does not reject any information by the dimensionality reduction, it is just converting the information to a new dimension, the phasor space, where the vector addition rules allow fit-free analyses otherwise not simple to accomplish. The reduction in the data dimension is due to the transformation to a simpler matrix than the (hyper)cube represented in Fig 1B. In the phasor space, all the spectra information is encoded in the real and imaginary components of the Fourier transform. For imaging, obtaining the phasor plot but not the spectrum isn’t typically a limitation where simple direct analysis and interpretation of millions of pixels in a single image is preferred.

What were the equations that governed the design of the filter and how do they relate to the equations used in Cutrale et al., 2017? What are the rationales for choosing a specific harmonic number?

The phasor plot transformation uses a mathematical operation in which the spectral information is multiplied by a sine or cosine function and normalized by the total intensity. In our approach, this operation is achieved by a set of interference filters with transmissions that represent the sine and cosine functions. By normalization with the total intensity the G and S coordinates are obtained. The range of the sine/cosine transmission depends on the wavelength range we decided to cover and the spectral sensitivity of the detectors used. Higher harmonics can improve the resolution in a specific wavelength range at the cost of conceding a univocal total range.

The authors mention that the filters were designed to reach near-zero transmissions at phases of 270°/180° resulting in a lower accuracy of the spectral information obtained near those minima. How much does this affect detection and quantification of fluorophores at these minima? Are there specific recommendations for the choice of fluorophores, e.g. avoid those with emission around 550– 650 nm?

Instead of avoiding areas at the filter transmission minima, we intend to allow a minimal transmission (~10%) in the next generation of filters. By knowing the minimum transmission as instrument parameter, the resulting phasor shift can be compensated in the calculations of the S and G coordinates.

To further improve the imaging speed, the authors suggest using several cameras or splitting the images on several parts of a chip. This will reduce the available light further. Given that the transmission is already low between 550-600 nm, do the authors think that this might be an issue?

Speed is crucial for fast 3D hyperspectral time lapse imaging (5D). For acquisitions on the millisecond timescale the acquisition two parallel channels (sine/cosine) plus the total intensity will be fundamental, which image splitting on the same camera will allow. Considering the light absorption of beam splitters is minimal, we do not expect a reduction of light throughput when splitting the sin/cos filter images. To address the reduced light throughput at the filter transmission minima, we intend to redesign the filters with a ~10% transmission minimum. The shift in S and G coordinates can be compensated in the calculations. Also, we do not need to split the intensity for the three channels equally as long as the split ratio is accounted for in the analysis. Considering that the total intensity is higher than the sine/cosine channels, we can enable higher light throughput by adjusting the split ratio.

The authors mention interesting applications for ultraviolet and infrared What new biology could be discovered in this range and what excitation wavelengths would be a good starting point for such follow-up experiment?

By designing adequate sets of filters, we can cover any range from the ultraviolet (UV) to infrared (IR). NIR-fluorescent probes and proteins have been developed for deep tissue imaging as the 650-950 nm wavelength window can avoid absorption of visible and IR light by living specimen. This wavelength range can be especially important for medical imaging and spectroscopy. Also, our hyperspectral imaging approach is not limited to detecting fluorescence and there are many other fields relying on hyperspectral imaging including satellite imaging in geology and for environment and agriculture surveillance as well as applications in astronomy.