Background

Cells contain hundreds of distinct membrane-bound organelles and macromolecular assemblies that drive the cell’s activities. Although there is considerable knowledge regarding the biochemical and genetic information of these structures, there is little knowledge on the 3D distributions, morphologies, and interactions throughout a cell, at nanometric resolution. Until recently, the main hindrance to obtaining such ultrastructural detail were technological limitations. Modern focused ion beam scanning electron microscopy (FIB-SEM), however, is one of the tools that has addressed these limitations, and provides near-isotropic resolution of cellular structures with nanometric voxel sizes through an entire cell. In parallel, innovations allowing the automatic analysis of the very large volume EM data have provided a means for accessing this detailed information. In their work, Heinrich et al (1) developed an analysis pipeline based on deep learning architectures for segmentation, allowing comprehensive reconstruction and analysis of organelles within entire cells imaged by FIB-SEM.

Key findings and developments

To obtain ground truth annotation for training machine learning algorithms,  the authors selected regions of interest for manual annotation of up to 35 different organelles and macromolecular structures, based on morphological features previously described in the literature. They distinguished structures such as mitochondria, microtubules, endoplasmic reticulum, nuclear envelope, and various components of the endolysosomal system, among others, and noted that each structure can be identified with different levels of complexity. Moreover, they mention that manual annotation of the entire cell would take a person 60 years, thus highlighting the need for automation. Noting that the performance of deep learning-based methods depends on representative training data, the authors wanted to generate a method for automatic segmentation that could generalize across cell types and variations in imaging parameters. Training blocks were used to train large, multi-channel 3D-U-Net architectures to predict boundary distances using binary labels for each organelle class. Using the trained networks, they predicted organelle segmentation on whole cells. The authors then developed a manual evaluation method based on pairwise comparisons of whole-cell predictions to determine the optimal training iteration and network architecture for unseen datasets, without a costly generation of validation blocks. They further assessed the performance of the machine learning method, and validated the manual approach in additional holdout blocks. Overall, the results showed good performance scores for organelles well represented in the training data, and setups including more organelles tended to perform slightly better. The authors report that for some features, further refinements were applied to improve segmentation quality, including smoothing, size filtering, watershed segmentation and agglomeration, and masking. They report that the reconstruction of some organelles yielded information previously unknown, for instance, the complexity of the Golgi apparatus.

Investigating microtubules. In the four cells (two HeLa cells, a Jurkat cell and a macrophage cell), the authors show the different quantitative information that can be obtained using the automated pipeline, including count, volume, and surface area of each organelle. Moreover, the possibility to segment out membrane-bound organelles as well as single microtubules allowed investigating the question of how many different organelles are contacting a single microtubule at any time. The authors note these are unprecedented findings that open up new questions related to how organelle distributions and motions are coordinated along a single microtubule.

Investigating the ER. As a further example, the authors quantified ER morphology, including its relationship to other organelles, and partitioned the ER into planar and tubular regions based on its curvature.  The authors went on to compare the planar and tubular regions between HeLa cells and macrophages, and showed that a larger fraction of the planar ER regions are supported by mitochondria in the HeLa cells. Given the differences between planar and tubular ER, the authors examined also the relative abundance of ribosomes associated with these ER domains, and using contact site distance, categorized ribosomes as either ER-bound or cytosolic. It appeared that the percentage of ribosomes bound to tubular peripheral ER and the nuclear envelope was consistent across the four cells. Conversely, the relative percentage of ribosomes bound to planar peripheral ER was 3-fold higher in the macrophage than in the other cells.

4nm resolution vs. 8nm resolution. As many FIB-SEM data are acquired at 8nm resolution to reduce imaging time, the authors trained 8nm versions of some of the setups initially acquired at 4nm, by adding unsampling blocks to the 3D-U-Net architecture and randomly downsampling the raw data for training. They found the segmentation and results, satisfactory.

CLEM. Fluorescent light microscopy, in combination with highly specific molecular markers, provides complementary information to high resolution non-specific EM. Using the predictions for mitochondrial membranes, the authors automated the registration process for CLEM datasets, and demonstrated its use in a previously published COS-7 cell transiently expressing ER luminal and mitochondria membrane markers, imaged by PALM and SIM, and FIB-SEM. Comparing manual and automatic registration, the authors report that automatic registration differed from the human annotator by 0.11 µm, while human evaluators differed from each other by 0.03 µm. They conclude that the automated registration agrees with human evaluators in areas with clear mitochondrial signals. Comparing independent registrations of EM to both PALM and SIM showed that errors are small, especially near mitochondria, suggesting that registration is consistent across modalities. This led to the conclusion that automatic organelle segmentation can be used to successfully register CLEM images, making it accessible to less experienced users, and reducing the analysis time significantly.

The authors conclude that to be applicable across domains and new questions, automatic reconstruction methods must generalize robustly across cell types, tissues, and preparation methods. In their work, the authors present a method to fully automatically reconstruct a large number of cellular organelles from FIB-SEM volumes of diverse cell types. Large deep learning architectures were trained to simultaneously reconstruct a large number of cellular organelles and sub-organelle structures at different input resolutions. Moreover, the authors developed the repository “OpenOrganelle” openorganelle.janelia.org, where the raw datasets, training data, reconstructions, open source code, and models are available to the public. Finally, the authors encourage users to explore the datasets and further contribute to scientific discovery.

What I like about this preprint

I like that the authors developed an invaluable tool for cell biology. Often, while hardware based tools take important accelerated steps forward, for instance allowing imaging at resolution never before achieved, the software and analysis pipelines to handle such data often is a few steps back. I think it’s great the authors developed a tool to analyse these rich datasets, that they tested them in various cells, that they identified possible future needs, and that they made all of it publicly available with the hope that open science contributes to the advancement of scientific discovery.

 

Open questions

1. This is a fantastic step for cell biology. You mention that producing training data that is robust enough to allow generalizations across cell types is important. We know that there are important differences across cell types – even more so in cell-cell interactions and upon comparing cells in health and disease. What parameters do you think are important to identify for this baseline training set to apply to as many cell types as possible?
2. You mention that your training data contains no tissue data. This adds another step of complexity whereby cell lines differ from primary cells in their native tissues. Do you know how comparable are the parameters in cell lines as opposed to the primary cells they are models for?
3. How does fixation and sample preparation influence possible changes in different organelles in different cells, and how do you control for this so as to avoid introducing ultrastructural changes in this process (and confusing it with the cell’s natural ultrastructure)?
4. Many pathological conditions affect cell ultrastructure. In your conclusions, you invite the scientific community to explore the datasets here produced as well as the tools you have developed. How do you envisage in the future, that both baseline datasets, and automated FIB-SEM image analysis can be integrated to understand cellular changes in health and disease? For instance, integrating information already known from other types of microscopy to further understand organelle and molecular changes?
5. You analysed 35 different organelles and macro-molecular structures. Can you expand further on important findings, unexpected findings/limitations, challenges or paradigm-shifting observations you were able to identify in your work?

References

1. Heinrich et al, Automatic whole cell organelle segmentation in volumetric electron microscopy, bioRxiv, 2020.

 

doi: https://doi.org/10.1242/prelights.26599

Read preprint (No Ratings Yet)