10x Genomics
Chromium Single Cell Gene Expression

Cell Ranger3.0, printed on 11/21/2024

Aggregating Multiple GEM Wells with cellranger aggr

When doing large studies involving multiple GEM wells, run cellranger count on FASTQ data from each of the GEM wells individually, and then pool the results using cellranger aggr, as described here.

The cellranger aggr command takes a CSV file specifying a list of cellranger count output files (specifically the molecule_info.h5 from each run), and produces a single feature-barcode matrix containing all the data.

When combining multiple GEM wells, the barcode sequences for each channel are distinguished by a GEM well suffix appended to the barcode sequence (see GEM wells).

By default, the reads from each GEM well are subsampled such that all GEM wells have the same effective sequencing depth, measured in terms of reads that are confidently mapped to the transcriptome or assigned to the feature IDs per cell. However, it is possible to change the depth normalization mode (see Depth Normalization).

Requirements

The first step is to run cellranger count on each individual GEM well prepared using the 10x Chromium™ platform, as described in Single-GEM Well Analysis.

For example, suppose you ran three count pipelines as follows:

$ cd /opt/runs
$ cellranger count --id=LV123 ...
... wait for pipeline to finish ...
$ cellranger count --id=LB456 ...
... wait for pipeline to finish ...
$ cellranger count --id=LP789 ...
... wait for pipeline to finish ...

Now you can aggregate these three runs to get a single feature-barcode matrix and analysis. In order to do so, you need to create an Aggregation CSV.

Setting Up An Aggregation CSV

Create a CSV file with a header line containing the following columns:

• library_id: Unique identifier for this input GEM well. This will be used for labeling purposes only; it doesn't need to match any previous ID you've assigned to the GEM well.
• molecule_h5: Path to the molecule_info.h5 file produced by cellranger count. For example, if you processed your GEM well by calling cellranger count --id=ID in some directory /DIR, this path would be /DIR/ID/outs/molecule_info.h5.

You can either make the CSV file in a text editor, or create it in Excel and export to CSV. Continuing the example from the previous section, your Excel spreadsheet would look like this:

When you save it as a CSV, the result would look like this:

library_id,molecule_h5
LV123,/opt/runs/LV123/outs/molecule_info.h5
LB456,/opt/runs/LB456/outs/molecule_info.h5
LP789,/opt/runs/LP789/outs/molecule_info.h5

In addition to the CSV columns expected by cellranger aggr, you may optionally supply additional columns containing library meta-data (e.g., lab or sample origin). These custom library annotations do not affect the analysis pipeline but can be visualized downstream in the Loupe Cell Browser (see below). Note that unlike other CSV inputs to Cell Ranger, these custom columns may contain characters outside the ASCII range (e.g., non-Latin characters).

Aggregating Libraries With Different Chemistry Versions

If you are aggregating libraries generated by different chemistry versions of the Single Cell Gene Expression Reagents, you might observe systematic differences in gene expression profiles between libraries. cellranger aggr provides the batch effect correction (algorithm details) to overcome this. To enable this module, you should include the following column in your aggregation CSV file:

• batch: (optional) Unique identifier for the batch that this GEM well belongs to. Libraries with the same batch identifier will be considered to be in the same batch.

For example, if the LV123 sample in the previous example is a v2 library, and the LB456 and LP789 samples are v3 libraries, you would set up the aggregation CSV file like this:

library_id,molecule_h5,batch
LV123,/opt/runs/LV123/outs/molecule_info.h5,v2_lib
LB456,/opt/runs/LB456/outs/molecule_info.h5,v3_lib
LP789,/opt/runs/LP789/outs/molecule_info.h5,v3_lib

The v2_lib and v3_lib identifiers are just example identifiers. Every sample from a given batch has to have the same batch identifier, but otherwise the identifier itself is arbitrary.

About Chemistry Batch Correction

• This Chemistry Batch Correction is specifically intended to correct for systematic variability in gene expression profiles caused by different versions of the Single Cell Gene Expression chemistry. 10x has tested and verified its effectiveness primarily on aggregating Single Cell Gene Expression v2 and v3 chemistry with well-matched input material. The module may be useful in other scenarios but will require careful validation of results.
• Chemistry batch correction affects the PCA, t-SNE visualization and clustering results. Values in the aggregated feature-barcode matrix are not adjusted by Chemistry Batch Correction. Differential expression analysis is still performed on the feature-barcode count matrix.
• The batch effect score (described below) is recommended to compare the performance of batch correction. Besides the batch effect, it also depends on the composition of the cell population across batches.
• When the chemistry batch correction is enabled, the FBPCA is used to perform dimensionality reduction, instead of the IRLBA PCA.
• The minimum System Requirements of 64GB RAM will allow batch correction on datasets with a total number of 128k cells.

Aggregating 5' and 3' Gene Expression Data

cellranger aggr and the Chemistry Batch Correction module can aggregrate results for a combination of 5' and 3' v2 or 3' v3 Gene Expression data. Enabling Chemistry Batch Correction in this scenario improves the mixing of the batches in the t-SNE visualization and clustering results. Therefore we recommend using Chemistry Batch correction. However, residual batch effects may still be present, and we advise careful validation of the results. In particular for the V(D)J genes, the 5' assay will generally count the V gene segments of the immune recepetor (e.g. TRBV12-1 or IGH4-2), while the 3' assay will count the C gene segments (e.g. TRBC or IGHA), which may pose additional analysis challenges.

Command Line Interface

These are the most common command line arguments (run cellranger aggr --help for a full list):

After specifying these input arguments, run cellranger aggr:

$ cd /home/jdoe/runs
$ cellranger aggr --id=AGG123 \
                  --csv=AGG123_libraries.csv \
                  --normalize=mapped

The pipeline will begin to run, creating a new folder named with the aggregation ID you specified (e.g. /home/jdoe/runs/AGG123) for its output. If this folder already exists, cellranger will assume it is an existing pipestance and attempt to resume running it.

Pipeline Outputs

The cellranger aggr pipeline generates output files that contain all of the data from the individual input jobs, aggregated into single output files, for convenient multi-sample analysis. The GEM well suffix of each barcode be updated to prevent barcode collisions, as described below.

Each output file produced by cellranger aggr follows the format described in the Understanding Output section of the documentation, but includes the union of all the relevant barcodes from each input jobs.

A successful run should conclude with a message similar to this:

2018-10-04 13:36:33 [runtime] (run:local)       ID.AGG123.SC_RNA_AGGREGATOR_CS.SC_RNA_AGGREGATOR.SUMMARIZE_AGGREGATED_REPORTS.fork0.join
2018-10-04 13:36:36 [runtime] (join_complete)   ID.AGG123.SC_RNA_AGGREGATOR_CS.SC_RNA_AGGREGATOR.SUMMARIZE_AGGREGATED_REPORTS
2018-10-04 13:36:45 [runtime] VDR killed 210 files, 29MB.
 
Outputs:
- Aggregation metrics summary HTML:         /home/jdoe/runs/AGG123/outs/web_summary.html
- Aggregation metrics summary JSON:         /home/jdoe/runs/AGG123/outs/summary.json
- Secondary analysis output CSV:            /home/jdoe/runs/AGG123/outs/analysis
- Filtered feature-barcode matrices MEX:    /home/jdoe/runs/AGG123/outs/filtered_feature_bc_matrix
- Filtered feature-barcode matrices HDF5:   /home/jdoe/runs/AGG123/outs/filtered_feature_bc_matrix.h5
- Unfiltered feature-barcode matrices MEX:  /home/jdoe/runs/AGG123/outs/raw_feature_bc_matrix
- Unfiltered feature-barcode matrices HDF5: /home/jdoe/runs/AGG123/outs/raw_feature_bc_matrix.h5
- Unfiltered molecule-level info:           /home/jdoe/runs/AGG123/outs/raw_molecules.h5
- Barcodes of cell-containing partitions:   /home/jdoe/runs/AGG123/outs/cell_barcodes.csv
- Copy of the input aggregation CSV:        /home/jdoe/runs/AGG123/outs/aggregation.csv
- Loupe Cell Browser file:                  /home/jdoe/runs/AGG123/outs/cloupe.cloupe
 
Pipestance completed successfully!

Once cellranger aggr has successfully completed, you can browse the resulting summary HTML file in any supported web browser, open the .cloupe file in Loupe Cell Browser, or refer to the Understanding Output section to explore the data by hand. For machine-readable versions of the summary metrics, refer to the cellranger aggr section of the Summary Metrics page.

Understanding GEM Wells

Each GEM well is a physically distinct set of GEM partitions, but draws barcode sequences randomly from the pool of valid barcodes, known as the barcode whitelist. To keep the barcodes unique when aggregating multiple libraries, we append a small integer identifying the GEM well to the barcode nucleotide sequence, and use that nucleotide sequence plus ID as the unique identifier in the feature-barcode matrix. For example, AGACCATTGAGACTTA-1 and AGACCATTGAGACTTA-2 are distinct cell barcodes from different GEM wells, despite having the same barcode nucleotide sequence.

This number, which tells us which GEM well this barcode sequence came from, is called the GEM well suffix. The numbering of the GEM wells will reflect the order that the GEM wells were provided in the Aggregation CSV.

Depth Normalization

When combining data from multiple GEM wells, the cellranger aggr pipeline automatically equalizes the read depth between groups before merging, which is the recommended approach in order to avoid the batch effect introduced by sequencing depth. It is possible to turn off normalization or change the way normalization is done. The none option may be appropriate if you want to maximize sensitivity and plan to deal with depth normalization in a downstream step.

There are two normalization modes:

• mapped (default): For each library type, subsample reads from higher-depth GEM wells until they all have an equal number of reads that are confidently mapped to the transcriptome or assigned to the feature IDs per cell.
• none: Do not normalize at all.