Explore Zarr files

Overview of Xenium Zarr Output Files

<LinkableSection id={"decoding-metric-calc"} title={"Negative control metrics"}>

The [XOA output files](/support/software/xenium-onboard-analysis/3.1/analysis/xoa-output-understanding-outputs) `analysis_summary.html` and `metrics_summary.csv` report negative control metrics.

The "Negative control probe counts per control per cell" metric is derived first from calculating the absolute number of negative control probe counts and then calculating the metric per control probe per cell. The "Estimated number of false positive transcripts per cell" is derived from this metric. Derived values may differ slightly from those reported in summary outputs due to using rounded values.

They are calculated using the following inputs:

- Total high quality decoded transcripts: displayed on the Summary and Decoding tabs, includes genes (Q-Score ≥ 20) and negative controls
- Percent of transcripts within cells: displayed on the Cell Segmentation tab
- **Adjusted negative control probe rate**: displayed on the Decoding tab
- Number of target genes: the sum of genes in the pre-designed panel and any add-on custom genes, displayed on the Summary tab
- **Number of negative control probes in the panel**: in Xenium v1, each pre-designed and add-on panel contains 20 negative control targets (except runs that use the Xenium Mouse Brain base panel, which have 27 negative targets). In Xenium Prime, each pre-designed and add-on panel contains 40 negative control targets.
- Number of cells detected: hero metric on the Summary tab

1. The _Negative control probe count_ is calculated as:

 $$ \footnotesize{\text{Q20 decoded transcripts in cells}\times{\text{Adjusted negative control probe rate}}\times{\frac{\text{Number of negative control probes}}{\text{Number of target genes}}}} $$

 where _Q20 decoded transcripts in cells_ is calculated as:

 $$ \footnotesize{\text{Total high quality decoded transcripts}\times{\text{Percent of transcripts within cells}}} $$

2. The `Negative control probe count per control per cell` is then calculated as:

 $$ \footnotesize{{\text{Negative control probe count}}\div{\text{Number of negative control probes}}\div{\text{Number of cells detected}}} $$

3. The `Estimated number of false positive transcripts per cell` is calculated as:

 $$ \footnotesize{{\text{Negative control probe counts per control per cell}}\times{\text{Number of target genes}}} $$

The calculations for "Genomic control probe counts per control per cell" and "Estimated number of false positive transcripts per cell including genomic counts" are analogous to the calculations shown above. Both metrics are available in outputs generated with Xenium Onboard Analysis v3.0.

The inputs are:

- Total high quality decoded transcripts
- Percent of transcripts within cells
- **Adjusted genomic control probe rate**
- Number of target genes
- **Number of genomic control probes in the panel** (21 for Xenium Prime 5K pre-designed panels)
- Number of cells detected

</LinkableSection>

Metric Calculations

Calculating Negative Control Metrics

What impacts Xenium Analyzer instrument run time?

- Transcript density: impacted by sample preparation (fresh frozen vs. FFPE), number of genes on the panel, tissue type, and disease state
- Tissue thickness
- Tissue area (up to 2.35 cm2 per slide)
- Cell segmentation staining protocol (additional 2 cycles)

The RNA quality, section thickness, expression profile, and the genes on the specific panel affect the detected transcript density. Fresh frozen tissue typically has higher RNA integrity and is sectioned to 10 &micro;m thickness. On average, this results in a higher 2D density of detected transcripts compared to FFPE tissues.

<LinkableSection id={"example"} title={"Run time estimates by assay"}>

The plots below show typical instrument run times for FFPE (left) and fresh frozen (right) samples. Data generation and analysis were performed with Xenium Analyzer software v3.0 and the Xenium Onboard Analysis pipeline v3.0.

Samples were either prepared following the Xenium In Situ Gene Expression protocol ("Xenium v1"; red points) or Xenium Prime In Situ Gene Expression protocol ("Xenium Prime 5K"; blue points), both with Cell Segmentation Staining. Error bars indicate the standard deviation of runtime across a broad range of samples used in testing.

On the x-axis, tissue area is plotted per run (~2.35 cm2 x 2 Xenium Slides) to demonstrate the trend. On the y-axis, grid lines are shown in 6 hour increments.

<Image src={"https://cdn.10xgenomics.com/image/upload/v1718996844/software-support/Xenium-onboard-analysis/XA_runtime_estimate.png"} />

</LinkableSection>

Estimating Run Time

Estimating Xenium Analyzer Instrument Run Time

<LinkableSection id={"metadata"} title={"Field descriptions"}>
The `experiment.xenium` file is an output from the Xenium Onboard Analysis pipeline. The metadata fields in this file are described below:

| Field | Description |
| ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `major_version` | Indicates major version of analysis output file formats read by Xenium Explorer |
| `minor_version` | Indicates minor version of analysis output file formats read by Xenium Explorer |
| `patch_version` | Patch version of analysis output file formats read by Xenium Explorer |
| `run_name` | User-specified Run Name entered on instrument |
| `run_start_time` | Instrument run start time |
| `region_name` | User-specified name for region selected on instrument |
| `preservation_method` | User-specified sample preservation method |
| `num_cells` | Cells detected by Xenium Onboard Analysis pipeline |
| `transcripts_per_cell` | Median transcripts per cell calculated by Xenium Onboard Analysis pipeline |
| `transcripts_per_100um` | Transcripts per 100 &micro;m2 calculated by Xenium Onboard Analysis pipeline |
| `region_area` | Total area of imaged field of views (FOVs) |
| `total_cell_area` | Summed area of detected cells |
| `thickness_of_high_quality_decoded_transcripts` | Width in Z of high-quality transcripts measured in microns |
| `non_zero_matrix_entries` | Number of nonzero entries in the sparse CSC cell-feature matrix in the `cell_feature_matrix.zarr.zip` file |
| `nuclear_transcripts_per_100um` | Number of high-quality, decoded-to-gene nuclear transcripts divided by the total segmented nuclear area |
| `num_transcripts` | Total number of transcripts including low-quality transcripts |
| `num_transcripts_high_quality` | Total high quality decoded transcripts (≥ Q20) |
| `fraction_transcripts_assigned` | Percent of high-quality transcripts that are found within cells |
| `segmented_cell_stain_frac` | Sum of `segmented_cell_boundary_frac` and `segmented_cell_interior_frac` |
| `segmented_cell_boundary_frac` | Number and percent of cells where the cell segmentation boundary is derived from the boundary stain |
| `segmented_cell_interior_frac` | Number and percent of cells where the boundary is from expansion of the nucleus using interior stain information |
| `segmented_cell_nuc_expansion_frac` | Number and percent of cells where the boundary is an isotropic expansion from the nucleus boundary |
| `imported_cell_frac` | Fraction of cells that were imported by Xenium Ranger `import-segmentation` |
| `cassette_name` | User-specified Xenium Cassette Name entered on instrument |
| `slide_id` | User-specified Xenium Slide ID entered on instrument |
| `panel_type` | Panel type specified by the `gene_panel.json` file ("Predesigned", "custom", or "add_on") |
| `panel_design_id` | Panel design ID specified by panel selection on instrument (additionally `panel_predesigned_id` is included for add-on custom panel designs) |
| `panel_designer_version` | Xenium Panel Designer software version used for the custom panel design |
| `panel_name` | Panel name specified by panel selection on instrument |
| `panel_organism` | Sample organism specified by selected gene panel |
| `panel_tissue_type` | User-specified tissue type selected on instrument |
| `panel_num_targets_predesigned` | Number of gene targets from the pre-designed gene panel |
| `panel_num_targets_custom` | Number of gene targets from add-on custom panel if included in panel design |
| `chemistry_version` | Assay chemistry specified by the `gene_panel.json` file ("v1" = Xenium v1, "v2" = Xenium Prime) |
| `pixel_size` | Pixel size in the `morphology.ome.tif` image file (in &micro;m) |
| `instrument_sn` | Xenium Analyzer instrument serial number |
| `instrument_sw_version` | Version of the Xenium Analyzer firmware used during analysis run |
| `analysis_sw_version` | Version of Xenium Onboard Analysis pipeline used to analyze data |
| `analysis_uuid` | Instrument metadata |
| `experiment_uuid` | Instrument metadata |
| `cassette_uuid` | Instrument metadata |
| `roi_uuid` | Instrument metadata |
| `z_step_size` | Z-step size (in &micro;m) used for [subsampling](/support/software/xenium-onboard-analysis/3.1/algorithms-overview/image-processing#dapi) the `morphology.ome.tif` image Z-stacks |
| `well_uuid` | Instrument metadata |
| `calibration_uuid` | Instrument metadata |
| `images` | Specifies the file paths to the morphology image files; used by Xenium Explorer to find input files |
| `xenium_explorer_files` | Specifies the file paths to transcript, cell, secondary analysis, and analysis summary files; used by Xenium Explorer to find input files |
| `segmentation_stain` | Specifies the stain method selected on instrument for cell segmentation ("Xenium Multi-Tissue Stain" or "Nuclei (DAPI)"). |
| `xenium_ranger` | If the data was reanalyzed with Xenium Ranger, this section specifies the `run_id`, Xenium Ranger version, and commands used to analyze the data. |

</LinkableSection>

<LinkableSection id={"expt-file-versions"} title={"Experiment file versions"}>

The `major_version` field in this file indicates when there are changes to Xenium Onboard Analysis (XOA) output file formats that affect forwards-compatibility with other software.

Updates to `major_version` are associated with new releases of the XOA software on the Xenium Analyzer (XA) instrument. This format version is used by Xenium Explorer (XE) and Xenium Ranger (XR) to determine input file compatibility across the Xenium software suite. This table summarizes when file format major versions were introduced by XOA software version.

| XOA version | `experiment.xenium` major version |
| --------------- | --------------------------------- |
| XOA v3.1 | 5.1 † |
| XOA v3.0 | 5.0 |
| XOA v2.0 | 4.0 |
| XOA v1.7 - v1.9 | 3.0 |
| XOA v1.6 | 2.0.2 |
| XOA v1.5 | 2.0.1 |
| XOA v1.3 - v1.4 | 2.0 |
| XOA v1.2 | 1.1 |
| XOA v1.0 - v1.1 | (no version fields) |

† Reanalysis with Xenium Ranger v3.0 generates outputs that align with the XA/XOA v3.1 output format and `experiment.xenium` major version v5.1.

</LinkableSection>

Experiment File

Xenium Experiment File

On this page, we provide some example code from a few commonly used software packages to help you start exploring several Xenium Onboard Analysis output file formats. Read this [Analysis Guide](/analysis-guides/continuing-your-journey-after-xenium-analyzer) for an introduction to a variety of analysis tools for data exploration and downstream analysis.

<SupportCallout level={"info"}>
<Text as={"span"}>10x Genomics does not provide support for community-developed tools and makes no guarantees regarding their function or performance. Please contact tool developers with any questions.</Text>
</SupportCallout>

<LinkableSection id={"view-imgs"} title={"Multi-file OME-TIFF images"}>

[Multi-file OME-TIFF images](/support/software/xenium-onboard-analysis/3.1/analysis/xoa-output-understanding-outputs#images) can be viewed with the commonly used Python package, [Tifffile](https://pypi.org/project/tifffile). However, it does not open downsampled multi-file images by default. Downsampled images may be useful for converting to other image file types or for viewing the image if the full resolution is too large. The code snippet below illustrates how to open either full or downsampled resolution images.

To open all of the multi-tissue stain images together, all four `morphology_focus_xxxx.ome.tif` files must be present in the `morphology_focus/` directory.

```python language="python"
# Import Python libraries
# This code uses python v3.12.0, tifffile v2023.9.26, matplotlib v3.8.2
import tifffile
import matplotlib.pyplot as plt

# Option 1: Load full resolution image channels
# The following may produce a warning: 'OME series cannot read multi-file pyramids'. This is because tifffile does not support loading a pyramidal multi-file OME-TIFF file. Only the full resolution (level=0) data will load for all channels in the directory.
fullres_multich_img = tifffile.imread(
"morphology_focus/morphology_focus_0000.ome.tif", is_ome=True, level=0, aszarr=False)

# Examine shape of array (number of channels, height, width), e.g. (4, 40867, 31318)
fullres_multich_img.shape

# Extract number of channels, e.g. 4
n_ch = fullres_multich_img.shape[0]

# Plot each channel
fig, axes = plt.subplots(ncols=n_ch, nrows=1, squeeze=False)
for i in range(n_ch):
 axes[0, i].imshow(fullres_multich_img[i], cmap="gray")
 axes[0, i].set_title(f"Channel: {i}")
plt.savefig('tifffile_fullres_four_channels.png')

# Option 2: Load a single channel image at any resolution, e.g., level=0 or level=1. Note 'is_ome' is set to False.
# Load one of the multi-file OME-TIFF files as a regular TIFF file at full resolution.
fullres_img_tiff = tifffile.imread(
 "morphology_focus/morphology_focus_0000.ome.tif", is_ome=False, level=0)

# Now load the file at downsampled resolution
downsampled_img = tifffile.imread(
 "morphology_focus/morphology_focus_0000.ome.tif", is_ome=False, level=1)
# Examine shape of array (number of channels, height, width)
downsampled_img.shape

# Plot the full resolution and downsampled images side-by-side
fig, axes = plt.subplots(ncols=2, nrows=1, squeeze=False)
axes[0, 0].imshow(fullres_img_tiff, cmap="gray")
axes[0, 0].set_title(f"Full resolution: {fullres_img_tiff.shape}")
axes[0, 1].imshow(downsampled_img, cmap="gray")
axes[0, 1].set_title(f"Downsampled: {downsampled_img.shape}")
plt.savefig('example_fullres_downsample.png')
```

</LinkableSection>

<LinkableSection id={"parquet"} title={"Parquet files"}>

Some Xenium Onboard Analysis output files are provided in the [Parquet](https://parquet.apache.org/docs/file-format) file format to enable faster loading and reading of data. The code below can be used to explore any of the XOA [cell Parquet files](/support/software/xenium-onboard-analysis/3.1/analysis/xoa-output-understanding-outputs#segmentation) (`cell_boundaries.parquet`, `cells.parquet`, and `nucleus_boundaries.parquet`) or [transcript Parquet file](/support/software/xenium-onboard-analysis/3.1/analysis/xoa-output-understanding-outputs#transcript-file) (`transcripts.parquet`).

**We highly recommend working with the Parquet file format itself.** However, if the CSV format is needed, the examples below demonstrate how to use either Python or R packages to view and convert the `transcripts.parquet` file to CSV format. For larger datasets, you may encounter memory limits for reading and converting the data to CSV. In these situations, we suggest writing the CSV file in chunks.

The example for Python uses [dask](https://docs.dask.org/en/latest/install.html). See the documentation for installation and usage guidance.

```python language="python"
# Import Python libraries
# Examples tested with python v3.12.0, dask v2024.5.2
import dask.dataframe as dd

# Read in the parquet file, edit path to where parquet file saved
df = dd.read_parquet('path/to/your/transcripts.parquet')
# Print information about the data frame
df.info()
# Print first 5 rows of the dask data frame
df.head()

# Optional: convert parquet data frame to CSV, edit path and output name for new file
# dask automatically figures out chunk size and saves each as a CSV file in the directory path specified below
df.to_csv('path/to/your/transcripts_csv_directory', index=False)
```

The example for R uses the [arrow](https://arrow.apache.org/docs/r) package. See the documentation for installation and usage guidance.

A note on installation:

- For Windows or Linux, install in R with `install.packages('arrow')`
- The arrow v16.1.0 package for [MacOS on CRAN is not feature complete](https://github.com/apache/arrow/issues/41952) as of this writing (June 2024), please use conda to install: `conda create -n r -c conda-forge r-base r-arrow`

```r language="r"
# For Windows or Linux, install arrow with this command:
# install.packages('arrow')
# If not already installed, install 'data.table' as well.

# Import R package
library(arrow)

# Path to your parquet file, edit path to where parquet file saved
PATH <- 'transcripts.parquet'
# Edit path and output name for new file
OUTPUT <- gsub('\\.parquet$', '.csv', PATH)
# Specify chunk size
CHUNK_SIZE <- 1e6

# Read in the parquet file
# This reads in the data as an arrow table
parquet_file <- arrow::read_parquet(PATH, as_data_frame = FALSE)
# To read in the table as a tibble, set data frame to true:
# parquet_file <- arrow::read_parquet(PATH, as_data_frame = TRUE)

# Optional: convert parquet data frame to CSV.
start <- 0
while(start < parquet_file$num_rows) {
 end <- min(start + CHUNK_SIZE, parquet_file$num_rows)
 chunk <- as.data.frame(parquet_file$Slice(start, end - start))
 data.table::fwrite(chunk, OUTPUT, append = start != 0)
 start <- end
}
if(require('R.utils', quietly = TRUE)) {
 R.utils::gzip(OUTPUT)
}
```

</LinkableSection>

<LinkableSection id={"json"} title={"Panel JSON file"}>

Python or other tools can be used to parse the [gene panel JSON file](/support/software/xenium-onboard-analysis/3.1/analysis/xoa-output-understanding-outputs#panel). Here is example Python code to extract gene name, Ensembl ID, and gene coverage information for each gene target in a given panel:

```python language="python"
# Import Python libraries
# Example with python v3.12, pandas v2.1.1
import json
import pandas as pd

# Open JSON file
f = open('gene_panel.json') # Edit file name here

# Return JSON object as a dictionary
data = json.load(f)

# Create lists to store extracted information
gene = []
ensembl = []
cov = []
# Iterate through the JSON list to extract information
for i in data['payload']['targets']:
 if (i['type']['descriptor'] == "gene"): # Only collect info for genes, not controls
 gene_name = i['type']['data']['name']
 ensembl_id = i['type']['data']['id']
 coverage = str(i['info']['gene_coverage'])

 gene.append(gene_name)
 ensembl.append(ensembl_id)
 cov.append(coverage)

# Create output CSV file
out_df = pd.DataFrame(list(zip(gene, ensembl, cov)), columns=['Gene name', 'Ensembl ID', 'Gene coverage'])
out_df.to_csv('my_panel_gene_info.csv', index=False)

# Close file
f.close()
```

</LinkableSection>

<LinkableSection id={"zarr"} title={"Zarr files"}>

The `zarr` Python library documentation and tutorials are available [here](https://zarr.readthedocs.io/en/stable/tutorial.html). These code snippets below show how to read a Zarr array into `numpy` [N-dimensional arrays](https://numpy.org/doc/stable/reference/arrays.ndarray.html). This code can be used to explore any of the XOA [Zarr output files](/support/software/xenium-onboard-analysis/3.1/advanced/xoa-output-zarr): `cells.zarr.zip`, `analysis.zarr.zip`, `cell_feature_matrix.zarr.zip`, and `transcripts.zarr.zip`.

In this first example, we read in the `cells.zarr.zip` file and take a look at the file's structure:

```python language="python"
# Import Python libraries
# This script was tested with zarr v2.13.6
import zarr
import numpy as np

# Function to open a Zarr file
def open_zarr(path: str) -> zarr.Group:
 store = (zarr.ZipStore(path, mode="r")
 if path.endswith(".zip")
 else zarr.DirectoryStore(path)
 )
 return zarr.group(store=store)

# For example, use the above function to open the cells Zarr file, which contains segmentation mask Zarr arrays
root = open_zarr("cells.zarr.zip")

# Look at group array info and structure
root.info
root.tree() # shows structure, array dimensions, data types

# Create cell and nucleus segmentation mask np array objects to read or modify
cellseg_mask = np.array(root["masks"][1])
nucseg_mask = np.array(root["masks"][0])

# Show dimensions of the 2D segmentation mask arrays (also shown in .tree())
# .ndim() shows number of dimensions
# The shape should match the number of pixels in the morphology image.
cellseg_mask.shape
nucseg_mask.shape

# Show max value of cells in the masks (value=0 are background pixels)
# The .max() method counts all the values that are not 0, which should equal
# the total cells detected in the dataset (reported in e.g., analysis_summary.html
# summary tab metric).
cellseg_mask.max()
nucseg_mask.max()

# Examples for exploring file contents
# How to show array
root["masks"][0][0:9] # or root["masks/0"]
root["cell_summary"][0:9]
# How to show attribute values
root.attrs["major_version"]
root.attrs["segmentation_methods"]
# How to list out attribute names and values
dict(root.attrs.items())
dict(root['cell_summary'].attrs.items())
```

Using the same Python function as above to read in the file, here are a few example lines to view the `analysis.zarr.zip` and `transcripts.zarr.zip` arrays and attributes:

```python language="python"
# Read in secondary analysis Zarr arrays
root = open_zarr("analysis.zarr.zip")
# Examples for exploring file contents
# How to show a slice of the clustering_index arrays
root["cell_groups"][0]["indices"][0:9]
# How to show attributes
root["cell_groups"].attrs["group_names"]


# Read in transcripts Zarr arrays
root = open_zarr("transcripts.zarr.zip")
# Examples for exploring file contents
# How to show array info
root['grids'][0]['0,0']['gene_identity'].shape
root['grids'][0]['0,0']['quality_score'][0:9]
root['grids'][0]['0,0']['location'][0:9,]
# How to show array attributes
root.attrs['major_version']
root['density']['gene'].attrs['gene_names'][0:9]
```

</LinkableSection>

Code examples

Viewing Output Files with Example Code

Overview

Getting Started

Analysis

Algorithms

Example Datasets

Release Notes

Advanced

Xenium Analyzer

Xenium Explorer

Xenium Ranger

Analysis Guides

Xenium Onboard Analysis 3.1

Xenium Onboard Analysis

A table of Xenium onboard analysis pipeline output files.

Outputs at a Glance

Overview of all Xenium onboard analysis pipeline output files.

Understanding Xenium Outputs

Overview of the interactive Xenium analysis summary output file metrics and plots.

Analysis Summary Overview

Troubleshooting guide for the Xenium analysis summary alerts.

Analysis Summary Alerts

Storing Xenium raw data for reanalysis or grant requirements.

Archiving Xenium Data

The Xenium In Situ software suite is a set of software applications for analyzing and visualizing in situ gene expression data produced by the Xenium Analyzer. The Xenium Onboard Analysis pipeline enables simultaneous on-instrument data collection and analysis.

What is Xenium Onboard Analysis?

Learn about Xenium Onboard Analysis outputs, algorithms, and software compatibility.

Getting Started with Xenium Onboard Analysis

New to Xenium Onboard Analysis?

View the latest updates to Xenium Onboard Analysis algorithms.

View the latest updates to Xenium Onboard Analysis.

Powerful in situ profiling platform using curated panels, powerful visualization software, and easy-to-follow workflow.

Visualize data from Xenium In Situ solutions.

Reanalyze data from Xenium In Situ solutions.

Browse Q&As from the 10x Genomics support team.

A collection of introductions, tutorials, and blogs for data analysis beyond 10x Genomics software.