Support homeSpace RangerAnalysis
Feature Slice H5

Feature Slice H5

This is an HDF5 dataset containing Visium HD Spatial Gene Expression data stored in a manner to support easy and efficient fetching of 2 µm resolution "image" slices for a single gene or multiple genes. Data can be binned to arbitrary scales or plotted against the microscope tissue image.

An HDF5 dataset is often composed of groups, which are made of other groups or datasets. Datasets are raw byte arrays stored in binary compressed manner. Additionally, each group or dataset can have attributes associated with them.

There are five groups that make an hd_feature_slice.h5 dataset:

$ h5ls hd_feature_slice.h5 feature_slices Group features Group images Group masks Group umis Group

Each group is documented below.

This group contains datasets related to the feature (gene) names and ids in the reference transcriptome. It is identical to the features group stored in the HDF5 raw/filtered feature-barcode matrix output from the pipeline.

$ h5ls hd_feature_slice.h5/features _all_tag_keys Dataset {1} feature_type Dataset {32285} genome Dataset {32285} id Dataset {32285} name Dataset {32285} target_sets Group $ h5ls hd_feature_slice.h5/features/target_sets Visium\ Mouse\ Transcriptome\ Probe\ Set\ v2.0 Dataset

This group contains a group for the index of each feature (gene) listed in hd_feature_slice.h5/features/{id} where {id} is the index of the feature in the /features group. Only features that have at least one total UMI are stored here, i.e., if a specific feature is missing, that feature had no UMIs observed in this sample.

$ h5ls hd_feature_slice.h5/feature_slices 0 Group 10 Group 100 Group 1000 Group 10007 Group . . 999 Group 9990 Group 9991 Group

Each gene specific group contains the matrix row, col and data that compose the slice of the gene expression data for that gene.

$ h5ls hd_feature_slice.h5/feature_slices/0 col Dataset {48/Inf} data Dataset {48/Inf} row Dataset {48/Inf}

This group contains the total UMI spatial matrix organized similar to the per-gene group (hd_feature_slice.h5/feature_slices/index).

$ h5ls hd_feature_slice.h5/umis/total col Dataset {5451603/Inf} data Dataset {5451603/Inf} row Dataset {5451603/Inf}

This group contains the grayscale microscope and CytAssist images projected onto the grid of 2 µm squares.

$ h5ls hd_feature_slice.h5/images/microscope col Dataset {11785262/Inf} data Dataset {11785262/Inf} row Dataset {11785262/Inf}

This group contains a binary image mask marking bins under tissue. There are four different groups:

$ h5ls hd_feature_slice.h5/masks filtered Group square_008um Group square_020um Group square_050um Group

The filtered group corresponds to the raw resolution bins (square_002um).

Each of the groups again stores a matrix for the mask:

$ h5ls hd_feature_slice.h5/masks/square_008um col Dataset {391917/Inf} data Dataset {391917/Inf} row Dataset {391917/Inf}

Here is some example python code for how to bin a feature slice.

import h5py as h5 import numpy as np ROW_DATASET_NAME = "row" COL_DATASET_NAME = "col" DATA_DATASET_NAME = "data" METADATA_JSON_ATTR_NAME = "metadata_json" UMIS_GROUP_NAME = "umis" TOTAL_UMIS_GROUP_NAME = "total" class CooMatrix: row: list[int] col: list[int] data: list[int | float] @classmethod def from_hdf5(cls, group): return cls( row=group[ROW_DATASET_NAME][:], col=group[COL_DATASET_NAME][:], data=group[DATA_DATASET_NAME][:], ) def to_ndarray(self, nrows, ncols, binning_scale = 1): """Convert the COO matrix representation to a dense ndarray at the specified binning scale.""" ncols_binned = int(np.ceil(ncols / binning_scale)) nrows_binned = int(np.ceil(nrows / binning_scale)) result = np.zeros((nrows_binned, ncols_binned), dtype="int32") for row, col, data in zip(self.row, self.col, self.data): result[row // binning_scale, col // binning_scale] += data return result # Load total UMIs at 8um bin size with h5.File("hd_feature_slide.h5", "r") as h5_file: metadata = json.loads(h5_file.attrs[METADATA_JSON_ATTR_NAME]) umis_8um = CooMatrix.from_hdf5(h5_file[UMIS_GROUP_NAME][TOTAL_UMIS_GROUP_NAME]).to_ndarray( nrows=metadata["nrows"], ncols=metadata["ncols"], binning_scale=4 )

Each hd_feature_slice.h5 file also contains metadata in the following format that may be useful for example to translate between the original microscope image and the barcoded array space, or to obtain the full dimensions of the barcoded array:

$ h5dump -a /metadata_json hd_feature_slice.h5 HDF5 "hd_feature_slice.h5" { ATTRIBUTE "metadata_json" { DATATYPE H5T_STRING { STRSIZE H5T_VARIABLE; STRPAD H5T_STR_NULLTERM; CSET H5T_CSET_UTF8; CTYPE H5T_C_S1; } DATASPACE SCALAR DATA { (content reformatted below for readability) (0): { "sample_id": "10001_10002", "sample_desc": "SJ0118_SJ118-7-D_100_100_10_Brain_5_mouse_Brain_D", "slide_name": "visium_hd_rc1", "nrows": 3350, "ncols": 3350, "spot_pitch": 2.0, "hd_layout_json": { "slide_uid": "UNKNOWN", "file_format": "n/a", "aligner_version": "n/a", "input_hash": "n/a", "slide_design": "n/a", "transform": [ 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0 ] }, "transform_matrices": { "spot_colrow_to_microscope_colrow": [ [ 0.041985535298565406, -7.298152402766004, 25585.255831045426 ], [ 7.298152402766004, 0.041985535298565406, 2956.752264439149 ], [ 0.0, 0.0, 1.0 ] ], "microscope_colrow_to_spot_colrow": [ [ 0.000788241806459152, 0.13701644608939972, -425.29105551521997 ], [ -0.13701644608939972, 0.000788241806459152, 3503.2701905117615 ], [ 0.0, 0.0, 1.0 ] ], "spot_colrow_to_cytassist_colrow": null, "cytassist_colrow_to_spot_colrow": null } } } } }