cellxgenedp 1.0.1
This package is available in Bioconductor version 3.15 and later. The following code installs cellxgenedp as well as other packages required for this vignette.
pkgs <- c("cellxgenedp", "zellkonverter", "SingleCellExperiment", "HDF5Array")
required_pkgs <- pkgs[!pkgs %in% rownames(installed.packages())]
BiocManager::install(required_pkgs)Use the following pkgs vector to install from GitHub (latest,
unchecked, development version) instead
pkgs <- c(
"mtmorgan/cellxgenedp", "zellkonverter", "SingleCellExperiment", "HDF5Array"
)Load the package into your current R session. We make extensive use of the dplyr packages, and at the end of the vignette use SingleCellExperiment and zellkonverter, so load those as well.
suppressPackageStartupMessages({
library(zellkonverter)
library(SingleCellExperiment) # load early to avoid masking dplyr::count()
library(dplyr)
library(cellxgenedp)
})cxg() Provides a ‘shiny’ interfaceThe following sections outline how to use the cellxgenedp package
in an R script; most functionality is also available in the cxg()
shiny application, providing an easy way to identify, download, and
visualize one or several datasets. Start the app
cxg()choose a project on the first tab, and a dataset for visualization, or one or more datasets for download!
Retrieve metadata about resources available at the cellxgene data
portal using db():
db <- db()Printing the db object provides a brief overview of the available
data, as well as hints, in the form of functions like collections(),
for further exploration.
db## cellxgene_db
## number of collections(): 88
## number of datasets(): 486
## number of files(): 1454The portal organizes data hierarchically, with ‘collections’ (research studies, approximately), ‘datasets’, and ‘files’. Discover data using the corresponding functions.
collections(db)## # A tibble: 88 × 16
## collec…¹ acces…² conta…³ conta…⁴ curat…⁵ data_…⁶ descr…⁷ genes…⁸ links name
## <chr> <chr> <chr> <chr> <chr> <chr> <chr> <lgl> <list> <chr>
## 1 6e8c541… READ a.vano… Alexan… Jennif… 2.0 A Sing… NA <list> Mura…
## 2 03f821b… READ km16@s… Kersti… Batuha… 2.0 It is … NA <list> Loca…
## 3 3472f32… READ wongcb… Raymon… Batuha… 2.0 The re… NA <list> A si…
## 4 83ed3be… READ tom.ta… Tom Ta… Jennif… 2.0 During… NA <list> Inte…
## 5 2a79d19… READ neal.r… Neal G… Jennif… 2.0 Single… NA <list> Sing…
## 6 92fde06… READ c.nove… Claudi… Jason … 2.0 The ce… NA <list> A co…
## 7 eb735cc… READ rv4@sa… Roser … Batuha… 2.0 Human … NA <list> Samp…
## 8 e75342a… READ nhuebn… Norber… Jennif… 2.0 Pathog… NA <list> Path…
## 9 125eef5… READ my4@sa… Matthe… Jason … 2.0 Unders… NA <list> Sing…
## 10 48d354f… READ Nathan… Nathan… Jennif… 2.0 We gen… NA <list> Lung…
## # … with 78 more rows, 6 more variables: publisher_metadata <list>,
## # visibility <chr>, created_at <date>, published_at <date>,
## # revised_at <date>, updated_at <date>, and abbreviated variable names
## # ¹collection_id, ²access_type, ³contact_email, ⁴contact_name, ⁵curator_name,
## # ⁶data_submission_policy_version, ⁷description, ⁸genesetsdatasets(db)## # A tibble: 486 × 27
## dataset_id colle…¹ assay cell_…² cell_…³ datas…⁴ devel…⁵ disease ethni…⁶
## <chr> <chr> <list> <int> <list> <chr> <list> <list> <list>
## 1 b07e5164-baf6… 6e8c54… <list> 2126 <list> https:… <list> <list> <list>
## 2 edc8d3fe-153c… 03f821… <list> 236977 <list> https:… <list> <list> <list>
## 3 2a498ace-872a… 03f821… <list> 422220 <list> https:… <list> <list> <list>
## 4 d5c67a4e-a8d9… 3472f3… <list> 19694 <list> https:… <list> <list> <list>
## 5 11ff73e8-d3e4… 83ed3b… <list> 71732 <list> https:… <list> <list> <list>
## 6 030faa69-ff79… 2a79d1… <list> 77650 <list> https:… <list> <list> <list>
## 7 42bb7f78-cef8… 92fde0… <list> 141401 <list> https:… <list> <list> <list>
## 8 c2a461b1-0c15… eb735c… <list> 97499 <list> https:… <list> <list> <list>
## 9 1252c5fb-945f… e75342… <list> 170281 <list> https:… <list> <list> <list>
## 10 bdf69f8d-5a96… e75342… <list> 2576 <list> https:… <list> <list> <list>
## # … with 476 more rows, 18 more variables: is_primary_data <chr>,
## # is_valid <lgl>, linked_genesets <lgl>, mean_genes_per_cell <dbl>,
## # name <chr>, organism <list>, processing_status <list>, published <lgl>,
## # revision <int>, schema_version <chr>, sex <list>, tissue <list>,
## # tombstone <lgl>, x_normalization <chr>, created_at <date>,
## # published_at <date>, revised_at <date>, updated_at <date>, and abbreviated
## # variable names ¹collection_id, ²cell_count, ³cell_type, …files(db)## # A tibble: 1,454 × 8
## file_id datas…¹ filen…² filet…³ s3_uri user_…⁴ created_at updated_at
## <chr> <chr> <chr> <chr> <chr> <lgl> <date> <date>
## 1 8fa6b88d-077b-4… b07e51… explor… CXG s3://… TRUE 2021-10-27 2021-10-27
## 2 90bedf16-31d7-4… b07e51… local.… H5AD s3://… TRUE 2021-10-27 2021-10-27
## 3 39b5c80f-c303-4… b07e51… local.… RDS s3://… TRUE 2021-10-27 2021-12-21
## 4 1e788c65-3422-4… edc8d3… local.… H5AD s3://… TRUE 2022-08-22 2022-08-23
## 5 24d08b4a-7314-4… edc8d3… local.… RDS s3://… TRUE 2022-08-22 2022-08-23
## 6 88e0bdc6-8d3c-4… edc8d3… explor… CXG s3://… TRUE 2022-08-22 2022-08-23
## 7 f93c1a77-ab7a-4… 2a498a… local.… H5AD s3://… TRUE 2022-08-22 2022-08-23
## 8 d17fddad-4907-4… 2a498a… local.… RDS s3://… TRUE 2022-08-22 2022-08-23
## 9 f68fa37e-fb6f-4… 2a498a… explor… CXG s3://… TRUE 2022-08-22 2022-08-23
## 10 0d143085-cf3f-4… d5c67a… local.… H5AD s3://… TRUE 2022-08-16 2022-08-16
## # … with 1,444 more rows, and abbreviated variable names ¹dataset_id,
## # ²filename, ³filetype, ⁴user_submittedEach of these resources has a unique primary identifier (e.g.,
file_id) as well as an identifier describing the relationship of the
resource to other components of the database (e.g.,
dataset_id). These identifiers can be used to ‘join’ information
across tables.
facets() provides information on ‘levels’ present in specific columnsNotice that some columns are ‘lists’ rather than atomic vectors like ‘character’ or ‘integer’.
datasets(db) |>
select(where(is.list))## # A tibble: 486 × 9
## assay cell_…¹ devel…² disease ethni…³ organ…⁴ processing…⁵ sex tissue
## <list> <list> <list> <list> <list> <list> <list> <list> <list>
## 1 <list [1]> <list> <list> <list> <list> <list> <named list> <list> <list>
## 2 <list [2]> <list> <list> <list> <list> <list> <named list> <list> <list>
## 3 <list [1]> <list> <list> <list> <list> <list> <named list> <list> <list>
## 4 <list [1]> <list> <list> <list> <list> <list> <named list> <list> <list>
## 5 <list [1]> <list> <list> <list> <list> <list> <named list> <list> <list>
## 6 <list [1]> <list> <list> <list> <list> <list> <named list> <list> <list>
## 7 <list [5]> <list> <list> <list> <list> <list> <named list> <list> <list>
## 8 <list [1]> <list> <list> <list> <list> <list> <named list> <list> <list>
## 9 <list [2]> <list> <list> <list> <list> <list> <named list> <list> <list>
## 10 <list [2]> <list> <list> <list> <list> <list> <named list> <list> <list>
## # … with 476 more rows, and abbreviated variable names ¹cell_type,
## # ²development_stage, ³ethnicity, ⁴organism, ⁵processing_statusThis indicates that at least some of the datasets had more than one
type of assay, cell_type, etc. The facets() function provides a
convenient way of discovering possible levels of each column, e.g.,
assay, organism, ethnicity, or sex, and the number of datasets with
each label.
facets(db, "assay")## # A tibble: 24 × 4
## facet label ontology_term_id n
## <chr> <chr> <chr> <int>
## 1 assay 10x 3' v3 EFO:0009922 172
## 2 assay 10x 3' v2 EFO:0009899 137
## 3 assay Slide-seq EFO:0009920 129
## 4 assay Visium Spatial Gene Expression EFO:0010961 35
## 5 assay Smart-seq2 EFO:0008931 32
## 6 assay 10x technology EFO:0008995 30
## 7 assay 10x 5' v1 EFO:0011025 20
## 8 assay 10x 3' transcription profiling EFO:0030003 10
## 9 assay 10x 5' v2 EFO:0009900 8
## 10 assay Patch-seq EFO:0008853 7
## # … with 14 more rowsfacets(db, "ethnicity")## # A tibble: 18 × 4
## facet label ontol…¹ n
## <chr> <chr> <chr> <int>
## 1 ethnicity European HANCES… 173
## 2 ethnicity unknown unknown 172
## 3 ethnicity na na 164
## 4 ethnicity Asian HANCES… 55
## 5 ethnicity African American HANCES… 34
## 6 ethnicity admixed ancestry HANCES… 24
## 7 ethnicity Greater Middle Eastern (Middle Eastern, North Afric… HANCES… 21
## 8 ethnicity Hispanic or Latin American HANCES… 15
## 9 ethnicity African American or Afro-Caribbean HANCES… 5
## 10 ethnicity East Asian HANCES… 4
## 11 ethnicity African HANCES… 3
## 12 ethnicity South Asian HANCES… 2
## 13 ethnicity Chinese HANCES… 1
## 14 ethnicity Eskimo HANCES… 1
## 15 ethnicity Finnish HANCES… 1
## 16 ethnicity Han Chinese HANCES… 1
## 17 ethnicity Oceanian HANCES… 1
## 18 ethnicity Pacific Islander HANCES… 1
## # … with abbreviated variable name ¹ontology_term_idfacets(db, "sex")## # A tibble: 3 × 4
## facet label ontology_term_id n
## <chr> <chr> <chr> <int>
## 1 sex male PATO:0000384 388
## 2 sex female PATO:0000383 266
## 3 sex unknown unknown 47Suppose we were interested in finding datasets from the 10x 3’ v3
assay (ontology_term_id of EFO:0009922) containing individuals of
African American ethnicity, and female sex. Use the facets_filter()
utility function to filter data sets as needed
african_american_female <-
datasets(db) |>
filter(
facets_filter(assay, "ontology_term_id", "EFO:0009922"),
facets_filter(ethnicity, "label", "African American"),
facets_filter(sex, "label", "female")
)There are 21 datasets satisfying our criteria. It looks like there are up to
african_american_female |>
summarise(total_cell_count = sum(cell_count))## # A tibble: 1 × 1
## total_cell_count
## <int>
## 1 2608650cells sequenced (each dataset may contain cells from several
ethnicities, as well as males or individuals of unknown gender, so we
do not know the actual number of cells available without downloading
files). Use left_join to identify the corresponding collections:
## collections
left_join(
african_american_female |> select(collection_id) |> distinct(),
collections(db),
by = "collection_id"
)## # A tibble: 7 × 16
## collect…¹ acces…² conta…³ conta…⁴ curat…⁵ data_…⁶ descr…⁷ genes…⁸ links name
## <chr> <chr> <chr> <chr> <chr> <chr> <chr> <lgl> <list> <chr>
## 1 c9706a92… READ hnaksh… Harikr… Jennif… 2.0 "Singl… NA <list> A si…
## 2 2f75d249… READ rsatij… Rahul … Jennif… 2.0 "This … NA <list> Azim…
## 3 bcb61471… READ info@k… KPMP Jennif… 2.0 "Under… NA <list> An a…
## 4 b9fc3d70… READ bruce.… Bruce … Jennif… 2.0 "Numer… NA <list> A We…
## 5 62e8f058… READ chanj3… Joseph… Jennif… 2.0 "155,0… NA <list> HTAN…
## 6 625f6bf4… READ a5wang… Allen … Jennif… 2.0 "Large… NA <list> Lung…
## 7 b953c942… READ icobos… Inma C… Jennif… 2.0 "Tau a… NA <list> Sing…
## # … with 6 more variables: publisher_metadata <list>, visibility <chr>,
## # created_at <date>, published_at <date>, revised_at <date>,
## # updated_at <date>, and abbreviated variable names ¹collection_id,
## # ²access_type, ³contact_email, ⁴contact_name, ⁵curator_name,
## # ⁶data_submission_policy_version, ⁷description, ⁸genesetscellxgeneDiscover files associated with our first selected dataset
selected_files <-
left_join(
african_american_female |> select(dataset_id),
files(db),
by = "dataset_id"
)
selected_files## # A tibble: 63 × 8
## dataset_id file_id filen…¹ filet…² s3_uri user_…³ created_at updated_at
## <chr> <chr> <chr> <chr> <chr> <lgl> <date> <date>
## 1 de985818-285f-4… d5c6de… local.… H5AD s3://… TRUE 2021-09-24 2021-09-24
## 2 de985818-285f-4… a0bcae… local.… RDS s3://… TRUE 2021-09-24 2021-12-21
## 3 de985818-285f-4… dff842… explor… CXG s3://… TRUE 2021-09-24 2021-09-24
## 4 f72958f5-7f42-4… 2e7374… local.… RDS s3://… TRUE 2021-10-07 2021-12-21
## 5 f72958f5-7f42-4… bbce34… explor… CXG s3://… TRUE 2021-10-07 2021-10-07
## 6 f72958f5-7f42-4… 091323… local.… H5AD s3://… TRUE 2021-10-07 2021-10-07
## 7 07854d9c-5375-4… f6f812… local.… H5AD s3://… TRUE 2022-02-15 2022-02-15
## 8 07854d9c-5375-4… 7ae7df… explor… CXG s3://… TRUE 2022-02-15 2022-02-15
## 9 07854d9c-5375-4… 3f995f… local.… RDS s3://… TRUE 2022-02-15 2022-02-15
## 10 0b75c598-0893-4… 38eaca… local.… H5AD s3://… TRUE 2022-02-15 2022-02-15
## # … with 53 more rows, and abbreviated variable names ¹filename, ²filetype,
## # ³user_submittedThe filetype column lists the type of each file. The cellxgene service
can be used to visualize datasets that have CXG files.
selected_files |>
filter(filetype == "CXG") |>
slice(1) |> # visualize a single dataset
datasets_visualize()Visualization is an interactive process, so datasets_visualize()
will only open up to 5 browser tabs per call.
Datasets usually contain CXG (cellxgene visualization), H5AD
(files produced by the python AnnData module), and Rds (serialized
files produced by the R Seurat package). There are no public parsers
for CXG, and the Rds files may be unreadable if the version of
Seurat used to create the file is different from the version used to
read the file. We therefore focus on the H5AD files. For
illustration, we download one of our selected files.
local_file <-
selected_files |>
filter(
dataset_id == "3de0ad6d-4378-4f62-b37b-ec0b75a50d94",
filetype == "H5AD"
) |>
files_download(dry.run = FALSE)
basename(local_file)## [1] "f4065ffa-c2d6-45bf-b596-5a69e36d8fcd.H5AD"These are downloaded to a local cache (use the internal function
cellxgenedp:::.cellxgenedb_cache_path() for the location of the
cache), so the process is only time-consuming the first time.
H5AD files can be converted to R / Bioconductor objects using
the zellkonverter package.
h5ad <- readH5AD(local_file, reader = "R", use_hdf5 = TRUE)
h5ad## class: SingleCellExperiment
## dim: 26329 46500
## metadata(5): X_normalization cell_type_ontology_term_id_colors
## layer_descriptions schema_version title
## assays(1): X
## rownames: NULL
## rowData names(5): feature_biotype feature_id feature_is_filtered
## feature_name feature_reference
## colnames(46500): D032_AACAAGACAGCCCACA D032_AACAGGGGTCCAGCGT ...
## D231_CGAGTGCTCAACCCGG D231_TTCGCTGAGGAACATT
## colData names(25): assay assay_ontology_term_id ... tissue
## tissue_ontology_term_id
## reducedDimNames(1): X_umap
## mainExpName: NULL
## altExpNames(0):The SingleCellExperiment object is a matrix-like object with rows
corresponding to genes and columns to cells. Thus we can easily
explore the cells present in the data.
h5ad |>
colData(h5ad) |>
as_tibble() |>
count(sex, donor_id)## # A tibble: 9 × 3
## sex donor_id n
## <fct> <fct> <int>
## 1 female D088 5903
## 2 female D139 5217
## 3 female D175 1778
## 4 female D231 4680
## 5 male D032 4970
## 6 male D046 8894
## 7 male D062 4852
## 8 male D122 3935
## 9 male D150 6271The Orchestrating Single-Cell Analysis with Bioconductor
online resource provides an excellent introduction to analysis and
visualization of single-cell data in R / Bioconductor. Extensive
opportunities for working with AnnData objects in R but using the
native python interface are briefly described in, e.g., ?AnnData2SCE
help page of zellkonverter.
The hca package provides programmatic access to the Human Cell Atlas data portal, allowing retrieval of primary as well as derived single-cell data files.
## R version 4.2.1 (2022-06-23)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 20.04.4 LTS
##
## Matrix products: default
## BLAS: /home/biocbuild/bbs-3.15-bioc/R/lib/libRblas.so
## LAPACK: /home/biocbuild/bbs-3.15-bioc/R/lib/libRlapack.so
##
## locale:
## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
## [3] LC_TIME=en_GB LC_COLLATE=C
## [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
## [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
## [9] LC_ADDRESS=C LC_TELEPHONE=C
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
##
## attached base packages:
## [1] stats4 stats graphics grDevices utils datasets methods
## [8] base
##
## other attached packages:
## [1] cellxgenedp_1.0.1 dplyr_1.0.9
## [3] SingleCellExperiment_1.18.0 SummarizedExperiment_1.26.1
## [5] Biobase_2.56.0 GenomicRanges_1.48.0
## [7] GenomeInfoDb_1.32.3 IRanges_2.30.1
## [9] S4Vectors_0.34.0 BiocGenerics_0.42.0
## [11] MatrixGenerics_1.8.1 matrixStats_0.62.0
## [13] zellkonverter_1.6.4 BiocStyle_2.24.0
##
## loaded via a namespace (and not attached):
## [1] httr_1.4.4 sass_0.4.2 jsonlite_1.8.0
## [4] bslib_0.4.0 shiny_1.7.2 assertthat_0.2.1
## [7] BiocManager_1.30.18 GenomeInfoDbData_1.2.8 yaml_2.3.5
## [10] pillar_1.8.1 lattice_0.20-45 glue_1.6.2
## [13] reticulate_1.25 digest_0.6.29 promises_1.2.0.1
## [16] XVector_0.36.0 htmltools_0.5.3 httpuv_1.6.5
## [19] Matrix_1.4-1 pkgconfig_2.0.3 dir.expiry_1.4.0
## [22] bookdown_0.28 zlibbioc_1.42.0 purrr_0.3.4
## [25] xtable_1.8-4 HDF5Array_1.24.2 later_1.3.0
## [28] tibble_3.1.8 generics_0.1.3 ellipsis_0.3.2
## [31] DT_0.24 cachem_1.0.6 cli_3.3.0
## [34] magrittr_2.0.3 crayon_1.5.1 mime_0.12
## [37] evaluate_0.16 fansi_1.0.3 tools_4.2.1
## [40] lifecycle_1.0.1 basilisk.utils_1.8.0 stringr_1.4.1
## [43] Rhdf5lib_1.18.2 DelayedArray_0.22.0 compiler_4.2.1
## [46] jquerylib_0.1.4 rlang_1.0.4 rhdf5_2.40.0
## [49] grid_4.2.1 RCurl_1.98-1.8 rhdf5filters_1.8.0
## [52] htmlwidgets_1.5.4 bitops_1.0-7 rmarkdown_2.15
## [55] basilisk_1.8.0 DBI_1.1.3 curl_4.3.2
## [58] R6_2.5.1 knitr_1.39 fastmap_1.1.0
## [61] utf8_1.2.2 filelock_1.0.2 stringi_1.7.8
## [64] parallel_4.2.1 Rcpp_1.0.9 vctrs_0.4.1
## [67] png_0.1-7 tidyselect_1.1.2 xfun_0.32