Query & analyze

import lamindb as ln
import lnschema_bionty as lb

lb.settings.species = "human"

💡 loaded instance: testuser1/test-flow (lamindb 0.54.1)

ln.track()

💡 notebook imports: anndata==0.9.2 lamindb==0.54.1 lnschema_bionty==0.31.2 scanpy==1.9.5

💡 Transform(id='wukchS8V976Uz8', name='Query & analyze', short_name='facs2', version='0', type=notebook, updated_at=2023-09-26 10:02:45, created_by_id='DzTjkKse')

💡 Run(id='13ARiS3AbtQpdKXb739B', run_at=2023-09-26 10:02:45, transform_id='wukchS8V976Uz8', created_by_id='DzTjkKse')

Inspect the CellMarker registry

Inspect your aggregated cell marker registry as a DataFrame:

lb.CellMarker.filter().df().head()

	name	synonyms	gene_symbol	ncbi_gene_id	uniprotkb_id	species_id	bionty_source_id	updated_at	created_by_id
id
c3dZKHFOdllB	CD33		CD33	945	P20138	uHJU	CgXV	2023-09-26 10:02:27	DzTjkKse
fpPkjlGv15C9	Ccr6		CCR6	1235	P51684	uHJU	CgXV	2023-09-26 10:02:27	DzTjkKse
YA5Ezh6SAy10	DNA1		None	None	None	uHJU	CgXV	2023-09-26 10:02:27	DzTjkKse
agQD0dEzuoNA	CXCR3		CXCR3	2833	P49682	uHJU	CgXV	2023-09-26 10:02:27	DzTjkKse
N2F6Qv9CxJch	CD11B		ITGAM	3684	P11215	uHJU	CgXV	2023-09-26 10:02:27	DzTjkKse

Search for a marker (synonyms aware):

lb.CellMarker.search("PD-1").head(2)

	id	synonyms	__ratio__
name
PD1	2VeZenLi2dj5	PID1|PD-1|PD 1	100.0
Cd14	roEbL8zuLC5k		50.0

Look up markers with auto-complete:

markers = lb.CellMarker.lookup()

markers.cd14

CellMarker(id='roEbL8zuLC5k', name='Cd14', synonyms='', gene_symbol='CD14', ncbi_gene_id='4695', uniprotkb_id='O43678', updated_at=2023-09-26 10:02:27, species_id='uHJU', bionty_source_id='CgXV', created_by_id='DzTjkKse')

Query files by markers

Query panels and datasets based on markers, e.g., which datasets have 'CD14' in the flow panel:

panels_with_cd14 = ln.FeatureSet.filter(cell_markers=markers.cd14).all()

ln.File.filter(feature_sets__in=panels_with_cd14).df()

	storage_id	key	suffix	accessor	description	version	size	hash	hash_type	transform_id	run_id	initial_version_id	updated_at	created_by_id
id
gfsTnxaVBSGvvuNky0Mm	oieE0n5L	None	.h5ad	AnnData	Flow cytometry file 2	None	6837528	t6plg-pXZMxqmQN9naNeuw	md5	SmQmhrhigFPLz8	b13zNeaKscxptfaMtKWi	None	2023-09-26 10:02:41	DzTjkKse
6iw33IVOetRxoZOczFal	oieE0n5L	None	.h5ad	AnnData	Alpert19	None	33369696	Piw2n0vdnoNoAV7ZxgsW-g	md5	OWuTtS4SAponz8	HDsxREvsdSEFMCUgXjYW	None	2023-09-26 10:02:32	DzTjkKse

Access registries:

features = ln.Feature.lookup()
efs = lb.ExperimentalFactor.lookup()
species = lb.Species.lookup()

Find shared cell markers between two files:

files = ln.File.filter(feature_sets__in=panels_with_cd14, species=species.human).list()
file1, file2 = files[0], files[1]

shared_markers = file1.features["var"] & file2.features["var"]
shared_markers.list("name")

['CD8', 'Cd19', 'CD3', 'CD127', 'CD57', 'CD28', 'Cd14', 'CD27', 'Ccr7', 'Cd4']

Concatenate & analyze queried files

Load files into memory and concatenate:

adata1 = file1.load()
adata2 = file2.load()

import anndata as ad

adata = ad.concat(
    [adata1, adata2],
    label="file",
    keys=[file1.description, file2.description],
)
adata

/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/anndata/_core/anndata.py:1838: UserWarning: Observation names are not unique. To make them unique, call `.obs_names_make_unique`.
  utils.warn_names_duplicates("obs")

AnnData object with n_obs × n_vars = 231130 × 10
    obs: 'file'

import scanpy as sc

sc.pp.pca(adata)
sc.pl.pca(adata, color=markers.cd14.name)

Register a sharded dataset

If we believe that we’ll need this dataset, again, we can register a sharded version:

dataset = ln.Dataset([file1, file2], name="Batch 1 and batch 2")
dataset.save()
dataset.view_flow()

# clean up test instance
!lamin delete --force test-flow
!rm -r test-flow

💡 deleting instance testuser1/test-flow
✅     deleted instance settings file: /home/runner/.lamin/instance--testuser1--test-flow.env

✅     instance cache deleted
✅     deleted '.lndb' sqlite file
❗     consider manually deleting your stored data: /home/runner/work/lamin-usecases/lamin-usecases/docs/test-flow