mulitHIVE API

class multiHIVE.multiHIVE(adata: AnnData, n_genes: int, n_regions: int, n_proteins: int, n_latent: int = 20, n_hidden: int | None = None, n_layers_encoder: int = 2, n_layers_decoder: int = 2, dropout_rate: float = 0.1, region_factors: bool = True, use_batch_norm: Literal['encoder', 'decoder', 'none', 'both'] = 'both', use_layer_norm: Literal['encoder', 'decoder', 'none', 'both'] = 'none', gene_dispersion: Literal['gene', 'gene-batch', 'gene-label', 'gene-cell'] = 'gene', protein_dispersion: Literal['protein', 'protein-batch', 'protein-label'] = 'protein', gene_likelihood: Literal['zinb', 'nb'] = 'nb', latent_distribution: Literal['normal', 'ln'] = 'normal', empirical_protein_background_prior: bool | None = None, override_missing_proteins: bool = False, deeply_inject_covariates: bool = False, encode_covariates: bool = True, fully_paired: bool = False, kl_dot_product: bool = False, **model_kwargs)

Initialize the MultiVI model.

Parameters:

• adata (AnnData) – AnnOrMuData object that has been registered via setup_anndata(). Features of the AnnData object should include gene expression followed by region accessibility(if exits). If protein expression is available, it should be present in obsm.

• n_genes (int) – Number of genes in the dataset.

• n_regions (int) – Number of regions in the dataset.

• n_proteins (int) – Number of proteins in the dataset.

• n_latent (int, optional, default=20) – Dimensionality of the latent space.

• n_hidden (Optional[int], optional) – Number of hidden units in the neural network layers.

• n_layers_encoder (int, optional, default=2) – Number of layers in the encoder neural network.

• n_layers_decoder (int, optional, default=2) – Number of layers in the decoder neural network.

• dropout_rate (float, optional, default=0.1) – Dropout rate for the neural network layers.

• region_factors (bool, optional, default=True) – Whether to learn region-specific scaling factors for the accessibility regions.

• use_batch_norm (Literal["encoder", "decoder", "none", "both"], optional, default="both") – Specifies where to use batch normalization.

• use_layer_norm (Literal["encoder", "decoder", "none", "both"], optional, default="none") – Specifies where to use layer normalization.

• gene_dispersion (Literal["gene", "gene-batch", "gene-label", "gene-cell"], optional, default="gene") – One of the following: * 'gene' - genes_dispersion parameter of NB is constant per gene across cells * 'gene-batch' - genes_dispersion can differ between different batches * 'gene-label' - genes_dispersion can differ between different labels * 'gene-cell' - genes_dispersion can differ between different cells

• protein_dispersion (Literal["protein", "protein-batch", "protein-label"], optional, default="protein") – One of the following: * 'protein' - protein_dispersion parameter is constant per protein across cells * 'protein-batch' - protein_dispersion can differ between different batches NOT TESTED * 'protein-label' - protein_dispersion can differ between different labels NOT TESTED

• gene_likelihood (Literal["zinb", "nb"], optional, default="nb") – One of: * 'nb' - Negative binomial distribution * 'zinb' - Zero-inflated negative binomial distribution

• latent_distribution (Literal["normal", "ln"], optional, default="normal") – One of: * 'normal' - Normal distribution * 'ln' - Logistic normal distribution (Normal(0, I) transformed by softmax)

• empirical_protein_background_prior (Optional[bool], optional) – Set the initialization of protein background prior empirically. This option fits a GMM for each of 100 cells per batch and averages the distributions. Note that even with this option set to True, this only initializes a parameter that is learned during inference. If False, randomly initializes. The default (None), sets this to True if greater than 10 proteins are used.

• override_missing_proteins (bool, optional, default=False) – If True, will not treat proteins with all 0 expression in a particular batch as missing.

• deeply_inject_covariates (bool, optional, default=False) – Whether to deeply inject covariates into the model.

• encode_covariates (bool, optional, default=True) – Whether to encode covariates in the model.

• fully_paired (bool, optional, default=False) – Whether the dataset is fully paired.

• kl_dot_product (bool, optional, default=False) – Whether to use KL divergence with dot product.

• **model_kwargs – Additional keyword arguments for the model.

Notes

This class initializes the MultiVI model with the specified parameters and prepares the module for training.

differential_expression(adata: AnnData | None = None, groupby: str | None = None, group1: Iterable[str] | None = None, group2: str | None = None, idx1: Sequence[int] | Sequence[bool] | str | None = None, idx2: Sequence[int] | Sequence[bool] | str | None = None, mode: Literal['vanilla', 'change'] = 'change', delta: float = 0.25, batch_size: int | None = None, all_stats: bool = True, batch_correction: bool = False, batchid1: Iterable[str] | None = None, batchid2: Iterable[str] | None = None, fdr_target: float = 0.05, silent: bool = False, protein_prior_count: float = 0.1, scale_protein: bool = False, sample_protein_mixing: bool = False, include_protein_background: bool = False, **kwargs) → DataFrame

A unified method for differential expression analysis.

Implements “vanilla” DE [2]. and “change” mode DE [1].

Parameters:

• adata – AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.

• groupby – The key of the observations grouping to consider.

• group1 – Subset of groups, e.g. [‘g1’, ‘g2’, ‘g3’], to which comparison shall be restricted, or all groups in groupby (default).

• group2 – If None, compare each group in group1 to the union of the rest of the groups in groupby. If a group identifier, compare with respect to this group.

• idx1 – idx1 and idx2 can be used as an alternative to the AnnData keys. Custom identifier for group1 that can be of three sorts: (1) a boolean mask, (2) indices, or (3) a string. If it is a string, then it will query indices that verifies conditions on adata.obs, as described in pandas.DataFrame.query() If idx1 is not None, this option overrides group1 and group2.

• idx2 – Custom identifier for group2 that has the same properties as idx1. By default, includes all cells not specified in idx1.

• mode – Method for differential expression. See user guide for full explanation.

• delta – specific case of region inducing differential expression. In this case, we suppose that \(R \setminus [-\delta, \delta]\) does not induce differential expression (change model default case).

• batch_size – Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.

• all_stats – Concatenate count statistics (e.g., mean expression group 1) to DE results.

• batch_correction – Whether to correct for batch effects in DE inference.

• batchid1 – Subset of categories from batch_key registered in setup_anndata, e.g. [‘batch1’, ‘batch2’, ‘batch3’], for group1. Only used if batch_correction is True, and by default all categories are used.

• batchid2 – Same as batchid1 for group2. batchid2 must either have null intersection with batchid1, or be exactly equal to batchid1. When the two sets are exactly equal, cells are compared by decoding on the same batch. When sets have null intersection, cells from group1 and group2 are decoded on each group in group1 and group2, respectively.

• fdr_target – Tag features as DE based on posterior expected false discovery rate.

• silent – If True, disables the progress bar. Default: False.

• protein_prior_count – Prior count added to protein expression before LFC computation

• scale_protein – Force protein values to sum to one in every single cell (post-hoc normalization)

• sample_protein_mixing – Sample the protein mixture component, i.e., use the parameter to sample a Bernoulli that determines if expression is from foreground/background.

• include_protein_background – Include the protein background component as part of the protein expression

• use_field – By default uses protein and RNA field disable here to perform only RNA or protein DE.

• pseudocounts – pseudocount offset used for the mode change. When None, observations from non-expressed genes are used to estimate its value.

• **kwargs – Keyword args for scvi.model.base.DifferentialComputation.get_bayes_factors()

Return type:

Differential expression DataFrame.

get_accessibility_estimates(adata: AnnData | None = None, indices: Sequence[int] = None, n_samples_overall: int | None = None, region_list: Sequence[str] | None = None, transform_batch: str | int | None = None, use_z_mean: bool = True, threshold: float | None = None, normalize_cells: bool = False, normalize_regions: bool = False, batch_size: int = 128, return_numpy: bool = False) → ndarray | csr_matrix | DataFrame

Impute the full accessibility matrix.

Returns a matrix of accessibility probabilities for each cell and genomic region in the input (for return matrix A, A[i,j] is the probability that region j is accessible in cell i).

Parameters:

• adata – AnnData object that has been registered with scvi. If None, defaults to the AnnData object used to initialize the model.

• indices – Indices of cells in adata to use. If None, all cells are used.

• n_samples_overall – Number of samples to return in total

• region_indices – Indices of regions to use. if None, all regions are used.

• transform_batch –

  Batch to condition on. If transform_batch is:

  • None, then real observed batch is used

  • int, then batch transform_batch is used

• use_z_mean – If True (default), use the distribution mean. Otherwise, sample from the distribution.

• threshold – If provided, values below the threshold are replaced with 0 and a sparse matrix is returned instead. This is recommended for very large matrices. Must be between 0 and 1.

• normalize_cells – Whether to reintroduce library size factors to scale the normalized probabilities. This makes the estimates closer to the input, but removes the library size correction. False by default.

• normalize_regions – Whether to reintroduce region factors to scale the normalized probabilities. This makes the estimates closer to the input, but removes the region-level bias correction. False by default.

• batch_size – Minibatch size for data loading into model

get_feature_correlation_matrix(adata=None, indices=None, n_samples: int = 10, batch_size: int = 64, rna_size_factor: int = 1000, transform_batch: Sequence[Number | str] | None = None, correlation_type: Literal['spearman', 'pearson'] = 'spearman', log_transform: bool = False) → pd.DataFrame

Generate gene-gene correlation matrix using scvi uncertainty and expression.

Parameters:

• adata – AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.

• indices – Indices of cells in adata to use. If None, all cells are used.

• n_samples – Number of posterior samples to use for estimation.

• batch_size – Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.

• rna_size_factor – size factor for RNA prior to sampling gamma distribution

• transform_batch –

  Batches to condition on. If transform_batch is:

  • None, then real observed batch is used

  • int, then batch transform_batch is used

  • list of int, then values are averaged over provided batches.

• correlation_type – One of “pearson”, “spearman”.

• log_transform – Whether to log transform denoised values prior to correlation calculation.

Return type:

Gene-protein-gene-protein correlation matrix

get_latent_library_size(adata: AnnData | None = None, indices: Sequence[int] | None = None, give_mean: bool = True, batch_size: int | None = None) → ndarray

Returns the latent library size for each cell.

This is denoted as \(\ell_n\) in the totalVI paper.

Parameters:

• adata – AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.

• indices – Indices of cells in adata to use. If None, all cells are used.

• give_mean – Return the mean or a sample from the posterior distribution.

• batch_size – Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.

get_latent_representation(adata: AnnData | None = None, indices: Sequence[int] | None = None, give_mean: bool = True, mc_samples: int = 5000, batch_size: int | None = None, return_dist: bool = False, add_latents_to_adata: bool = True)

Return the latent representation for each cell. This is typically denoted as \(z, z^{s1}, z^{s2}, z^r, z^a, z^p\).

\(z\) is the latent representation of the cell for downstream tasks

\(z_{s1}\) is the latent representation of the first level heirarchical representation

\(z_{s2}\) is the latent representation of the second level heirarchical representation

\(z^r\) is the latent representation of the genes

\(z^a\) is the latent representation of accessibility. (if applicable)

\(z^p\) is the latent representation of protein. (if applicable)

Parameters:

• adata – AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.

• indices – Indices of cells in adata to use. If None, all cells are used.

• give_mean – Give mean of distribution or sample from it.

• mc_samples – For distributions with no closed-form mean (e.g., logistic normal), how many Monte Carlo samples to take for computing mean.

• batch_size – Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.

• return_dist – Return (mean, variance) of distributions instead of just the mean. If True, ignores give_mean and mc_samples. In the case of the latter, mc_samples is used to compute the mean of a transformed distribution. If return_dist is true the untransformed mean and variance are returned.

Return type:

Low-dimensional representation for each cell or a tuple containing its mean and variance.

get_likelihood_parameters(adata: AnnData | None = None, indices: Sequence[int] | None = None, n_samples: int | None = 1, give_mean: bool | None = False, batch_size: int | None = None) → Dict[str, ndarray]

Estimates for the parameters of the likelihood \(p(x, y \mid z)\).

Parameters:

• adata – AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.

• indices – Indices of cells in adata to use. If None, all cells are used.

• n_samples – Number of posterior samples to use for estimation.

• give_mean – Return expected value of parameters or a samples

• batch_size – Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.

get_normalized_expression(adata: AnnData | None = None, indices: Sequence[int] | None = None, n_samples_overall: int | None = None, transform_batch: Sequence[Number | str] | None = None, gene_list: Sequence[str] | None = None, use_z_mean: bool = True, n_samples: int = 1, batch_size: int | None = None, return_mean: bool = True, return_numpy: bool = False) → np.ndarray | pd.DataFrame

Returns the normalized (decoded) gene expression.

This is denoted as \(\rho_n\) in the scVI paper.

Parameters:

• adata – AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.

• indices – Indices of cells in adata to use. If None, all cells are used.

• transform_batch –

  Batch to condition on. If transform_batch is:

  • None, then real observed batch is used.

  • int, then batch transform_batch is used.

• gene_list – Return frequencies of expression for a subset of genes. This can save memory when working with large datasets and few genes are of interest.

• library_size – Scale the expression frequencies to a common library size. This allows gene expression levels to be interpreted on a common scale of relevant magnitude. If set to “latent”, use the latent library size.

• use_z_mean – If True, use the mean of the latent distribution, otherwise sample from it

• n_samples – Number of posterior samples to use for estimation.

• batch_size – Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.

• return_mean – Whether to return the mean of the samples.

Returns:

• If n_samples > 1 and return_mean is False, then the shape is (samples, cells, genes).

• Otherwise, shape is (cells, genes). In this case, return type is DataFrame unless return_numpy is True.

get_protein_foreground_probability(adata: AnnData | None = None, indices: Sequence[int] | None = None, transform_batch: Sequence[Number | str] | None = None, protein_list: Sequence[str] | None = None, n_samples: int = 1, batch_size: int | None = None, return_mean: bool = True, return_numpy: bool | None = None)

Returns the foreground probability for proteins.

This is denoted as \((1 - \pi_{nt})\) in the totalVI paper.

Parameters:

• adata – AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.

• indices – Indices of cells in adata to use. If None, all cells are used.

• transform_batch –

  Batch to condition on. If transform_batch is:

  • None, then real observed batch is used

  • int, then batch transform_batch is used

  • List[int], then average over batches in list

• protein_list – Return protein expression for a subset of genes. This can save memory when working with large datasets and few genes are of interest.

• n_samples – Number of posterior samples to use for estimation.

• batch_size – Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.

• return_mean – Whether to return the mean of the samples.

• return_numpy – Return a ndarray instead of a DataFrame. DataFrame includes gene names as columns. If either n_samples=1 or return_mean=True, defaults to False. Otherwise, it defaults to True.

Returns:

• - **foreground_probability* - probability foreground for each protein*

• If n_samples > 1 and return_mean is False, then the shape is (samples, cells, genes).

• Otherwise, shape is (cells, genes). In this case, return type is DataFrame unless return_numpy is True.

posterior_predictive_sample(adata: AnnData | None = None, indices: Sequence[int] | None = None, n_samples: int = 1, batch_size: int | None = None, gene_list: Sequence[str] | None = None, protein_list: Sequence[str] | None = None, atac_list: Sequence[str] | None = None, swap_latent=False) → ndarray

Generate observation samples from the posterior predictive distribution.

The posterior predictive distribution is written as \(p(\hat{x}, \hat{y} \mid x, y)\).

Parameters:

• adata – AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.

• indices – Indices of cells in adata to use. If None, all cells are used.

• n_samples – Number of required samples for each cell

• batch_size – Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.

• gene_list – Names of genes of interest

• protein_list – Names of proteins of interest

• swap_latent – uses z2 instead of z1 while regenerating gene

Returns:

x_new – tensor with shape (n_cells, n_genes, n_samples)

Return type:

ndarray

classmethod setup_anndata(adata: AnnData, layer: str | None = None, batch_key: str | None = None, size_factor_key: str | None = None, categorical_covariate_keys: list[str] | None = None, continuous_covariate_keys: list[str] | None = None, protein_expression_obsm_key: str | None = None, protein_names_uns_key: str | None = None, **kwargs)

Sets up the AnnData object for this model.

A mapping will be created between data fields used by this model to their respective locations in adata. None of the data in adata are modified. Only adds fields to adata.

Parameters:

• adata – AnnData object. Rows represent cells, columns represent features.

• layer – if not None, uses this as the key in adata.layers for raw count data.

• batch_key – key in adata.obs for batch information. Categories will automatically be converted into integer categories and saved to adata.obs[‘_scvi_batch’]. If None, assigns the same batch to all the data.

• size_factor_key – key in adata.obs for size factor information. Instead of using library size as a size factor, the provided size factor column will be used as offset in the mean of the likelihood. Assumed to be on linear scale.

• categorical_covariate_keys – keys in adata.obs that correspond to categorical data. These covariates can be added in addition to the batch covariate and are also treated as nuisance factors (i.e., the model tries to minimize their effects on the latent space). Thus, these should not be used for biologically-relevant factors that you do _not_ want to correct for.

• continuous_covariate_keys – keys in adata.obs that correspond to continuous data. These covariates can be added in addition to the batch covariate and are also treated as nuisance factors (i.e., the model tries to minimize their effects on the latent space). Thus, these should not be used for biologically-relevant factors that you do _not_ want to correct for.

• protein_expression_obsm_key – key in adata.obsm for protein expression data.

• protein_names_uns_key – key in adata.uns for protein names. If None, will use the column names of adata.obsm[protein_expression_obsm_key] if it is a DataFrame, else will assign sequential names to proteins.

train(max_epochs: int | None = None, lr: float = 0.004, accelerator: str = 'auto', devices: int | list[int] | str = 'auto', train_size: float | None = None, validation_size: float | None = None, shuffle_set_split: bool = True, batch_size: int = 256, early_stopping: bool = True, check_val_every_n_epoch: int | None = None, reduce_lr_on_plateau: bool = True, n_steps_kl_warmup: int | None = None, n_epochs_kl_warmup: int | None = None, adversarial_classifier: bool | None = None, datasplitter_kwargs: dict | None = None, plan_kwargs: dict | None = None, external_indexing: list[array] = None, **kwargs)

Trains the model using amortized variational inference.

Parameters:

• max_epochs – Number of passes through the dataset.

• lr – Learning rate for optimization.

• accelerator – Supports passing different accelerator types (“cpu”, “gpu”, “tpu”, “ipu”, “hpu”, “mps, “auto”) as well as custom accelerator instances.

• devices – The devices to use. Can be set to a non-negative index (int or str), a sequence of device indices (list or comma-separated str), the value -1 to indicate all available devices, or “auto” for automatic selection based on the chosen accelerator. If set to “auto” and accelerator is not determined to be “cpu”, then devices will be set to the first available device.

• train_size – Size of training set in the range [0.0, 1.0].

• validation_size – Size of the test set. If None, defaults to 1 - train_size. If train_size + validation_size < 1, the remaining cells belong to a test set.

• shuffle_set_split – Whether to shuffle indices before splitting. If False, the val, train, and test set are split in the sequential order of the data according to validation_size and train_size percentages.

• batch_size – Minibatch size to use during training.

• early_stopping – Whether to perform early stopping with respect to the validation set.

• check_val_every_n_epoch – Check val every n train epochs. By default, val is not checked, unless early_stopping is True or reduce_lr_on_plateau is True. If either of the latter conditions are met, val is checked every epoch.

• reduce_lr_on_plateau – Reduce learning rate on plateau of validation metric (default is ELBO).

• n_steps_kl_warmup – Number of training steps (minibatches) to scale weight on KL divergences from 0 to 1. Only activated when n_epochs_kl_warmup is set to None. If None, defaults to floor(0.75 * adata.n_obs).

• n_epochs_kl_warmup – Number of epochs to scale weight on KL divergences from 0 to 1. Overrides n_steps_kl_warmup when both are not None.

• adversarial_classifier – Whether to use adversarial classifier in the latent space. This helps mixing when there are missing proteins in any of the batches. Defaults to True is missing proteins are detected.

• datasplitter_kwargs – Additional keyword arguments passed into DataSplitter.

• plan_kwargs – Keyword args for AdversarialTrainingPlan. Keyword arguments passed to train() will overwrite values present in plan_kwargs, when appropriate.

• external_indexing – A list of data split indices in the order of training, validation, and test sets. Validation and test set are not required and can be left empty.

• **kwargs – Other keyword args for Trainer.