Overview

This repository gathers various convenience tools for analyzing and generating Markov Chains with a finite state space. The following subsections increase in conceptual complexity and are intended to be read in order.

Discrete time Markov chains

The following sections contain a review of Markov Chains and an introduction to functionality in the repository.

1. Transfer Operators and Markov Chains: Generate a Markov Chain from a transfer operator
2. Data-driven Transfer Operators: Construct empirical transition matrices and distributions from a Markov Chain
3. Convergence of Transfer Operators: Verify convergence of empirical transition matrices and distributions to the true transition operator

Continuous time Markov chains

The following sections contain more advanced mathematical concepts using continuous time Markov processes

1. Generators and Markov Chains: Create a Markov chain from a generator matrix
2. Data-driven Generator: Construct empirical generators from a Markov Chain
3. Convergence of Generators: Verify convergence of empirical generators to the true generator

Uncertainty Quantification

The following sections introduce uncertainty quantification for the estimate of Generators from finite data

1. Bayesian Empirical Generator: Estimate the uncertainty of the entries in the empirical generator from finite data
2. Constructing Prior Distributions: Construct and use prior distributions for the entries in the empirical generator
3. Sampling from the Bayesian Generator: Sample from the Bayesian empirical generator to propagate uncertainties