Numerical analysis of stochastic models
Solving very large Markov chains quickly — phase-type distributions, Markovian arrival processes and the EM algorithm.
Discrete- and continuous-time Markov chains are used far beyond reliability, notably in the performance evaluation of queueing systems. In reliability they matter especially because the measures of interest often need an accuracy that simulation cannot reach, which makes model-based analysis essential.
When a real information system is written as a stochastic model, the number of states grows exponentially with the complexity of its behaviour, and computation quickly becomes intractable. We study numerical algorithms for Markov chains with very large state spaces: fast and scalable methods for steady-state and transient measures, approximations using phase-type distributions and Markovian arrival processes, and algorithms suited to parallel computation.
Estimation matters just as much. We develop methods that take the limitations of real data as given — an EM algorithm that estimates the parameters of a Markovian arrival process from grouped data, and phase-type estimation for data with left truncation and right censoring. We also work on model description with stochastic Petri nets and fault trees, and on the numerical analysis of those descriptions.
Research highlights in this area
Interactive demos in this area
Rock-paper-scissors × reinforcement learning (opens an external site)
Play against an agent that learns your habits, and watch the statistics update in real time.
Cart-pole balancing with reinforcement learning (opens an external site)
Watch a reinforcement-learning agent learn to keep a pole upright through trial and error.
Probabilistic geometric coverage reliability (opens an external site)
Where should sensors go to cover a region reliably? Binary decision diagrams give an exact answer, recomputed in the browser as you move sensors.
Software from this area
PhaseTypeInference.jl Julia
Phase-type distribution fitting for incompletely observed data, with AIC/EIC model selection.
NMarkov.jl Julia
Numerical computation for Markov chains in Julia.
DEQuadrature.jl Julia
Numerical quadrature on finite and semi-infinite intervals using double-exponential formulas.
mapfit R
An R package for fitting phase-type distributions and Markovian arrival processes. Available on CRAN.
deformula R
An R package for one-dimensional numerical integration with double-exponential formulas. Available on CRAN.