Transition-rate matrix

In probability theory, a transition-rate matrix (also known as a Q-matrix,[1] intensity matrix,[2] or infinitesimal generator matrix[3]) is an array of numbers describing the instantaneous rate at which a continuous-time Markov chain transitions between states.

In a transition-rate matrix $Q$ (sometimes written $A$ [4]), element $q_{ij}$ (for $i\neq j$ ) denotes the rate departing from $i$ and arriving in state $j$ . The rates $q_{ij}\geq 0$ , and the diagonal elements $q_{ii}$ are defined such that

q_{ii}=-\sum _{j\neq i}q_{ij}

,

and therefore the rows of the matrix sum to zero.

Up to a global sign, a large class of examples of such matrices is provided by the Laplacian of a directed, weighted graph. The vertices of the graph correspond to the Markov chain's states.

Properties

The transition-rate matrix has following properties:[5]

There is at least one eigenvector with a vanishing eigenvalue, exactly one if the graph of $Q$ is strongly connected.
All other eigenvalues $\lambda$ fulfill $0>\mathrm {Re} \{\lambda \}\geq 2\min _{i}q_{ii}$ .
All eigenvectors $v$ with a non-zero eigenvalue fulfill $\sum _{i}v_{i}=0$ .

Example

An M/M/1 queue, a model which counts the number of jobs in a queueing system with arrivals at rate λ and services at rate μ, has transition-rate matrix

Q={\begin{pmatrix}-\lambda &\lambda \\\mu &-(\mu +\lambda )&\lambda \\&\mu &-(\mu +\lambda )&\lambda \\&&\mu &-(\mu +\lambda )&\ddots &\\&&&\ddots &\ddots \end{pmatrix}}.

References

Suhov & Kelbert 2008, Definition 2.1.1.
Asmussen, S. R. (2003). "Markov Jump Processes". Applied Probability and Queues. Stochastic Modelling and Applied Probability. Vol. 51. pp. 39–59. doi:10.1007/0-387-21525-5_2. ISBN 978-0-387-00211-8.
Trivedi, K. S.; Kulkarni, V. G. (1993). "FSPNs: Fluid stochastic Petri nets". Application and Theory of Petri Nets 1993. Lecture Notes in Computer Science. Vol. 691. p. 24. doi:10.1007/3-540-56863-8_38. ISBN 978-3-540-56863-6.
Rubino, Gerardo; Sericola, Bruno (1989). "Sojourn Times in Finite Markov Processes" (PDF). Journal of Applied Probability. Applied Probability Trust. 26 (4): 744–756. doi:10.2307/3214379. JSTOR 3214379. S2CID 54623773.
Keizer, Joel (1972-11-01). "On the solutions and the steady states of a master equation". Journal of Statistical Physics. 6 (2): 67–72. Bibcode:1972JSP.....6...67K. doi:10.1007/BF01023679. ISSN 1572-9613. S2CID 120377514.

Norris, J. R. (1997). Markov Chains. doi:10.1017/CBO9780511810633.005. ISBN 9780511810633.
Suhov, Yuri; Kelbert, Mark (2008). Markov chains: a primer in random processes and their applications. Cambridge University Press.
Syski, R. (1992). Passage Times for Markov Chains. IOS Press. ISBN 90-5199-060-X.

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

[FOOTNOTESuhovKelbert2008Definition_2.1.1-1] Suhov & Kelbert 2008, Definition 2.1.1.

[2] Asmussen, S. R. (2003). "Markov Jump Processes". Applied Probability and Queues. Stochastic Modelling and Applied Probability. Vol. 51. pp. 39–59. doi:10.1007/0-387-21525-5_2. ISBN 978-0-387-00211-8.

[3] Trivedi, K. S.; Kulkarni, V. G. (1993). "FSPNs: Fluid stochastic Petri nets". Application and Theory of Petri Nets 1993. Lecture Notes in Computer Science. Vol. 691. p. 24. doi:10.1007/3-540-56863-8_38. ISBN 978-3-540-56863-6.

[4] Rubino, Gerardo; Sericola, Bruno (1989). "Sojourn Times in Finite Markov Processes" (PDF). Journal of Applied Probability. Applied Probability Trust. 26 (4): 744–756. doi:10.2307/3214379. JSTOR 3214379. S2CID 54623773.

[5] Keizer, Joel (1972-11-01). "On the solutions and the steady states of a master equation". Journal of Statistical Physics. 6 (2): 67–72. Bibcode:1972JSP.....6...67K. doi:10.1007/BF01023679. ISSN 1572-9613. S2CID 120377514.

Transition-rate matrix

Properties

Example

See also

References