Numerical analysis of infinite Markov processes

Abstract

The estimation of the steady state probability distribution of infinite discrete state Markov processes with an state space is the subject of this thesis. The measurement and analysis of complex queueing systems is the motive for this investigation, since a classical approach to analysing queueing systems is to imbed the model in a Markov process. The literature pertaining to the numercal solution of Markov processes is surveyed, with the object of finding a method which will be applicable to a wide class of processes with a minimum of prior analysis. A general method of numbering discrete states in an infinite domain is developed and used to map the discrete state spaces of Markov processes into the positive integers, for the purpose of applying standard numerical techniques. A theoretical result which has not been previously employed, is implemented and compared with two other algorithms which were developed for use with finite slate space Markov processes. Some problems with no closed form analytic solution are studied numerically and steady state and marginal distributions are found.