Intelligent and Bayesian Models for Estimating the Transition Probability Matrix of Markov chains: Applications to the Analysis of Sustainable Development Indicators

Abstract

In discrete-time Markov chains, the correct estimation of the transition probability matrix is important in defining the dynamics of the system. But classical estimators may become erratic when the data that is available is sparse or irregular. The study will discuss four ways of estimating the transition matrix, including the Maximum Likelihood Estimator (MLE), Bayesian-Dirichlet estimator, the shrinkage estimator, and an intelligent optimization-based estimator built on the Artificial Bee Colony (ABC) algorithm. The developed methodology is implemented to the real-life data associated with the Sustainable Development Goal 4 (Quality Education) i.e. the annual literacy level of adults (percentage of the population aged 15 and above) in the course of a long-term. Three measures were used to ensure a good comparison between the estimators: the total variance of the transition matrix, the spectral gap and the mixing time. As suggested by the experimental results MLE is most fluctuating and slowest to converge especially when the transition structures are weak. The Bayesian-Dirichlet estimator has significant benefits in the form of probabilistic smoothing alone, whilst the shrinkage estimator has additional benefits in the form of a reduction in the variance and a faster convergence rate at the expense of empirical information and structured targets. Among the estimators we have experimented with, the ABC-based estimator was the best in terms of the smallest total variance, largest (lazy) spectral-gap proxy and shortest mixing time. These findings imply that smart optimization is a good alternative on estimating the well-conditioned transition matrices in irregular and complicated socio-educational time series.