Non-Homogeneous Markov Chains and Systems Theory and Applications

1. Foundations of Probability Theory. 1.1. Introductory Notes. 1.2. Some set theory and topology. 1.3. Important family of Sets in Probability theory. 1.4. Measurable spaces. 1.5. Probability spaces. 1.6. Filtration. 1.7. Random Variables. 1.8. Integration with respect to a probability measure. Expectation of a random variable. 1.9. Indicator functions. 1.10. The space L² is a Hilbert space. 1.11. Independent σ-algebras and random variables. 1.12. Convergence of sequences of random variables. 1.13. The Laws of Large Numbers and the Central Limit Theorem. 1.14. Conditional Distributions and Conditional Expectations. 1.15. Change of measure. 1.16. Existence and Uniqueness of Conditional Expectations. 1.17. Properties of Conditional Expectation. 2. A Small Review of Matrix Analysis. 2.1. Introductory Notes. 2.2. Matrices. 2.3. The minimal polynomial of A. 2.4. The Norm of a vector. 2.5. The matrix norm. 2.6. The Kronecker product of two matrices. 2.7. The Hadamard product of matrices. 2.8. Canonical forms of a matrix. 2.9. Generalized Inverses. 2.10. The Moore-Penrose Generalized Matrix. 2.11. The Drazin Inverse and the Group Inverse. 2.12. The Group Inverse and Markov Chains. 2.13. Sensitivity of Markov Chains. 3. Non-Homogeneous Markov Chains, Weak Ergodicity. 3.1. Introductory notes. 3.2. Stochastic processes. 3.3. Markov chain. 3.4. The life and work of A.A. Markov. 3.5. Probability distribution in the states of a nonhomogeneous Markov chain. 3.6. Examples. 3.7. Weak and Strong Ergodicity. 3.8. Structures for coefficients of ergodicity. 3.9. Conditions for weak ergodicity for general products of stochastic matrices. 3.10. The dominant role of the Dobrushin ergodicity coefficient. 3.11. Transition probability matrices are in chronological order. 3.12. Examples on the use of weak ergodicity theorems. 4. Non-Homogeneous Markov Chains, Strong Ergodicity. 4.1. Strong Ergodicity. 4.2. Characterization of ergodicity and geometric strong ergodicity. 4.3. Criteria for Strong Ergodicity for Non-Homogeneous Markov Chains.4.4 Convergence in the Cesaro Sense. 4.5. Uniform strong ergodicity with the use of mean visit times. 4.6. Strong Ergodicity for General Products of Matrices. 4.7. Sets of Matrices all Non-homogeneous Products of Which Converge. 4.8. A Geometric approach to ergodic Non-Homogeneous Markov chains. 4.9. Asymptotic behavior for Markov Chains with arbitrary stochastic matrices. 5. The Non-Homogeneous Markov System. 5.1. Introductory Notes. 5.2. The Non-Homogeneous Markov System in discrete time and space. 5.3. The expected and the relative expected population structure. 5.4. A range of NHMS’s environment. 5.5. The б--Non-homogeneous Markov system. 5.6. Change of measure in G-non-homogeneous Markov systems. 5.7. The space of random population structures as a Hilbert space. 5.8. Estimation of the transition probabilities of a NHMS. 5.9. Research Notes. 6. Asymptotic Behaviour of a Non-Homogeneous Markov System. 6.1. Introductory notes. 6.2. Asymptotic behavior of the expected population structure. 6.3. Asymptotic behavior of the relative expected population structure. 6.4. Asymptotic behavior of a contracting NHMS without forced wastage. 6.5. NHMS with non-homogeneous Poisson recruitment. 6.6. Asymptotic stability in NHMS’s. 6.7. A G-non-homogeneous Markov system with non-homogeneous Poisson input, as a martingale. 6.8. Research Notes. 7. Asymptotic Variability of NHMS. 7.1. Introductory Notes. 7.2. The V Matrices and their Properties. 7.3. An illustrative probable application. 7.4. Rate of convergence of the variability vector. 7.5. Research notes. 8. Cyclic Behavior of Non-Homogeneous Markov Systems. 8.1. Introductory Notes. 8.2. Cyclic Non-homogeneous Markov systems. 8.3. An illustrative example from a manpower system. 8.4. Rate of convergence of {μ} t=0∞ in a NHMS under cyclic behavior. 8.5. Research notes. 9. Stochastic Control in NHMS’S. 9.1. Introductory notes. 9.2. Maintainability in NHMS by input control. 9.3. Attainability in NHMS by input control. 9.4. Asymptotically Attainable Structures. 9.5. Periodicity of Asymptotically Attainable structures. 9.6. An illustrative example. 9.7. Control of asymptotic variability in non-homogeneous Markov systems. 9.8. The non-homogeneous Markov system in a stochastic environment. 9.9. Maintainability in a stochastic environment. 9.10. Strategies for attaining a structure in a S-NHMS. 9.11. An illustrative application. 9.12. Research notes. 10. Laws of Large Numbers for Non-Homogeneous Markov Systems. 10.1. Introductory notes. 10.2. Basic concepts and useful results. 10.3. Laws of Large Numbers for a NHMS. 10.4. An illustrative application. 10.5. Laws of Large Numbers for a Cy-NHMS. 10.6. An illustrative application. 10.7. Laws of Large Numbers for NHMS with arbitrary transition probability matrices. 10.8. An illustrative example. 11. The Non-Homogeneous Markov System in a Stochastic Environment in Continuous Time. 11.1. Introductory notes. 11.2. Asymptotic behavior of non-homogeneous Markov process in continuous time. 11.3. The S-NHMS in continuous time. 11.4. The expected population structure of the S-NHMSC. 11.5. The asymptotic behavior of the S-NHMSC. 11.6. An illustrative example from manpower planning. 11.7. Research notes. 12. The Perturbed Non-Homogeneous Markov System. 12.1. Introductory notes. 12.2. The Group inverse and the asymptotic behavior of a Markov chain. 12.3. The Perturbed Non-Homogeneous Markov System. 12.4. The oscillation of the matrix Q in manpower systems. 12.5. Asymptotic behavior of the P-NHMS. 12.6. Sensitivity of the asymptotic relative expected population structure of a NHMS in changes in the limiting input probabilities. 12.7. Asymptotic variability of the P-NHMS. 12.8. Research Notes. 13. Non-Homogeneous Markov Set System. 13.1. Introduction. 13.2. Non-Homogeneous Markov Set System. 13.3. The set of the expected relative population structures of a NHMSS. 13.4. Asymptotic behavior of NHMSS. 13.5. Properties of the limiting set Rng∞ (S;R0). 13.6. An illustrative representative example. 14. Markov Systems on a General State Space. 14.1. Introductory notes. 14.2. A Mental image for MSGS. 14.3. The foundation of a MSGS. 14.4. Asymptotic behavior or ergodicity of MSGS. 14.5. Total variability from the invariant measure. A coupling theorem. 14.6. Rate of convergence of MSGS. 14.7. Asymptotic periodicity of a MSGS. 14.8. Total Variability from the Invariant Measures in the periodic case. 15. The G—Non-Homogeneous Markov System of High Order. 15.1. Introductory notes. 15.2. The G-non-homogeneous Markov system of high order. 15.3. The population structure of the G-inhomogeneous Markov System. 15.4. The Inhomogeneous Mixture Transition Distribution Model. 15.5. The Asymptotic Behavior of the Inherent Markov Chain. 15.6. The asymptotic expected population structure. 15.7. An illustrative example from manpower systems. 15.8. Research Notes. References.

Biography

P.-C.G. Vassiliou is an Honorary Professor at the Department of Statistical Sciences, University College London. He has been affiliated with UCL for almost a decade.