Applied Probability(D)

APPLIED PROBABILITY (D)

Markov Chains is essential

Finite-state continuous-time Markov chains: basic properties. Q-matrix (or generator), backward and forward equations. The homogeneous Poisson process and its properties (thinning, superposition). Birth and death processes. [6]
General continuous-time Markov chains. Jump chains. Explosion. Minimal Chains. Communicating classes. Hitting times and probabilities. Recurrence and transience. Positive and null recurrence. Convergence to equilibrium. Reversibility. [6]
Applications: the M/M/1 and M/M/∞ queues. Burke’s theorem. Jackson’s theorem for queueing networks. The M/G/1 queue. [4]
Renewal theory: renewal theorems, equilibrium theory (proof of convergence only in discrete time). Renewal-reward processes. Little’s formula. [4]
Spatial Poisson processes in d dimensions. The superposition, mapping, and colouring theorems. R´enyi’s theorem. Applications including Olbers’ paradox. [4]

Appropriate books