Probability Review and Introduction to Stochastic Processes (SPs): Probability spaces, random variables and probability distributions, expectations, transforms and generating functions, convergence, LLNs, CLT.
Definition, examples and classification of random processes according to state space and parameter space.
Stationary Processes: Weakly stationary and strongly stationary processes, moving average and auto regressive processes
Discrete-time Markov Chains (DTMCs): Transition probability matrix, Chapman-Kolmogorov equations; n-step transition and limiting probabilities, ergodicity, stationary distribution, random walk and gambler’s ruin problem, applications of DTMCs.
Continuous-time Markov Chains (CTMCs): Kolmogorov differential equations for CTMCs, infinitesimal generator, Poisson and birth-death processes, stochastic Petri net, applications to queueing theory and communication networks.
Martingales: Conditional expectations, definition and examples of martingales, applications in finance.
Brownian Motion: Wiener process as a limit of random walk; process derived from Brownian motion, stochastic differential equation, stochastic integral equation, Ito formula, Some important SDEs and their solutions, applications to finance.
Renewal Processes: Renewal function and its properties, renewal theorems, cost/rewards associated with renewals, Markov renewal and regenerative processes, non Markovian queues, applications of Markov regenerative processes.
Branching Processes: Definition and examples branching processes, probability generating function, mean and variance, Galton-Watson branching process, probability of extinction.