Course syllabus: We will start by developing the theory of Markov chains further, explore their relations to martingales and give applications to the solution of Dirichlet problem and maximum (resp.