BSc Computer Science and Mathematics with Industrial Experience / Course details

The course introduces some simple stochastic processes, that is phenomena which evolve in time in a non-deterministic way. It applies the techniques developed in Probability and Statistics 1 and 2 together with the use of generating functions (or power series) to tackle problems such as the gambler's ruin problem, or calculating the probability of the extinction of certain populations.

The course unit unit aims to enable students to develop some understanding of the way that stochastic processes evolve in time, to become familiar with some simple techniques which help in their study, and to experience some real life applications of stochastic processes.

On completion of this unit successful students will be able to:

• Define probability generating functions, and use them to find the distribution of independent sums of random variables and random sums.
• Compute probabilities related to one-dimensional simple random walks on integer, in particular the probability of returning to its starting point and the probability of visiting one point before another.
• Compute quantities related to Galton-Watson branching processes, including the mean size of the population and the probability of ultimate extinction.
• Define renewal processes; find quantities related to Poisson processes, such as mean excess lifetime and current lifetime.

1.Review of conditional probability, probability distributions, random variables, means and variances. [2 lectures]

2.Independent random variables. Sums of independent identically distributed random variables. [1]

3.Probability generating functions and their application to sums of independent random variables and random sums. [2]

4.Random walks. Recurrence and transience. Gambler's ruin problem. [7]

5.Branching processes. The size of the nth generation and its probability generating function. The probability of extinction. [6]

6.Renewal processes. The counting processes and occurrence time processes. Renewal equations and real life applications including traffic flow. [6]

• Coursework; Weighting within unit 20%
• End of semester examination; Weighting within unit 80%

Feedback tutorials will provide an opportunity for students' work to be discussed and provide feedback on their understanding.  Coursework or in-class tests (where applicable) also provide an opportunity for students to receive feedback.  Students can also get feedback on their understanding directly from the lecturer, for example during the lecturer's office hour.

Rick Durret, Probability: Theory and Examples, Cambridge Series in Statistical and Probabilistic Mathematics, 2019. (recommended)

G.R. Grimmett and D.R. Stirzaker, Probability and Random Processes, Oxford University Press, 2000. (recommended)

S. Karlin and H.M. Taylor, A First Course in Stochastic Processes, Academic Press, 1975. (recommended)

The independent study hours will normally comprise the following. During each week of the taught part of the semester:

·         You will normally have approximately 60-75 minutes of video content. Normally you would spend approximately 2-2.5 hrs per week studying this content independently

·         You will normally have exercise or problem sheets, on which you might spend approximately 1.5hrs per week

·         There may be other tasks assigned to you on Blackboard, for example short quizzes or short-answer formative exercises

·         In some weeks you may be preparing coursework or revising for mid-semester tests

Together with the timetabled classes, you should be spending approximately 6 hours per week on this course unit.

The remaining independent study time comprises revision for and taking the end-of-semester assessment.