BSc Computer Science and Mathematics

The first part of this unit continues the development of Probability theory from Year 1 and provides an important basis for many later courses in Probability and Finance.   

The second part of this course unit provides students with the methodological foundations in model-based statistical learning, in particular likelihood estimation and inference and Bayesian learning. The theoretical and methodological discussions are complemented by practical computer application. 

The first part of the course studies probability calculus for univariate and multivariate random variables. This includes topics such as moment generating functions, as well as sampling and convergence. The second part explores statistical learning methods, focusing on information entropy, likelihood-based inference and testing, and Bayesian methods. 

The first part of this unit continues the development of Probability theory from Year 1 and provides an important basis for many later courses in Probability and Finance.  

The second part of this course unit equips students with the methodological foundations of model-based statistical learning, focusing on information entropy, likelihood estimation and inference, as well as Bayesian learning. During the weekly tutorials, some problems are solved using software such as R.

This course unit forms the core of Theme C and is delivered over two semesters. It includes three contact hours each week, consisting of two or three hours of lectures and one tutorial class every two weeks. Teaching materials will be available on Canvas for reference and review. 

On the successful completion of the course, students will be able to:

• calculate marginal distributions of and conditional distributions associated with multivariate random variables 
• describe a range of parametric families to model their probability distribution 
• calculate expectations and conditional expectations 
• evaluate the distribution of functions of random variables 
• use the properties of generating functions to derive moments of distributions, distributions of sums of random variables and limit distributions for sequences of random variables.
• apply model-based approaches in statistical data analysis; 
• derive maximum likelihood estimates and compute corresponding confidence intervals; 
• perform statistical testing from a likelihood perspective; 
• solve standard modelling and inference problems from a Bayesian point of view; 
• use R to apply techniques from the course on actual data.  

Syllabus: 
Semester 1: 
Chapter 1: Random Variables (4 lectures) Definition of Events and their probabilities; definition of Random variables and their distributions; features of Discrete and Continuous random variables; Functions of random variables and mixed random variables. 

Chapter 2: Multivariate random variables (6 lectures) Bivariate Distributions; Independence; Sums of several variables; Conditional distributions; the bivariate transform. 

Chapter 3: Expectation (6 lectures) Expectation of a univariate random variable; Variance and higher Moments; Expectation of a bivariate random variable and conditional expectation; Probability generating functions; Moment generating functions; Sums of random variables using generating functions. 

Chapter 4: Sampling and convergence (6 lectures); The sample mean; Central limit theorem; Chebyshev's Inequality; Poisson Limit Theorem and characteristic functions; Introduction to the multivariate normal distribution. 

Semester 2: 

Section I - Likelihood 

- Entropy foundations: Shannon and differential entropy, cross-entropy, Kullback-Leibler (KL) divergence, expected Fisher information, minimum KL divergence and maximum likelihood. 

- Likelihood-based estimation: Likelihood function, regular models, score function, maximum likelihood estimators (MLE), invariance principle, relationship to ordinary least-squares estimation (OLS), observed Fisher information. 

- Quadratic approximation and normal asymptotics: Quadratic approximation of log-likelihood function and normal distribution, quantifying the uncertainty of MLEs using Fisher information, (squared) Wald statistic, normal confidence intervals, non-regular models. 

-  Likelihood-based inference: Likelihood-based confidence interval, Wilks log-likelihood ratio statistic, likelihood ratio test, generalised likelihood ratio test, optimality properties. 

Section II - Bayes 

- Conditioning and Bayes rule:  Conditional probability, Bayes’ theorem, conditional mean and variance, conditional entropy and chain rules, complete data log-likelihood, observed data log-likelihood, learning unobservable states using Bayes theorem. 

- Principles of Bayesian learning:  Prior and posterior probabilities and densities over parameters, marginal likelihood, sequential updates, summaries of posterior distributions and credible intervals, Bayesian and frequentist interpretation of probability. 

- Standard models: Beta-binomial model (for a proportion), normal-normal model (for the mean), inverse-gamma-normal model (for the variance), properties of Bayesian learning. 

- Bayesian model comparison: Log-marginal likelihood as penalised likelihood, model complexity, Bayes factor, Schwarz approximation and Bayesian Information Criterion (BIC), Bayesian testing using false discovery rate. 

- Choosing priors and optimality properties: default priors, uninformative priors, empirical Bayes, shrinkage estimation, James-Stein estimator, Frequentist properties of Bayesian estimators, optimality of Bayes inference (e.g. Cox theorem). 

This course unit forms the core of Theme C, and as such is delivered over 2 semesters.  Teaching is composed of two hours of lectures per week, and one tutorial class per fortnight.  Teaching materials will be uploaded to Blackboard for reference and review.

Other: one mid-term online timed test and one mid-term written test, 40-50 mins each.

Marked scripts within a week of the in-class test and automatically given feedback following the online test. Generic feedback made available after marks are released.

Robert E. Gaunt. MATH27720 Probability 2 Lecture Notes.  Article URL: https://eu-st01.ext.exlibrisgroup.com/…

MATH27720 Statistics 2 Lecture Notes. Korbinian Strimmer. Website URL: https://strimmerlab.github.io/…

Probability and Distribution Refresher. Korbinian Strimmer. Website URL: https://strimmerlab.github.io/…

Matrix and Calculus Refresher. Korbinian Strimmer. Website URL: https://strimmerlab.github.io/…

Sahu, Sujit K. (2024). Introduction to probability, statistics and R: foundations for data-based sciences. Springer. ISBN: 9783031378652

Held, Leonhard, author. (2020). Springer,. ISBN: 978

Heard, Nicholas. (2021). An introduction to Bayesian inference, methods and computation. Springer. ISBN: 9783030828080

Gelman, Andrew. (2014). Bayesian data analysis. CRC Press. ISBN: 9781439840955

Jaynes, E. T. (2003). Probability theory: the logic of science. Cambridge University Press. ISBN: 9780511790423

Diaconis, Persi. (2018). Ten great ideas about chance. Princeton University Press. ISBN: 9781400888283 

Tuition fees for home students commencing their studies in September 2027 will be £10,050 per annum. Tuition fees for international students will be £39,700 per annum.

For general information please see the undergraduate finance pages.

All students should normally be able to complete their programme of study without incurring additional study costs over and above the tuition fee for that programme. Any unavoidable additional compulsory costs totalling more than 1% of the annual home undergraduate fee per annum, regardless of whether the programme in question is undergraduate or postgraduate taught, will be made clear to you at the point of application. Further information can be found in the University's Policy on additional costs incurred by students on undergraduate and postgraduate taught programmes (PDF document, 91KB).

The University of Manchester is committed to attracting and supporting the very best students. We have a focus on nurturing talent and ability and we want to make sure that you have the opportunity to study here, regardless of your financial circumstances.

For information about scholarships and bursaries please visit our  undergraduate student finance pages .