BSc Computer Science and Mathematics

Topic 1: Empirical risk minimization, regularisation; bias/variance theory and the relation to overfitting; probabilistic view: likelihood vs loss, introducing exponential families.

Topic 2: Information theory: KL-divergence vs. cross-entropy, mutual information; the view of ML as compression.

Topic 3: Optimization theory (calculus). Why GD?  What are convex and non-convex functions? How do gradients inform how we optimize a function? How can we use second order properties? How can we prove whether a method will converge?

Topic 4: Dimensionality reduction (matrix algebra). “refine, denoise, and visualise your data”.  Data usually has limited degrees of interest, living on a low-dimensional manifold within a high-dimensional space. This topic will introduce students to matrix-algebra-intensive methods used to learn feature dimensions that can aid in your model fitting process. Examples include PCA, spectral embedding, Fisher discriminant analysis, etc. These allow visualisation, denoising and enhancing the separation of data for classification.

COMP24112 is a pre-requisite.

This course introduces the theoretical foundations of supervised machine learning. We aim at students who want to move beyond just using models, to understanding why they work . This will be a rigorous mathematical and conceptual treatment that may require developing your knowledge of probability theory, matrix algebra, and calculus. You'll learn how we formalize learning problems, interpret performance through the lens of probability theory, and reason about why some models generalize while others do not. We focus on foundational ideas—risk, generalization, capacity, and trade-offs—that underlie all learning systems, regardless of the algorithm used. This could be seen as preparing you to engage with modern research literature, perhaps as an enabler toward a research career, in industry or academia. 

ILO 1: Define supervised learning problems using probabilistic assumptions and notation, identifying appropriate loss functions and models for a given scenario.

ILO 2: Describe how Rademacher complexity provides a framework for data-dependent generalisation bounds, and compute the bounds for learning scenarios with linear models.

ILO 3: Critically assess the limitations of classical learning theory in the context of modern machine learning models (e.g., overparameterized models).

ILO 4: Evaluate/critique how model combination affects generalization error, and describe learning algorithms that support this.

ILO 5: Describe and analyze the behavior of gradient descent algorithms using formal mathematical properties, such as convexity and Lipschitzness.

ILO 6: Derive and compare theoretical frameworks for understanding generalisation error, including: bias–variance, approximation–estimation, and PAC bounds.

Week 1 – The Foundations of Supervised Learning 
This week lays the groundwork by formalizing supervised learning. We ask: what are we optimizing, over what space, and why? By the end, you will understand learning as the problem of finding a function that performs well—not just on the training data—but on the unknown probability distribution that generated it.

Week 2 – The Probabilistic View of Learning 
This week shows how common loss functions arise naturally from probabilistic modeling assumptions. By deriving squared and cross-entropy losses from Gaussian and Bernoulli likelihoods, we see why these choices are principled. The connection between negative log-likelihood and KL divergence clarifies what it means for a model to “fit” the data.

Week 3 – The Problem with Learning a Model 
Here the focus turns to generalization failure and overfitting. The bias–variance decomposition is derived to separate sources of error. Regularization and the double-descent phenomenon illustrate how model capacity influences performance.

Week 4 – Combining Models 
We explore how ensembling multiple hypotheses can reduce variance and improve generalization. The lecture introduces Bagging, majority-vote theory, and the bias–variance–diversity decomposition. These ideas provide both a theoretical and intuitive foundation for why ensembles work.

Week 5 – Understanding the Sources of Risk 
This week examines how hypothesis-class complexity drives the trade-off between approximation, estimation, and optimization error. The formal decomposition clarifies why larger models can both help and hinder generalization. Examples link these abstract ideas back to practical model families.

Week 6 – How to be Probably Approximately Correct 
The course introduces Hoeffding’s inequality and PAC learning as a framework for probabilistic generalization guarantees. We learn to bound the gap between empirical and true risk. The session closes with a discussion on the realism and limits of PAC assumptions in modern ML.

Week 7 – Optimization and the Learning Algorithm 
Attention shifts from theory about models to the algorithms that find them. We examine empirical risk minimization as an optimization problem and how algorithmic choices affect generalization. Gradient descent is introduced conceptually as the foundation for modern learning.

Week 8 – A Formal Introduction to Gradient Descent 
This week develops the mathematical underpinnings of gradient descent. We study convergence under convexity, step-size selection, and Lipschitz continuity. The aim is to understand when and why gradient-based optimization actually finds a good solution.

Week 9 – Gradient Descent in Overparameterized Linear Regression 
The analysis focuses on linear regression with more parameters than data points. We see that gradient descent implicitly selects the minimum-norm solution, introducing the idea of algorithmic bias. This provides a bridge between optimization dynamics and generalization behaviour.

Week 10 – A Measure of Capacity: Rademacher Complexity 
Here we meet Rademacher complexity as a formal measure of a function class’s ability to fit random noise. They learn how this quantity bounds generalization error in data-dependent ways. The material extends PAC ideas to richer hypothesis spaces.

Week 11 – Bounding Rademacher Complexity in Practice 
The final week applies the Rademacher framework to simple hypothesis classes such as linear models. Worked examples illustrate how to compute and interpret capacity bounds. The course concludes by connecting these ideas back to the central question of why learning works.

Unit will consist of 4 major topics, delivered in 2-week blocks before Easter.  Each topic will consist of videos to watch, and readings to cover, before the interactive sessions. The interactive sessions will act to reinforce the videos/readings.  Weekly MCQs in class will be used as formative and summative assessments.

After Easter, a series of carefully selected classic research papers will be read week by week in groups, introducing students to the methods in how to read/interpret research results.  Presentations of the papers will consolidate the depth of understanding.

Poster Presentation

Correct answers discussed the following week

Selected chapters: Machine Learning, A Probabilistic Perspective by Kevin Murphy;

Selected chapters: Probability in Data Science, by Sidney Chan

Tuition fees for home students commencing their studies in September 2027 will be £10,050 per annum. Tuition fees for international students will be £39,700 per annum.

For general information please see the undergraduate finance pages.

All students should normally be able to complete their programme of study without incurring additional study costs over and above the tuition fee for that programme. Any unavoidable additional compulsory costs totalling more than 1% of the annual home undergraduate fee per annum, regardless of whether the programme in question is undergraduate or postgraduate taught, will be made clear to you at the point of application. Further information can be found in the University's Policy on additional costs incurred by students on undergraduate and postgraduate taught programmes (PDF document, 91KB).

The University of Manchester is committed to attracting and supporting the very best students. We have a focus on nurturing talent and ability and we want to make sure that you have the opportunity to study here, regardless of your financial circumstances.

For information about scholarships and bursaries please visit our  undergraduate student finance pages .