MSc ACS: Data and Knowledge Management

Year of entry: 2023

Course unit details:
Representation Learning

Course unit fact file
Unit code COMP61021
Credit rating 15
Unit level FHEQ level 7 – master's degree or fourth year of an integrated master's degree
Teaching period(s) Semester 1
Offered by
Available as a free choice unit? Yes

Overview

A major component of machine learning and AI is dealing with extracting the useful information underlying data, in particular, for high dimensional data. The recent success of deep learning is also attributed to effective data representation. As an emerging area in machine learning, representation learning can extract features from raw data, discover explanatory factors of variation behind data and tackle tough issues arising from high dimensional data. Representation learning has been successfully applied to many domains such as computer vision, audio/speech information processing, natural language processing/understanding, robotics and a variety of medical applications.


In this course, we will consider how to develop core algorithms for representation learning and gain insights into these algorithms from theoretical and empirical perspectives. We will demonstrate how essential algorithms are developed in a systematic way as well as how important algorithms can be applied through the use of examples and real world problems. We will cover related key approaches from machine learning, statistics and deep neural networks in this advanced machine learning course unit.

Pre/co-requisites

Unit title Unit code Requirement type Description
Foundations of Machine Learning COMP61011 Pre-Requisite Compulsory

Aims

This course unit aims to introduce students to classical and state-of-the-art approaches to representation learning and provides experience of research such as literature review and self-learning from research papers. In particular, transferable knowledge/skills essential to original researches are highlighted in this course unit.

Learning outcomes

On successful completion of this unit, a student will be able to:

  • understand the general motivation and main ideas underlying representation learning
  • describe the curse of dimensionality and its implication in different learning paradigms
  • understand the advantages and the disadvantages of the learning algorithms studied in the course unit and decide which is appropriate for a particular application
  • derive the linear representation learning algorithms from scratch
  • apply the learning algorithms studied in the course unit to simple data sets for feature extraction related applications and visualisation of high-dimensional data
  • implement the core learning algorithms studied in the course unit as well as apply those to real-world datasets
  • evaluate the performance of the core learning algorithms studied in the course unit and whether such a learning algorithm is appropriate for a particular problem
  • understand and appreciate main ideas underlying state-of-the-art representation learning algorithms
  • make a connection between representation learning and other relevant areas in machine learning

Syllabus

  • Foundation: representation learning background and mathematics essentials
  • Linear models: principal component analysis and canonical correlation analysis
  • Manifold learning: overview, MDS, ISOMAP and LLE algorithms
  • Autoencoder: motivation, theoretical foundation, encoder/decoder architecture, evidence lower bound (ELBO) and variational  autoencoder (VAE)
     

Teaching and learning methods

Lectures

three hours per week (5 weeks)

Laboratories

three hours per week (5 weeks)

Employability skills

Analytical skills
Innovation/creativity
Project management
Oral communication
Problem solving
Research
Written communication

Assessment methods

Method Weight
Written exam 50%
Written assignment (inc essay) 50%

Feedback methods

In general, feedback is available for the assessed work.

For coursework, the feedback to individuals will be offered.

For exam, the general feedback to the whole class will be given in writing.

Study hours

Scheduled activity hours
Lectures 15
Practical classes & workshops 15
Independent study hours
Independent study 120

Teaching staff

Staff member Role
Ke Chen Unit coordinator

Return to course details