MSc Machine Learning / Course details

Reinforcement Learning (COMP64202) is an advanced artificial intelligence course that explores how machines can learn to make sequential decisions through trial and error interaction with their environment, similar to how humans learn. The course covers both theoretical foundations and cutting-edge applications in areas such as game playing, robotics, and resource management.

The unit aims to provide an explanation of the key ideas and algorithms of reinforcement learning (RL). This course is meant to provide both an introductory and advanced treatment of reinforcement learning, emphasizing foundations and ideas as well as the latest developments and mathematical proofs. This course aims to make the work accessible to the broadest possible audience in artificial intelligence, control engineering, operations research, psychology and neuroscience.

 Knowledge and understanding:

1. List the strengths and limitations of modern deep RL approaches.

2. Explain the underlying concepts of RL methods and how they differ from each other.

3. Derive the objectives and constraints of selected algorithms.
 

Intellectual skills: 4. Analyze new tasks to decide which algorithms/architectures to apply. 5. Compare and evaluate different RL approaches for specific problem contexts.
 

Practical skills: 6. Implement selected RL algorithms and architectures. 7. Apply RL methods to solve practical problems.
 

Transferable skills and personal qualities: 8. Break down complex problems into manageable components. 9. Evaluate technical approaches against practical constraints. 10. Communicate technical concepts and results effectively.
 

The course covers the following topics:

• Fundamentals of Markov Decision Processes (MDPs)
• Dynamic Programming methods
• Monte Carlo methods
• Temporal-Difference learning
• Function approximation and Deep Reinforcement Learning
• Policy gradient methods
• Actor-critic algorithms
• Model-based Reinforcement Learning
• Partial observability and multi-agent systems
• Real-world applications and case studies

The learning and teaching in this course combines three complementary components. Lectures provide the theoretical foundations and key concepts of reinforcement learning. Tutorial sessions focus on solving theoretical exercises to deepen understanding and develop problem-solving skills. Computer lab sessions offer hands-on experience implementing and experimenting with reinforcement learning algorithms, allowing students to see the practical challenges and opportunities of these methods.

Feedback is provided through:

• Cohort and individual feedback for the practical assignment when marks are returned
• Discussion during tutorial sessions
• Peer feedback during group activities
• Office hours with teaching staff

An Introduction to Reinforcement Learning, R.S. Sutton and A.G. Barto. MIT Press, 2018.

Algorithms for Reinforcement Learning, C. Szepesvari, Morgan and Claypool, 2010.

This course unit is at Level 6 with 15 credits (7.5 ECTS). The unit is taught by Dr. Wei Pan from the School of Engineering.