Reinforcement Learning 101: Learn RL by building an AI agent to win at Pacman.
Adapting, Evolving, Winning: An Introduction to Reinforcement Learning Techniques
Get notified when we launch 🚀
- Understand what reinforcement learning is and be able to distinguish between the two primary classes of methods: policy-based and value-based methods.
- Gain a comprehensive understanding of the challenges associated with reinforcement learning and be well prepared for the next steps in your learning journey.
- Be proficient in using the RL library rllib, understanding how to construct and train RL solutions across various benchmark problems.
- Be capable of building, training, and testing an end-to-end reinforcement learning algorithm to successfully 'WIN' at Pacman.
This course is designed as a practical introduction to reinforcement learning. We'll take a 'hands on' approach so we'll be focusing on building, training and testing rather than the maths and theory that underpin everything.
Reinforcement Learning is a hugely exciting area of Machine Learning and it's transforming how we build algorithms to solve task in all areas! From Large Language Models, to optimal control in engineering systems and robotics.
Understanding the core principles in Reinforcement Learning (RL) will give you the foundations to understand the current state of the art, the opportunities, risk and challenges in this important area of Machine Learning.
By completing this course, you'll build up a your own template code, that you can use to start applying RL to your own projects.
Prerequisites
-
This courses gives a very hands on introduction to Reinforcement Learning. You should be comfortable with the basics of using a Python library, but no special knowledge is required.
-
Occasionally, I'll introduce mathematical concepts and new notation. I'll ensure these are thoroughly explained but they aren't essential to the course. Whenever we use any specific mathematical notation, it will be clarified in detail.
-
While not essential, additional resources and courses are available for extra support. You can find these in the suggested courses listed below.
Section 1: Introduction to RL, Deep Learning & rllib
In this first section, we'll run through a high level overview of how reinforcement learning fits within the broader topic of machine learning. We'll also discuss how and why it's different from other types of learning like supervised learning.
We'll quickly give an introduction to the connections between deep learning (i.e. neural networks) and reinforcement learning. What this new set of methods has meant for the field, and how it's used in modern RL.
Finally, we will get our hands dirty and fire up rllib - a super Python library that lets you implement pretty advanced RL algorithms via a basic API. We will get you setup, and then look at our first example - "GRIDWORLD".
Figure 1. Simple installation of rllib gives high performance RL methods at your fingertips.
Section 2: Value Based Methods - Deep Q-Learning
In this section we're going to jump in and learn our first AI algorithm, which is a value based method called Q-Learning. We'll' show a practical implementation in rllib, and how we use deep learning methods along with this problem to start solving some tricky problems.
We are going to show this approach on a few benchmark problems to practically introduce the steps, methods and challenges.
Figure 2. Representation of Q-Learning for GRIDWORLD (thanks to NVidia)
Section 3: Policy Based Methods - A2C
Next we are going to look at policy based methods. We'll work our way through methods like REINFORCE with the high level principles, then quickly get on to implementing a methods called A2C (or Actor Critic)! This will use both the value based methods, and this new idea of policy based methods.
This section will focus on implementation and we'll benchmark the approach against those examples we see for DQN in the previous section.
Section 4: Build an Agent to WIN Pacman
Finally we'll bring it all together to build an agent to play and complete the wonderful classic game, Pacman.
I'll provide the code for you to download and test out yourself. This end to end build will give you really good clarity on the steps needed to build, train, test and deploy an AI agent, all around a fun classic game!
Figure 3. A bit of fun, but we are going to you to build a Pacman Algorithm!
🤖 Use AI to help you learn!
All digiLab Academy subscribers have access to an embedded AI tutor! This is great for...
- Helping to clarify concepts and ideas that you don't fully understand after completing a lesson.
- Explaining the code and algorithms covered during a lesson in more detail.
- Generating additional examples of whatever is covered in a lesson.
- Getting immediate feedback and support around the clock...when your course tutor is asleep!
