CS 294: Deep Reinforcement Learning, Fall 2017

If you are a UC Berkeley undergraduate student looking to enroll in the fall 2017 offering of this course: here is a form that you may fill out to provide us with some information about your background. Please do not email the instructors about enrollment: the form will be used to collect all information we need. This waitlist will be used as the official waitlist, not the one on CalCentral.
Instructors: Sergey Levine, Abhishek Gupta, Joshua Achiam



Spring 2017 Materials

Instructors: Sergey Levine, John Schulman, Chelsea Finn

Table of Contents

Lecture Videos

The course lectures are available below. The course is not being offered as an online course, and the videos are provided only for your personal informational and entertainment purposes. They are not part of any course requirement or degree-bearing university program.
For all videos, click here.
For live stream, click here.

Lectures, Readings, and Assignments

Below you can find an outline of the course. Slides and references will be posted as the course proceeds.

Prerequisites

CS189 or equivalent is a prerequisite for the course. This course will assume some familiarity with reinforcement learning, numerical optimization and machine learning. Students who are not familiar with the concepts below are encouraged to brush up using the references provided right below this list. We’ll review this material in class, but it will be rather cursory.

  • Reinforcement learning and MDPs
    • Definition of MDPs
    • Exact algorithms: policy and value iteration
    • Search algorithms
  • Numerical Optimization
    • gradient descent, stochastic gradient descent
    • backpropagation algorithm
  • Machine Learning
    • Classification and regression problems: what loss functions are used, how to fit linear and nonlinear models
    • Training/test error, overfitting.

For introductory material on RL and MDPs, see

For introductory material on machine learning and neural networks, see

John's lecture series at MLSS

  • Lecture 1: intro, derivative free optimization
  • Lecture 2: score function gradient estimation and policy gradients
  • Lecture 3: actor critic methods
  • Lecture 4: trust region and natural gradient methods, open problems

Courses

Relevant Textbooks

Previous Offerings

An abbreviated version of this course was offered in Fall 2015.