CS 294: Deep Reinforcement Learning, Spring 2017

Instructors: Sergey Levine, John Schulman, Chelsea Finn
Lectures: Mondays and Wednesdays, 9:00am-10:30am in 306 Soda Hall.
Office Hours: TBD
Communication: Piazza will be used for announcements, general questions about the course, clarifications about assignments, student questions to each other, discussions about material, and so on. To sign up, go to the Piazza website and sign up with “UC Berkeley” and “CS294-112” for your school and class.
**Wait List**: If you are not a graduate student in EECS and are interested in taking this class, please fill out this form.

Table of Contents


This course will assume some familiarity with reinforcement learning, numerical optimization and machine learning. Students who are not familiar with the concepts below are encouraged to brush up using the references provided right below this list. We’ll review this material in class, but it will be rather cursory.

  • Reinforcement learning and MDPs
    • Definition of MDPs
    • Exact algorithms: policy and value iteration
    • Search algorithms
  • Numerical Optimization
    • gradient descent, stochastic gradient descent
    • backpropagation algorithm
  • Machine Learning
    • Classification and regression problems: what loss functions are used, how to fit linear and nonlinear models
    • Training/test error, overfitting.

For introductory material on RL and MDPs, see

For introductory material on machine learning and neural networks, see


The syllabus will be made available in early 2017.

Lecture Videos

The course may be recorded this year. John also gave a lecture series at MLSS, and videos are available:

  • Lecture 1: intro, derivative free optimization
  • Lecture 2: score function gradient estimation and policy gradients
  • Lecture 3: actor critic methods
  • Lecture 4: trust region and natural gradient methods, open problems


Relavent Textbooks

Previous Offerings

An abbreviated version of this course was offered in Fall 2015.