Abstract — Autonomous learning of robotic motion skills promises to allow robots to acquire large motion repertoires for interacting with the real world with minimal human intervention. However, most current robotic motion skill representations are compact and lacking in expressiveness. This limitation is due in large part to the difficulty of learning rich representations in the context of optimal control and reinforcement learning. We discuss our recent experiments on using the guided policy search method to learn multilayer neural network policies for robotic manipulation skills. The key idea in this approach is to turn the task of policy search into a supervised learning problem, so that known, reliable optimization algorithms can be used to train expressive function approximators with thousands of parameters. While our networks are small compared to those used for perception (two hidden layers with 40 units each), they are significantly more expressive than most controller representations in reinforcement learning. We present results for neural network control of a PR2 robot, and discuss future work on learning policies that directly map rich sensory input to joint torques using deep neural networks.