{"id":665,"date":"2020-10-13T15:19:25","date_gmt":"2020-10-13T19:19:25","guid":{"rendered":"https:\/\/sites.bu.edu\/aolshevsky\/?page_id=665"},"modified":"2021-05-08T07:18:49","modified_gmt":"2021-05-08T11:18:49","slug":"foundations-of-reinforcement-learning","status":"publish","type":"page","link":"https:\/\/sites.bu.edu\/aolshevsky\/foundations-of-reinforcement-learning\/","title":{"rendered":"EC 700 A3, Spring 2021: Introduction to Reinforcement Learning"},"content":{"rendered":"

Course Description:<\/strong> Reinforcement learning is a subfield of artificial intelligence which deals with learning from repeated interactions with an environment.\u00a0 Reinforcement learning is the basis for state-of-the-art algorithms for playing strategy games such as Chess, Go, Backgammon, and Starcraft, as well as a number of problems throughout robotics, operations research, and other fields of engineering. In this course, we will study the fundamental principles of reinforcement learning.\u00a0 Our focus will be the main algorithms in the field: we’ll try to develop a mathematical understanding of why they work. While a moderate amount of coding will be required, the most important pre-requisite is mathematical maturity and a facility with constructing proofs.<\/p>\n

Instructor:\u00a0<\/strong>Alex Olshevsky<\/a><\/p>\n

Time:\u00a0<\/strong>MW 2:30-4:15<\/p>\n

Tentative Plan: <\/strong><\/p>\n

    \n
  1. Markov Decision Process and Dynamic Programming, Value Iteration and Policy Iteration.<\/li>\n
  2. Monte Carlo and Temporal Difference Methods. SARSA, DYNA, and Q-learning. Eligibility traces.<\/li>\n
  3. Sample Complexity Issues.<\/li>\n
  4. Using Deep Neural Networks for Approximation. Deep Q-learning. Approximation theorems, the benefits of depth, PAC-Bayes bounds for neural networks.<\/li>\n
  5. Policy Gradient, Actor-Critic methods.<\/li>\n
  6. Imitation learning, Inverse Reinforcement Learning, Distributed Reinforcement Learning (time permitting).<\/li>\n
  7. Frontiers of RL research: using LSTM, Attention models, Transformers with reinforcement learning (time permitting).<\/li>\n<\/ol>\n

    Prerequisites:\u00a0<\/strong>Some basic proficiency with Python. Strong familiarity with Markov chains. The most important prerequisite is mathematical maturity and a facility with constructing proofs.<\/p>\n

    Textbook: <\/strong>There will not be a single textbook four the course; rather, we will rely on lecture notes which will mix materials from different sources, most of which will come from research papers. In terms of textbooks, we’ll use bits and pieces from:<\/p>\n