Lec 20: Non-Parametric Function Approximators in Reinforcement Learning
Channel: Abhishek Gupta
Date: 30/03/2020

Welcome to the 20th lecture of ECE 8851: Reinforcement Learning, where we will discuss Non-parametric function approximators. In this lecture, we will cover various techniques used for non-parametric function approximators, including Local averaging, Local modeling, Global modeling, and Penalized modeling.

Local averaging is a technique that involves partitioning the estimate by using regression trees, Nadaraya Watson Kernel estimate, and k-nearest neighbor estimate. These techniques are used to estimate a function by averaging the response of similar input values.

Local modeling extends the Nadaraya Watson kernel estimate to a parametric function approximator. It uses the weighted sum of a fixed number of basis functions to approximate the unknown function.

Global modeling extends regression trees with basis functions in each rectangular region. This technique helps in finding a global approximation of the unknown function by breaking the input space into smaller regions and modeling each region using a linear function.

Penalized modeling extends global modeling with a penalty on derivatives of the function, splines in one dimension and multiple dimensions. This technique is used to overcome the problem of overfitting by smoothing the function.

In this lecture, we will discuss each of these techniques in detail, along with their advantages and disadvantages. We will also provide examples to illustrate the application of each technique.