The Method of Least Squares - Linear Algebra
Channel: Nick Space Cowboy
Date: 2022-05-29

This video covers the method of least squares in linear algebra. A method to find a least squares solution to an over determined system.

Code: https://github.com/nkphysics/Computational-Linear-Algebra-/tree/master/Unit2/10_Least_Squares

numpy.linalg.lstsq documentation: https://numpy.org/doc/stable/reference/generated/numpy.linalg.lstsq.html

Table of Contents:
0:00 - Introduction 
0:58 - What causes the problem with an over-determined system
3:22 - The Method of Least Squares
5:01 - Method of Least Squares without Numpy
8:42 - Method of Least Squares with Numpy

We discuss how to find an approximate solution to an over-determined system (a linear system with more rows than columns). We can find an approximate solution to an over-determined system with a little bit of knowledge of sub-spaces and projections. We obtain an approximation to an over-determined system by using a method called "The Method of Least Squares" which plays a crucial role on the back end of may routines involving machine learning, AI, and model fitting. This approximation is often referred to as a "Least Squares Solution".