  Finite linear least squares is one of the core problems of numerical linear
algebra, with countless applications across science and engineering.
Consequently, there is a rich and ongoing literature on algorithms for solving
linear least squares problems. In this paper, we explore a variant in which the
system's matrix has one infinite dimension (i.e., it is a quasimatrix). We call
such problems semi-infinite linear regression problems. As we show, the
semi-infinite case arises in several applications, such supervised learning and
function approximation, and allows for novel interpretations of existing
algorithms. We explore semi-infinite linear regression rigorously and
algorithmically. To that end, we give a formal framework for working with
quasimatrices, and generalize several algorithms designed for finite problem to
the infinite case. Finally, we suggest the use of various sampling methods for
obtaining an approximate solution.
