  We explore the connections between Green's functions for certain differential
equations, covariance functions for Gaussian processes, and the smoothing
splines problem. Conventionally, the smoothing spline problem is considered in
a setting of reproducing kernel Hilbert spaces, but here we present a more
direct approach. With this approach, some choices that are implicit in the
reproducing kernel Hilbert space setting stand out, one example being choice of
boundary conditions and more elaborate shape restrictions.
  The paper first explores the Laplace operator and the Poisson equation and
studies the corresponding Green's functions under various boundary conditions
and constraints. Explicit functional forms are derived in a range of examples.
These examples include several novel forms of the Green's function that, to the
author's knowledge, have not previously been presented. Next we present a
smoothing spline problem where we penalize the integrated squared derivative of
the function to be estimated. We then show how the solution can be explicitly
computed using the Green's function for the Laplace operator. In the last part
of the paper, we explore the connection between Gaussian processes and
differential equations, and show how the Laplace operator is related to
Brownian processes and how processes that arise due to boundary conditions and
shape constraints can be viewed as conditional Gaussian processes. The
presented connection between Green's functions for the Laplace operator and
covariance functions for Brownian processes allows us to introduce several new
novel Brownian processes with specific behaviors. Finally, we consider the
connection between Gaussian process priors and smoothing splines.
