  Let $\| A \|_{\max} := \max_{i,j} |A_{i,j}|$ denote the maximum magnitude of
entries of a given matrix $A$. In this paper we show that $$\max \left\{ \|U_r
\|_{\max},\|V_r\|_{\max} \right\} \le \frac{(Cr)^{6r}}{\sqrt{N}},$$ where $U_r$
and $V_r$ are the matrices whose columns are, respectively, the left and right
singular vectors of the $r$-th order finite difference matrix $D^{r}$ with $r
\geq 2$, and where $D$ is the $N\times N$ finite difference matrix with $1$ on
the diagonal, $-1$ on the sub-diagonal, and $0$ elsewhere. Here $C$ is a
universal constant that is independent of both $N$ and $r$. Among other things,
this establishes that both the right and left singular vectors of such finite
difference matrices are Bounded Orthonormal Systems (BOSs) with known upper
bounds on their BOS constants, objects of general interest in classical
compressive sensing theory. Such finite difference matrices are also
fundamental to standard $r^{\rm th}$ order Sigma-Delta quantization schemes
more specifically, and as a result the new bounds provided herein on the
maximum $\ell^{\infty}$-norms of their $\ell^2$-normalized singular vectors
allow for several previous Sigma-Delta quantization results to be generalized
and improved.
