  Motivated by the recent developments in terahertz spectroscopy using
pump-probe setups, we develop the theory of finite frequency nonlinear
electro-optical responses in centrosymmetric metals starting from basic time
dependent perturbation theory. We express the nonlinear current kernel as a sum
of several causal response functions. These functions cannot be evaluated using
perturbative field theory methods. Consequently, we associate each response
function with an imaginary time ordered current correlation function, which can
be factorized using Wick's theorem. The mapping between the response functions
and the correlation functions, suitably analytically continued to real
frequencies, is proven exactly. We derive constraints satisfied by the
nonlinear current kernel and we prove a generalized $f$-sum rule for the
nonlinear conductivity, all of which are consequences of particle number
conservation. We apply the theory to compute the gauge invariant nonlinear
conductivity of a system of noninteracting electrons in the presence of weak
disorder. As special cases of this generalized nonlinear response, we discuss
its third harmonic and its instantaneous terahertz Kerr signals. Our formalism
can be used to compute the nonlinear conductivity in symmetry broken phases
such as density waves and nematic states.
