two-mode quadrature correlation funciton for density matrix