Pull out a branch in the inner loop of DoPoissonMultiplication.  For gcc 4.7, this does eliminate one conditional jump in the generated assembly.
Stop making builds debug by default.
In DoNoiseMultiplication, draw frequencies from a queue rather than assigning ranges to each thread.  This should ensure that the threads all finish simultaneously (important if we attempt to oversubscribe cores).
An unnecessary vector assign was taking perhaps more than 10% of CPU time; replace it with resize.