Optimized reciprocal space convolution by caching scale factors