Revert "Split up Integral pointers for residual calculations"

This reverts commit 8358b8bda4cce6ea1f1e6f94ea959cecceae62d9.