I wanted them treated just as the normal maps here. The fact that they are sparsity maps should only have an impact on the construction of the sparsity.