Revert "Re #6191. Recover performance while not using SpectraDetectorMap"

This reverts commit 32e01973ddfba8984ba1a381012e847376755ba6.
Merge branch 'feature/6191_remove_spectradetectormap' into develop