Revert "Simplify toEmpirical conversions to respect original type only"

This reverts commit 78f270a7e3b648dd901696d07d0104ed74add201.