[backport] SI-8677 Duration: Zero - Inf should be MinusInf

Fixes #8677. Add basic tests.
This is a backport from 2.12.x of dead39dc5f21c6eac41788e93426c50ddd398c24.
Merge pull request #3866 from Blaisorblade/issue/8677-backport

SI-8677 Duration: Zero - Inf should be MinusInf