This can silently do the wrong thing if you add e.g. 1.2 to 2.3. I suggest doing something like:

    debug_assert!(other.fragment_index == num::zero() || other.glyph_index == num::zero());