Fix comparison of frozen sources.

This whole block of code just moved down to where `both_sources` is used below.