Not fetching relations before touching them

This should fix #3583

Didn't know how to write a proper test.