Properly handle when objects cannot be folded

Fix #579.