Store the object itself in the failed download set

Patches don't respond to "name".