Loading or using `set` for an asset gives it a refcount of 1. The 1 reference is the fact that it has been loaded, ie 1 thing cares that it is loaded. Any asset with a refcount of 0 means it should be unloaded.

siondream also has a good (separate) point.