[backend] be more careful with channels and patchinfos, they must not get put into useforbuild

Fixes the event_useforbuild handling as side effect...