Don't keep the full info of all children in memory

Backport of f88021dbbc3b53e4a1460a79185835770514722e from master
Show a proper error message when trying to scan the filesystem for a non existing user

Backport of e8280c80da52399594ba116ef78bddf16b9e7f25 from master