Remove categories only containing files

Also avoid hitting the server twice

Bug: 68705

Change-Id: Idaa1bb7983e6b052e83e258d2b2b588c4385d62c
Merge "Remove categories only containing files"