Only fade an entity on the sidebar if they're directly assigned, not if their assignment is implied via group membership