Also short circuit within segment when score mode is avg
Make the ChildrenQuery and ParentQuery not rely on the fact that score of 0 means that there isn't a match.