some refactoring on categorical node rebuilding, hopefully fix freezing issue

[#74295478]