refactored data processing. fixed inconsistent test data processing bug.