Some may have lost social privilege, and took refugee in areas south to Yangtze River