They and others can tick off the hard times with ease , which mills closed when , when the major retail stores downtown packed up and left , and how the people left with them , dropping the city 's population to about 21,000 now , less than half what it was as recently as 1970 .