Pareto Optimality is a situation where resources are allocated for production in such a way where nobody can be made better off by a change in the composition of production without somebody else being made worse off. You will notice that, on a particular line, the change is in the composition of output, not the distribution of resources between individuals. Everybody presumably gets a package of goods in the new production system that he or she feels is equally valuable as the package received under the old distribution of productive resources. For example, we now get four shirts and three shoes, which we think is as equally valuable as three shirts and four shoes. If I like shoes more than shirts, I can trade with the person who likes shirts more than shoes.
Nobody is made poorer by the change.
He did not consider this to be an increase in welfare. However, he did suggest that total welfare would be increased if the structure of production could be changed so that the first group got more of what they wanted while they still had enough extra to compensate the other group for the losses they absorbed in the changes needed to move to that higher level of community welfare. I don't think Pareto actually required that that compensation must be paid, since payment is a political question while the ability to make the payment is an economic question. But none of this had to do with impoverishment. I suppose that growth could impoverish one group if the new Pareto Optimal welfare line produced enough for compensation to be paid but it was not paid.