The more perfect union phrase from the Preamble to the United States Constitution is most likely a reference to the earlier union under the Articles of Confederation. The Constitutional Convention met in Philadelphia in 1787 to address problems with the Articles of Confederation, which had been ratified in 1781. The delegates at the Constitutional Convention decided to scrap the Articles and start fresh with a new constitution.
In an effort to make this new government more perfect than its predecessor, they included a number of checks and balances between different branches of government, as well as between state and federal governments. They also sought to protect individual rights by including them in the Constitution itself. And finally, they hoped that this new form of government would be more stable and durable than the old one.
It's impossible to know for sure who exactly came up with the phrase more perfect union. But it was probably someone who was familiar with both the Articles of Confederation and the Constitution, and who saw value in both documents. This person may have wanted to stress that although the Constitution was not perfect, it was still an improvement over what came before.