This is where gravity comes in. In the Universe, matter and heat radiation are not isolated, because they are free to engage in large-scale motion. This activity is subject to gravitational forces, and so we must include the gravitational field as part of the total system. In the lab, where gravity is negligible, the equilibrium end state of a gas is a uniform distribution. But taking gravity into account changes everything.
Gravity is an attractive force that tries to pull matter into clumps.
The ultimate triumph of this process is when material falls together completely to form a black hole. Applying thermodynamics to gravity, the black hole can be seen as the equilibrium end state. Attempting to find the equations that link gravity with thermodynamics is taxing the best brains in physics. But for a clue to how these two processes might be related from the standpoint of the arrow of time, it helps to think of order and disorder not in terms of clumpiness and smoothness, but in terms of information. A totally disordered state needs only a few bits of information to describe it.