So, there is approximately no energy transfer in your situation other than that between the molecules' kinetic energy and intemolecular potential energy. Exactly what the temperature does when a gas expands depends on the interactions between the gas molecules themselves. If they don't change, the kinetic energies of the gas molecules won't change either. The speeds of the gas molecules remain constant, and the temperature of the gas does too. If the gas molecules were so close together that moving apart would give them a lower potential energy, then the molecules repel each other.
Expansion lowers the intermolecular potential energy.
This means that expansion releases energy from the intermolecular potential, and that energy is picked up by the molecules' kinetic energy. In other words, the gas molecules speed up and the temperature increases. If the gas molecules were feeling lonely before expending, that is, if moving apart gives the gas molecules higher potential energy, then the gas molecules attract each other. Expansion raises the intermolecular potential energy, lowering the molecules' kinetic energy. The molecules slow down and the temperature decreases.