Colville had to go to England for his cure, and thus missed the battle of Salamanca, but returned to the Peninsula in October 1812 and commanded the 3rd division in winter quarters until superseded by the arrival of General Picton. He commanded his brigade only at the Battle of Vitoria, where he was slightly wounded, but was specially appointed by Lord Wellington to the temporary command of the 6th division from August to November 1813, when he reverted to the 3rd division, which he commanded at the battles of the Nivelle and the Nive. He was again superseded by the arrival of Sir Thomas Picton, but in February 1814 Lord Wellington appointed him permanently to the 5th division in the place, of Sir James Leith. With it he served under Sir John Hope in the siege of Bayonne, and it was Colville who superintended the final embarkation of the last English troops left in France.