Although Clifford's later years were devoted to service in the north and fighting the Scots (he took part in the decisive English victory at Flodden in 1513) he fell out with the King on numerous occasions. Clifford's was not an easy-going personality; his abrasiveness caused trouble with his neighbours, occasionally breaking out in violent feuds. This was not the behaviour the King expected from his lords. Furthermore, Clifford had married a cousin of the King, yet Clifford's infidelity to her was notorious among his contemporaries. This also drew the King's ire, to the extent that the couple's separation was mooted. Clifford's first wife had died by 1511, and Clifford remarried. This was also a tempestuous match, and on one occasion he and his wife ended up in court accusing each other of adultery. Clifford's relations with his eldest son and heir, the eventual Henry Clifford, 1st Earl of Cumberland, were equally turbulent. Clifford rarely attended the royal court himself, but sent his son to be raised with the King's heir, Prince Arthur. Clifford later complained that young Henry not only lived above his station, he consorted with men of bad influence; Clifford also accused his son of regularly beating up his father's servants on his return to Yorkshire.