In 1447 he married Anne, the eight-year-old daughter of Richard of York. However, in the Wars of the Roses, he remained loyal to Henry VI against the Yorkists. He was imprisoned at Wallingford Castle when York briefly seized power after the First Battle of St Albans in 1455. In 1458 he participated in The Love Day, an attempt at reconciliation between the rival factions. He was a commander at the Lancastrian victories at the Battle of Wakefield in 1460 and the Second Battle of St Albans in 1461, and in the decisive defeat at Towton in 1461. He fled to Scotland after the battle, and then joined Queen Margaret in her exile in France. He was attainted in 1461, and his estates were given to his wife, who separated from him in 1464. During the brief period of Henry VI's restoration he was able to regain many of his estates and posts.