His rise to dominance in the kingdom began with the disastrous defeat of his elder brother Archibald Douglas, 4th Earl of Douglas, at the Battle of Homildon Hill in September 1402, where he was captured by the English. James was now acting head of the main branch of the powerful Douglas family and was left to maintain their influence in southern Scotland. His successes in this regard preserved Douglas influence until the return of his elder brother from captivity in 1409, at which time he assumed the role of councillor and was rewarded with the grant of extensive estates. The most important of these was the stronghold of Abercorn Castle, which would become his principal residence for the rest of his life.