On March 30, 1780, Scott arrived in Charleston just as Clinton was laying siege to the city. He was captured when the city surrendered on May 12, 1780, and was held as a prisoner of war at Haddrell's Point near Charleston. Although he was a prisoner, he was given freedom to move within a six-mile radius and was allowed to correspond and trade with acquaintances in Virginia. With the death of William Woodford on November 13, 1780, he became primarily responsible for the welfare of the Virginia troops at Haddrell's Point. He requested his parole on account of ill health on January 30, 1781, and in late March, Charles Cornwallis granted the request.