In 1201, King John gave the "Three Castles" title to Hubert de Burgh. Hubert was a minor landowner who had become King John's household chamberlain while still a prince, and went on to become an increasingly powerful royal official once King John inherited the throne. Hubert began to upgrade his new castles, starting with Grosmont, where he rebuilt the hall block in stone. Hubert was captured fighting the French in 1205 and, while he was imprisoned, King John took back the castles and gave them to William de Braose, one of Hubert's rivals. King John subsequently fell out with William and dispossessed him of his lands in 1207, but de Braose's son, also called William, took the opportunity of the chaos during the First Barons' War to retake the castles.