The castle was sufficiently important that the Crown took an interest in its maintenance. Henry II of England had it garrisoned in 1165. In 1198 Roger Corbet re-built the tower, keep and curtain wall in stone. During the late 12th century a town or borough was founded in the large outer bailey. A royal grant of 50 marks was made in 1263 towards further building work when D-shape towers were added to the curtain wall. The town contained a church of St Margaret as well as the castle's church of St Nicholas, and the names of two thoroughfares, Castle Street and St Margaret Street are recorded. On the death of Beatrice Corbet in 1347, Caus passed to the Earl of Stafford. At its height, the borough had 58 burgesses resident in 1349, a year coinciding with the arrival of the Black Death in England.