Kempster was from Burford in Oxfordshire, England. He sold Cotswold stone from his quarry at Upton, near Burford, to rebuild London after the Great Fire of London in 1666. He was also Christopher Wren's master mason during the rebuilding of St Paul's Cathedral in London. His County Hall, built 1678–82 in Abingdon, is now the Abingdon County Hall Museum.