Many castles were originally built from earth and timber , but had their defences replaced later by stone .
Many castles were built with earth and wood at first often using manual labor , and then had their defences replaced by stone instead .