Lord Grimthorpe, 1816-1905