Yes! It is worth it to visit the Chatsworth House. With many great things to do in the area, it is recommended to stay at least one night.