There is no problem in taking your grandparents along with you to Chitrakoot, but they might have a problem in riding the hills, where some of the very important temples are located. These places have the service of taking the aged people safely, still, it is not completely ok to take them with you. You can then visit the temples that are on the ground.