In the reign of Richard III his father's loyalty was questioned to the house of York as support grew for Henry of Richmond (later Henry VII of England). Around the time of Richard's usurpation and Buckingham's rebellion in 1483, as a way of keeping Rhys ap Thomas's loyalty, Richard demanded an oath of allegiance from Gruffydd's father as well as demanding that his young son Gruffydd be in his own custody. Rhys assured Richard of his loyalty but refused to hand over young Gruffydd, who was only four or five at the time.