As a child, he saw his father, grandmother and great-grandfather, as well as members of their wider family, executed as Judaizers at the behest of the Spanish Inquisition; his mother was acquitted but died of the plague when he was 15 years old.
Si fëmijë, ai pa ekzekutimin e babait, gjyshes dhe gjyshit, anëtarë të familjes së gjerë, me urdhër të Inkuizicionit Spanjoll; nëna e tij u lirua por vdiq në një fatkeqësi kur Vivesi ishte 15 vjeçar.