Miles has three siblings: brothers, Mason and Marshall, and sister, Madeleine. All three brothers participated in Duke's basketball program and won championships with the team there. Mason was drafted at pick 22 in the 2013 NBA draft by the Brooklyn Nets. His sister, Madeleine, plays volleyball at the University of Notre Dame.