Born in Towcester, Parish joined the Academy at Aston Villa ahead of the 2006–07 season. He became the regular goalkeeper for Villa's under-18 side that won the 2007–08 Premier Academy League title and reached the semi-final of the FA Youth Cup the same year. While with the club, he represented the England under-20 team, playing in the 2009 FIFA U-20 World Cup. England manager Brian Eastick said that Parish was one of the best players at the tournament. Having travelled with the first-team squad during pre-season and in the Europa League, Parish became the regular goalkeeper for Aston Villa's reserve team during the 2010–11 season.