After being spotted playing for his school team, Fraser entered the youth system at First Division club Fulham at the age of 14 in 1967. He began his career as a midfielder, but switched to full back under reserve team manager George Cohen and signed his first professional contract in 1971. He made his senior debut during the 1971–72 Second Division season and gradually built on his appearance record year-by-year to play 25 games during the 1974–75 season. An injury to Les Strong allowed him a place in the starting lineup for the 1975 FA Cup Final, which was lost 2–0 to West Ham United. Fraser fell out of favour with manager Alec Stock during the 1975–76 pre-season and departed the club at the end of the season. He made 65 appearances and scored one goal as a first team player at Craven Cottage.