Gray was born in Harrogate, North Yorkshire. He signed for Leeds United which carried on the family tradition of his father Frank Gray and his uncle Eddie Gray. One of Gray's first senior appearances came for Leeds United in the 1996 League Cup final, when he was 18 years old. Leeds lost 3–0 to Aston Villa, Gray was one of the standout performers on the day for Leeds despite a 3–0 defeat.