After two years in the reserves, Kelsey made his first-team debut against Charlton Athletic on 24 February 1951 at Highbury, after Swindin had received an injury; However, Kelsey's first start was not an auspicious one, as Arsenal lost 5–2, their heaviest defeat at home since November 1928. Kelsey made a total of four appearances that season, but was dropped once Swindin returned and did not feature in Arsenal's 1951–52 run to the FA Cup final.