Playing for local side, West Ham boys, Charles was spotted by former West Ham United goalkeeper and scout, Ernie Gregory who recommended him to chief scout Wally St Pier. Charles signed as a youth player in 1959 and signed professionally, aged 17 in 1963. He captained West Ham's ; the first black player to lead a first-class side to a major trophy. Charles made his debut for the first team in May 1963 against Blackburn Rovers. Injury prevented Charles from being part of the West Ham teams which won the 1964 FA Cup Final and the 1965 UEFA Cup Winners' Cup Final. Nevertheless, playing most of his games alongside Bobby Moore, he made 142 appearances for West Ham in all competitions scoring two goals; one a League Cup game against Grimsby Town, the other the single West Ham goal in a famous Manchester United 6–1 victory at Upton Park in which Manchester United won the 1966-67 Football League trophy. From the autumn of 1969 until spring 1970 injury restricted Charles to only five games. He left West Ham in the summer of 1971. He was offered a move to Orient then managed by former West Ham player, Jimmy Bloomfield, but because of a recurring hamstring problem he retired from football aged only 26 to run the family's market greengrocer's stall.