Under Collier, Jones supervised the offensive backfield and the ends, but was also the Browns' offensive play-caller. He directed the team from the press box on Collier's behalf because Collier was hard of hearing and could not do so himself. The Browns won their first six games at the beginning of Jones's coaching career in the 1963 season, although a late slump cost them a spot in the championship game. The team regrouped the following year, ending with a 10–3–1 win-loss-tie record and winning the NFL's eastern division. Cleveland went on to win the championship game against the Baltimore Colts in 1964. The Browns advanced to the championship game again the following year, but lost to the Green Bay Packers. During his tenure as a coach, Jones was the primary position coach for running backs Jim Brown and Leroy Kelly, both of whom are members of the Pro Football Hall of Fame. He also coached receiver Paul Warfield, another hall of fame member who helped propel the Browns to the 1964 championship.