Silliman was a 6'6" forward from West Point, where he played for coach Bob Knight and took Army to the NIT Semi-Finals in 1964, 1965, and 1966. He participated in the 1968 Summer Olympics and won a gold medal as captain of the United States national basketball team. He also played for the United States men's national basketball team at the 1967 FIBA World Championship and the 1970 FIBA World Championship. He later played one season (1970–71) with the Buffalo Braves of the National Basketball Association. Silliman scored 91 points in 36 NBA games.