Colville was born in Toronto, Ontario in 1920 and moved to Amherst, Nova Scotia with his family in1929, where he grew up. As a child Colville would reconstruct his father’s blueprints and transform them into three dimensional pictures. As a child Colville used to construct model planes and trains. Colville grew up around horses because his father and grandfather owned them. Horse and Train would be personal to him because of his past experience with horses and technology. In 1938, Colville began attending Mount Allison University in Sackville, New Brunswick and received his bachelor for fine arts in 1942. Colville was involved in the Second World War; he became a war artist in the Canadian military.