Weymouth, Dorset
Ellison was born in Weymouth, Dorset and as a youth played for various clubs in his home town. In September 1930, he joined Southampton as an amateur, signing a professional contract in December.