Gzowski was born in Saint Petersburg to a noble Polish father, Count Stanisław Gzowski, who was then serving as a Captain in the Russian Imperial Guard. During the Polish November Uprising he served as combat engineer with the rank of podporuchik in the Polish army under command of Józef Dwernicki against the Russians. After their defeat he escaped to Austria and as an unwanted political prisoner was deported by the Austrians to the USA. Later his family also emigrated. He knew no English, but began to study law and was admitted into practice. His father was an engineer, and as this became his primary interest, Kazimierz became involved in railway construction in the United States. Eventually he was hired as an engineer to help in the construction of the New York and Erie Railway.