Born in Dresden, Germany in 1885, Kaye came to Neemah, Michigan with his parents when he was only three years old. In 1904 he moved to New York for a short time and there discovered his passion for art. Kaye and his mother moved to Germany where he studied engineering. He returned to the United States around 1914 and worked successfully as an engineer in the Midwest until the Stock Market Crash of 1929. He began painting "trompe-l'œil" oils in the tradition of the great 19th-century artists, William Harnett, Peto and Haberle. Many of these works include coins and bills. Because the United States enacted a ban on painting currency in 1909, Kaye did not sell his paintings but gave them as gifts to family members and close friends. He also created ink drawings of currency, and made etchings after Rembrandt, Whistler and Picasso, as well as other artists. He expanded on these prints with compositions of his own in ink and gouache.