One of the members of the group wants to know the origin of the phrase. Popular recordings were made by Cantor and by Billy Murray and by Billy Jones. The song became a hit, then a mega-hit, then a phenomenon. In England, the sheet music sold more than half a million copies during its first month of sales. Unlike most popular songs of the era, it never faded in popularity and even today it is one of the few songs that you can count on virtually anyone to know.
Legend has it that the songwriters overheard a Greek fruit stand owner utter the famous phrase.
It may be true, but the Greek fruitier was also a stock character in Vaudeville and it is just as likely that it was the song was inspired by a Vaudeville routine or written as yet another in a tradition of dialect songs. However it was inspired, you can safely say the song originated the phrase. They won and were awarded a share of the song's profits. Actually the song seems to be a pastiche of several popular pieces of music. The song everyone knows is actually just the chorus of the original song.