The term became sexualized around 1681 when prostitutes referred to their pimps as daddies. Modern culture has further propagated that meaning.