Oliver was born in Astoria, Queens. In 2001, while field reporting on a landlord-tenant dispute, he was involved in a heated exchange live on air with "Good Day New York" anchor Jim Ryan. Less than two months later, on the morning of September 11, Oliver had been on assignment in Lower Manhattan when the attack on the World Trade Center occurred. His live report was among the earliest news coverage of the event. He would retire from Fox the following year and teach journalism at Columbia University, Hunter College and New York University.