It is not a good idea to leave buffalo chicken dip out overnight. Bacteria can grow quickly on food left at room temperature for more than two hours. The U.S. FDA recommends discarding any perishable food left at room temperature for longer than two hours.