Leaving pudding out overnight is generally not recommended. Pudding is a milk-based dessert and, as such, is susceptible to spoilage. Milk contains high levels of lactose, which can encourage the growth of bacteria. Bacteria can cause food poisoning, so it's best to err on the side of caution and refrigerate pudding after it has been made.