All the food and gatherings happened in the big red barn next to the Big House