Kind of. While you can dedicate a good chunk of time just farming, gathering, cooking, livestock, and so on, I'm afraid you may need to progress on the story to unlock more buildings. The Fowler, which is the hen house, needs a type of wood only found in forests, which you open in Chapter 3. You can enjoy your farm, but you will need the main story to unlock its potential to the fullest.