Usually there's a good reason, but the animal can't tell you what that reason is. You have to get into the animal's head and figure it out for yourself. Draft animals must be worked regularly to keep their bodies in condition and to remind them of their training. Draft animals, like the family cat or dog, require regular health care, veterinary checks, and vaccinations. They need frequent hoof trimming and maybe shoes.
Draft animals eat, even when there's no work to be done.
You can't just drain the fuel and store them away for the season. If you like to be away from home often, you'll need to find a reliable friend, relative, or neighbor willing and able to take care of your livestock. If you have only an acre or two, you don't need a pair of heavy horses or hefty oxen, but you might do nicely with draft ponies or mules. The bottom line is that working with draft animals involves a trade-off of expense vs. time. Your rewards are the satisfaction that you are living closer to nature and the serentiy that comes with taking more notice of the natural world around you.