Let me give you a peek at the other fellow's yard. Being self-employed in today's world is pretty tough. The perks of working for other people are virtually non-existent. The worst, however, is the government. You put in 18 hour days, six and often seven days a week.
Vacations are so rare that a trip to the mall can be a special treat.
Yes, he failed to understand you. But, just as equally, you failed to understand him. That is intolerance, on both sides. It is not the stuff of concentration camps. It is merely two people who don't understand one another, and might have been cured, had one of you understood the other, or taken the time to explain.