The trick is learning how to work less by learning how to spend a lot less. Living cheap doesn't mean suffering and starving. You can live cheap and also enjoy a comfortable, plentiful life. But I like my job. There are some lucky people who have better jobs or who work at jobs where they do something they like.
Work can spoil anyting.
Cooking can be fun, but working as a cook in a busy restauant is hell. If you like your job now, you will like your job even more if you work less. Working less doesn't mean being unproductive. To make gardening pay, you would have to work like a farmer. But you can easily grow lots of vegetables, possibly enough to live on, simply by goofing around in your garden.