A lot of times, people have the impression that being in a position of leadership means that you get to make the important decisions and pick the things you want to do. There is some truth in that, but not nearly as much as you might think.
Leadership also means making unpopular decisions that have a long term goal in mind.
It means working over the weekend to finish personnel reviews so your subordinates get their raises. It means going to meetings and being accountable to managers above you for the actions of those employees. Even when their actions are not acceptable to you, you are responsible. It means dealing with that difficult person who has caused a problem between himself and another group. With the authority that comes with a position, there are responsibilities.