They are the guardians of the game and, like it or not, they should be in control. Step two involves the quality of referees. Now, we all know they are human and make mistakes. And let's assume for a minute that they are all in good faith, i.e. no bribes, no pressure, etc. The fact that they will make mistakes does not mean that some referees aren't better than others or that some make more mistakes than others.
Some players are better than others and they are the ones who are rewarded.
By the same token, the best referees should be rewarded and the worst referees should be punished. I don't mean the best officials should get Lamborghinis and the worst ones should be burned at the stake. Every league has a referee's association which evaluates a ref's performance in each match, often with the help of video evidence. At the end of the season, officials are generally promoted or demoted based on their performance. The problem is that this often happens behind closed doors.