The IRS only gets involved when you have a person who is indistinguishable from a long term ordinary employee that you treat as an outside, independent business. This means that the person works exclusively for you for long time periods, takes detailed work direction from a superior, and does mundane things. A consultant is the opposite of this. The consultant decides for themselves what to do, has many clients, and does difficult things. They don't have any.
Agencies supply warm bodies that do mundane things just like ordinary employees.
They are, however, highly skilled at slinging bull and buzz words, frequently without knowing what they mean or how they are applied, just like the agency sales people. Real consultants do not need agencies to advertise and certify them. They work for themselves and they keep all of their fees instead of having an agency take a cut of them. This is the main reason why talented people do not work through agencies.