Just to add a few extra data points, for those in the UK but outside London...

I've set up more than one small company, with various accountants. The typical fee for a bespoke deal (customising things like share classes if you want) always seems to work out around £300, one way or another. You could get a limited company much cheaper, probably under £50, if you don't need anything special and you're happy to buy one off the shelf and sort out all the paperwork yourself.

(Pro tip: If occasionally dropping a few hundred pounds on good professional services makes you wince, this is not the lifestyle for you. You need to understand the value of your time, how much time it really takes to deal with overheads, the real cost of getting official paperwork like contracts and financial statements wrong, and the benefits of different kinds of professional insurance and organisation memberships. Contractors charge much more then employees per hour for good reasons, and one of those reasons is that they have to deal with these overheads.)

IMHO, the tax and legal regulation isn't so bad, as long as you have a reasonable accountant. Producing annual financial statements for a personal contracting company shouldn't cost more than £1,000 unless you're exceptionally busy, and even good accountants might charge closer to half of that if you're only raising an invoice every few weeks and paying out occasional dividends so there isn't much work for them to do. Filing the other statutory paperwork and keeping track of the various deadlines is irritating and does waste a few hours each year, but it doesn't require specialist knowledge or skills you can't learn in an hour-long meeting with your accountant.

(Pro tip: If you're worrying about whether you can keep business records effectively and remember to file stuff, this is not the lifestyle for you. As a freelancer, whether you are self-employed or a company director, you will be legally responsible for getting this stuff right, and while most honest mistakes can be fixed without incurring major penalties, you certainly do hear about people who got it wrong and wound up paying tens of thousands in back taxes.)

Contract rates outside London are significantly lower, but still far higher on an hourly basis than working as an employee.

Some other general comments...

I second the recommendation in the article for PCG membership. To me, just having access to templates for standard contracting documents and to the tax and legal helplines was very useful when I first started out. There are also some benefits in terms of insurance, either directly in some cases or where membership gets you a discount with a third party on other professional insurance policies you might want/need. And there is a lot of other generally useful contracting advice to be had, forums that might help some people, etc.

The same does not apply to many other professional bodies. As a general rule, if membership costs a lot and lets you put letters after your name, but it's not a genuine university awarding a recognised qualification, there is about a 98% chance that it is a waste of money IME.

For finding work, recommendations/networking >> agencies. You probably know some friends or former colleagues who do contract work if you've been in the industry for a while, and they might well share potential opportunities if you just tell them you're going freelance and ask.

Being able to choose your own hours (including how many of them you work in a week) is a huge advantage to completely freelance work. There is no more putting in silly hours without overtime pay because your megalomaniac boss screwed up the project timescales and the deadline is coming up.

Working from home is both a blessing and a curse. It can be wonderful, but you do need to consider basic things like setting up a proper office in a dedicated room (particularly if you've got a partner/family) and how you're going to interact with other real people if you're not getting out of the house every day. Do check the implications for things like capital gains tax and planning consents with someone who knows what they're talking about as well.

Finally, as far as job security goes, I'm not convinced employment isn't a bit of a con here. Sure, firing someone is relatively difficult in the UK, but not if your company is going bust anyway, and with employment you're effectively locked into working with a single serious employer, often on quite nasty terms that interfere with or outright prohibit other commercial activities while not guaranteeing you any career progression. Who really has more financial security, the employee working for Big Employer (who lays off people 5,000 at a time when the axe falls) or the freelance contractor who can take on two contracts simultaneous, each using about half their time but with a different client, and who has developed relationships with half a dozen clients who come back with repeat business?

In a nutshell, if you're technically competent and you are organised and professional enough to deal with some basic business management, freelancing can be an excellent career choice. I think far more people in many technical/creative industries would choose to do it if they knew what it was really like, and you find far more people who say they wish they'd made the jump sooner than who give it up and went back to working for The Man. It isn't for everyone, but I'd certainly encourage anyone who has wondered if it might be for them to at least look into it properly.