I think so many people, including the comments here, are underestimating the scale of the problem. At the minute, we've got a relatively healthy level of scepticism regarding what we see online. We understand that an article may not be telling the truth or that a picture may be photoshopped. We can go in with a critical mind and weigh up possible motivations and can maybe do some digging to find out if it's true or not. We understand that the ability of someone to deceive is limited by one or more of the following:

money

time

intellect

influence

Those limitations are about to go out of the window. Imagine QAnon on steroids. Not only are people going to be able to create fake stories, they're going to be able to generate mass amounts of content to back up those stories, for little cost, in hardly any time, with not a lot of effort and perhaps even with no social networks.

Let's imagine it's the previous US general election. Trump has just found out he has lost. He has a militia on standby to take over the TV stations and newspapers and control what is being broadcasted. Instead of shutting down left leaning media, he now just broadcasts AI generated videos of the newscasters claiming he won the election. The left leaning newspapers are taken over and flooded with AI generated articles claiming he won the election.

The left wing journalists try to speak out. But you can find a video or a news article featuring the same journalist saying the exact opposite.

Who are you meant to trust anymore? Unless you know the journalist personally, as far as you're concerned, it could be the Democrats who are flooding the internet with false videos and articles.

That's just one scenario. We could have any number of campaigns being made by anyone with an agenda. Maybe they'll try show a minority group raping and pillaging in an area with high racial tensions. Maybe a teenage girl will generate and send a sex tape of her rival around the school to try and humiliate her, along with a completely fake AI chat history.

We could very easily get to the point where you cannot trust a single thing that you see on a screen.

What happens if we get to this point? Government (if they're not already compromised by something like the above situation) will try increasingly large measures to try and get the system back under control, with possibly equally destructive effects.

All social login ends up being tied to an insurance number so it's completely traceable and public. Photorealistic AI ends up banned. All other AI is limited to officially sanctioned apps. Companies and individuals caught not complying face harsh jail sentences akin to leaking state secrets. Of course now, whoever controls those official AI apps now shapes your reality however they see fit. In some ways it's not too different to the gatekeepers of old except this time it's more insidious: rather than broad manipulation, they can tailor it down to the individual. Anyone who tries to dissent, the AI tries to 'correct'. If that doesn't work, a subtle AI smear campaign is run against them, specifically tailored to the people they interact with in their community, until the dissenter is isolated and believed to be mentally ill by everyone around them and in need of help.

We are opening up Pandora's box but I'm not sure there's even hope at the bottom.