The immediate danger is from the media and the public against the makers of the A.I.

Look at how Google's stock price got wrecked when a demo went bad. Or how Bing had to make emergency changes to their chatbot after somebody provoked it into behaving belligerently. (Funny I demoed my A.I. to a group for the first time this morning and had no fear it would go wrong because I've used it every day since Dec 27)

If you are the first or second or third chatbot and it gets out that some sicko got it to write something sick then it is a big deal. If this is the 572th chatbot then it isn't news anymore.

As for "dangerous devices" that is surely overrated too. It's hard to make an effective bomb and you're the first person you're likely to blow up. Ted Kaczynski took nearly 20 years to learn how to make reliably deadly bombs. Dzhokhar Tsarnaev attended one of the best terrorist training camps in the world. The Morewrong prophet is afraid an LLM will spit out a DNA sequence that some rando will send to a mail order DNA synthesis firm to make a 100% deadly virus but it would take quite a bit more than that (maybe thousands of DNA sequences, a very good lab, and some highly trained researchers who aren't afraid to be killed by what they are working on) and if you had those resources you could do "gain-of-function" experiments and make very dangerous viruses without the LLM.

Long term what bothers me is not what goes on in public but what goes in private, maybe

https://abcnews.go.com/Technology/intimate-ai-chatbot-connec...

for instance we know people will follow a prophet like Jim Jones and commit suicide or that a follower of an blackpill incel could become a mass shooter, but all of those things go on in public. The CIA learned in MKULTRA that there is no magic pill or technique that can take the average person and make them into a Manchurian Candidate but if you're willing to wait for the right victim you can very much lead an isolated and directionless person down a rabbit hole.

For instance after 9/11 the FBI tried to bait a Muslim into a fake terrorist attack with a "Sting" operation and couldn't do it. Instead they found some poor black guy who thought Malcolm X was cool and he was so impressionable they were able to give him a fake bomb and fake guns and rent a synagogue for him to "blow up" and he was shouting "Allah Akbar!" when the police came for him and had no idea he was set up and knocked down like a bowling pin.

The system of a chatbot + a vulnerable human could go into very dangerous places whether or not the chatbot is specifically programmed to do that or through the mutual process of "reinforcement learning".

Now maybe that's like the fear that listening to rock music is going to make you become a drug addict or become pregnant, but it's a concern that will come up. See

https://abcnews.go.com/Technology/intimate-ai-chatbot-connec...

of course this is not something fundamentally new, as some people think the parasocial relationships that people develop on OnlyFans are dangerous and plenty of "Freezone" Scientologists have telepathic conversations with L. Ron Hubbard although it seems to me it really would be a gas to teach a chatbot to write like Ron and specifically complete all the O.T. levels that he never got around to finishing.