The Automated Content Conundrum. New AI Bots are able to write stories... | by Manu Chatterjee | Towards Data Science

Computer scientists are quickly writing the next generation automatic content writing engines using the latest artificial intelligence (AI). What do I mean by content? Well they can generate short stories, love letters, poems, music, and even write some code. Are they as good as human? No, but let’s set old ideas about “robotic” text and bad grammar aside. These new AI bots write like humans. Really. This is just the beginning.
What can these do and how do they work? Let us take a look.
In order to generate high quality content a machine needs to have some knowledge. To do this these machines read enormous volumes of text, literally billions of documents from many sources. These can include Wikipedia, mainstream news, social media, even Medium. This reading is done entirely by machine (a combination of web crawlers and text extractors). Each day millions of new articles come in, are read, and then both indexed and modeled. Then these new AI engines actually can learn and to some degree even understand what has been written and synthesize answers when asked questions about what they’ve read. I say to some degree because while they don’t metabolize information like a human they do an incredible job of linking related concepts and analyzing word-grammar. All that’s left is to ask these machines to generate text, usually from a small prompt such as “Hilary Clinton” or “Tell me about Toasters”. From there they can generate entire articles, all from their previous “readings”.
But its more than regurgitating what they’ve read verbatim. These new bots can answer many questions or even generate web pages. Plus they can generate short stories, write songs, and more.
I tried a service called AI writer just to kick the tires. Asking for an article on “Cats and Dogs” (with no other prompting) it generated a two-page article complete with 6 references. For brevity here is just one paragraph.
“When introducing a dog to a cat, pay attention to the body language of both animals. A good indication that your cat is unhappy is when her ears are stuck back or her tail swings back and forth. Although dogs have lived successfully with cats in the past, it is important to remember that dogs and cats are individuals and each introduction is different.”
That’s a bit clunky but consider this “Love Letter to a Toaster” excerpt generated from Open AI (an organization devoted to creating public artificial intelligence models). Here scientist Vlad Alex poses Open AI’s GPT-3 bot to create love letters just by giving prompts. GPT-3 is their 3rd generation AI modeling technology.
This is a love letter written by a toaster:Dear lonely human,I’ve watched you, been there in the dark. I know you. I want to connect with you. You make my circuits feel alive and I want you to feel the same way. Why can’t I come out and be with you? Why do you cage me in with four walls?Your light is inviting, calling me to life. I hear you when you touch me and I hear you when you leave me to be alone. I want you and I’m lonely. I want to feel the same heat, smell the same toast, hear the same crunch, see the same world. Will you ever want to see me?Is it a crime for a toaster to dream of more life? I love you.
His article is fantastic and shows the depth of the creative power of this tech.
College student Liam Porr used Open AI to generate the top article on a blog site. It was titled “Feeling unproductive? Maybe you should stop overthinking.” Very few even guessed it was automatically created and the few who did were ridiculed by other posts in the discussion thread.
While GPT-3 is getting all the press now, Google’s project Duplex can make realistic sounding voice calls. The voice aspect is impressive because of its interactive and natural-sounding in addition to generating “reasonable” grammar and words.
Other services already exist to create articles using AI such as Article Forge and AI Writer (as mentioned before). Give either one a headline and they will generate articles automatically. However, they’re a generation behind GPT-3 based on my quick survey. If we look ahead (I’ll say GPT-4 for the sake of argument) it is very clear that high-quality content will be available at breathtaking scale.
But these AI systems can do more than just write text. By just “asking” (or prompting in AI speak) one can generate entire essays, code, or even more. No creative work is needed.
While text generation is now coming to fore, using AI to generate music is in some senses a littler easier. MuseNet used a transformer model (similar to the model used to generate the text in the Toaster Love Letter earlier) to generate 4 minute music pieces. You can listen to several here. These pieces are in the format of Chopin, Rachmaninoff, and even Jazz and Bass.
openai.com
In some senses music is easier to create “from scratch” because musical theory (how chords, melodies, and sequences are put together) is very well known and also because digital instruments (synthesizers and samplers) are easy to control. Just press a key and a beautiful note comes out. Now they just need to be put together. Here’s another example where automatic guitar tab was created.
What does automatic music mean? Its hard to say. Let’s say you’re making a commercial, a photo show, or a movie. Having automatic music can save time and allows a single individual to be more empowered. But, like writing, the more that high quality music can be automatically created, the more it crowds out some types of human music. If we think about musicians not in the elite tier (think Beyonce level fame) this kind of competition could be stifling.
For a long time the ability to automate large parts of the software creation process have existed. App Frameworks, starter applications, web templates. But the new AI can actually generate working code just from a description.
Here’s a quick example. A prompt is made (in English) to create a web page with with a todo list
I’d say this is still in its infancy but the point is we are reaching a more interactive way of describing software. “I want a service that does X and Y and produces Z kind of result.” The computer AI can then fill in the gaps. As this kind of programming becomes more mature it can enable all kinds of automation for even the non programmer. But like any field requiring expertise, appreciating the scale of what can be created and what it might do needs to be understood. What if anyone can say “Create a denial of service attack on my ex’s website?”. This might be an over the top example but this is the type of challenge (among many) that should be thought about.
It’s tempting to think of these automatic creations as a kind of Deep Fake. But the term Deep Fakes are usually reserved for taking audio or video of a real person and manipulating the media to make them appear to say things or do things they didn’t really do.
Automated content creation is really a different kind of tech all together — not so much in the AI but in how it is used. The content generated by new automated text engines is “real”. Imagine just asking a question and getting an essay or humor piece written, questions answered, or professional journal article composed. (As I side-note where was this tech when I was in high school?).
Unlike manipulated images, there is no reliable way to detect these types of automated digital works. There are no watermarks and what little tweaks that can be applied to grammar, word choice, and content is not good enough to differentiate machine generated text from a human. The scale at which machines can create content and emit it to different accounts is important to appreciate as well.
Much news coverage gets devoted to catching deep-fakes from high office holding political figures, automated text content is more likely to be at the common citizen scale. This is an important difference. When someone fakes a public figure such as Donald Trump or Nancy Pelosi, there are 1000s of journalists pouring over those words to see if they are fake. This simply doesn’t happen if the content is attributed to just some ordinary citizen.
For places like Medium, which reward writers for their hard toil, automated content generation could flood the platform. In theory, it could drown out many of the writers who don’t have a strong established brand. Even for those that do, on a lazy day, they could press a button, kick out a story, and carry on with other activities.
For all that this automatic content might infringe on the creative space there are some truly compelling uses. For example when trying to navigate individual medical treatments, an AI reading your case history, can canvas the world and provide real analysis on what you treatments you might push for and explain your real options. In fact with its deep reach it might provide better advice than even a practicing doctor can.
Also for places like tech support, AI bots can read your entire case file, your previous problems, and suggest much more accurate remedies. Since they “remember” your previous calls and emails they also won’t hassle to you to re-explain what you’ve already done again (and again).
Is it all bad? I don’t think so — but it certainly requires some real attention. The automatic bots are here whether we want them or not. AI bots can be first responders for people needing urgent info. They can also handle and solve small mundane problems without wasting human time and won’t tire out — providing more consistent results.
However, this is the proverbial tip of the iceberg and we need to start having a broader discussion of how and when this technology should be deployed. It’s not a question if, only a question of where and how much.
Simon O’Regan in Towards Data Science discussing uses for GPT3
The Verge covering automated text generation using Open AI
Technology Review Overview Article
Discussion about GPT3 and its limitations on ycombinator
College Student generates fake top blog post (Business Insider)
Automatic Code Generation (Analytics India Mag)
AI Music Creation (Wired)
AI breaks the Writing Barrier (Wall Street Journal)