AI in Production: The Impact of AI on the Publishing Value Chain
By: Hederis Team
Tags: Publishing, AI, Machine Learning, Artificial Intelligence

A graph representing the metadata of thousands of archive documents. Source: Wikimedia Commons

AI is coming. AI is here. In the popular press, you read a lot of stuff about the future of artificial intelligence, and I do believe, as do many others, that it’s a technology that has a long way to go before it lives up to the hype (and the dread — cf. our companion Medium piece from this week, Retiring the T-700: Toward a Mandalorian Conception of Artificial Intelligence) of its name.

I’m reminded of a quote from Stack Overflow and Discourse.org co-founder Jeff Atwood when asked if he had any ideas for utilizing artificial intelligence algorithms. His short response: “I do not. I’m not sure ‘artificial intelligence’ is a particularly useful concept at this time. “machine learning” is at least a little more accurate in describing where we are. 😉” I think what Atwood’s perhaps curmudgeonly response points to is that we’re still seeing machines and algorithms trained to do only one thing or a couple of things well enough to be called intelligent or learning.

In a discussion with Hederis team members on the subject of AI in publishing, founding team member and head of operations Erica Warren elucidated one reason why some tech insiders like Atwood remain wary of the AI label:

Part of the issue is that when people think of AI/ML, they generally think of complete automation — and our experience with building our automation product in the past (at Macmillan) is that while even 95%–99% of cases might be straightforward, the 1% over the course of an entire book is actually a lot. So to go that route, publishers either have to (a) lower their standards of quality — which is a non-starter for professional publishers competing with self-publishing, or (b) review all of the changes manually anyway, which defeats the time savings of using AI in the first place.

So while much of the dread and fear around AI relates to too much unchecked autonomy, the healthy skepticism toward all these products labeled AI relates to the fact that the machines are just not autonomous enough.

But as Erica pointed out, when we stop expecting AI to do all the work and accept a symbiotic role with AI as assistant, we can start to understand the potential these technologies have for apps like Hederis and publishing in general:

And the way to do that is to avoid entirely automated “push-button” systems. What we strive for is the concept of an “AI-assistant.” Computers are good at processing large amounts of data quickly, and matching new data to patterns in historical data. The challenge with publishing applications, in fact with any application that deals with art, is that the heart of creativity is novelty. ML applications are NOT good with handling something new that they haven’t seen before. Humans, though, are great at problem solving with new information. What we’re not good at is doing rote, repetitive work accurately for an extended period of time. So you can see that this is an ideal situation to sort of share the load, with the computer being optimized for the repetitive work and the people handling the unique situations that the computer is not able to handle — and in the end we get a higher-quality product in less time.

In publishing, AI and machine learning seem to be trained, broadly, on doing a few sets of tasks fairly well, and your position in the publishing value chain will definitely affect how you benefit (or suffer) from AI’s impact. In a keynote for Digital Book World’s inaugural AI conference (held virtually in May 2020), Lee Huang, a former digital executive and AI expert, described some broad areas of problems that AI is typically trained on — namely, natural language processing, semantic analysis, disambiguation, classification, and taxonomy — and offered up his CAPA framework for utilizing AI in a publishing setting. CAPA stands for Content Creation, Answers & Predictions, and Automation. Giving examples from each category, Huang’s presentation offered clues to how C-suite executives at major publishers are thinking of AI: as a place to generate engagement and revenue at a lower cost. Much of the dread that surrounds AI involves the fear that intelligent machines will replace intelligent humans in the workplace. To Huang’s credit, he frames a lot of the cost-saving benefits of AI for publishers as ways for a publisher to expand into new experiences — automatically generated AR tied to what the machine reads in your text, “deep voice” audiobook narration, automatically generated plain-language summaries of data sets and bodies of research — as opposed to ways to downsize existing staffs. but it still equates to growth in profits and capacity without growing staff. The logic of downsizing is, of course, looming in the shadows, if it’s not outright staring us in the face.

Although the executive view of AI may give the folks grinding it out in content creation, editing, and production the feeling that they’re headed for planned obsolescence, the other keynote from the DBW panel, given by Jason Boog, a journalist who writes about and experiments with AI for outlets such as Publishers Weekly and Towards Data Science, offered a view to ways content creators, and writers specifically, can form personal and open relationships to AI. Boog wrote a step-by-step tutorial about how to use the OpenAI API to access GPT-2, a powerful, interactive NLP language model that allows users to train a computer to do various tasks, including generating pieces of longform fiction. He presented this and other examples of how everyday people without massive R&D budgets or computer science PhDs on staff can start building and training their own AI assistants.

One theme of Boog’s talk was that, while the output of many of these publicly accessible AIs is a predictably mixed bag, AI can provide enough raw material for the right creatives and editors to do great work with less rote effort.

Boog’s sentiments resonate with Hederis’s own philosophy of AI adoption and use, which we are excited to develop and roll out as we head into our third quarter out of beta. The sentiment was echoed by these final comments from Erica Warren:

With enough data, it’s definitely possible to help users get more books through their system quickly by automating the boring stuff and letting humans use their time to actually make meaningful decisions, and use their considerable knowledge and skills instead of just doing manual labor.

While some “classic” production problems, such as classification and tagging of manuscripts are getting the AI/ML treatment from established publishing players and startups (this is an area where Hederis will be developing further capabilities over the coming year), we believe there are other questions, about what constitutes “skilled” type design and other production decision-making scenarios, that have yet to be asked and answered with AI. We’re excited for the next year, and to be a part of this conversation around the future of book production.