A Primer of 29 Interactions for AI
By: Chris Noessel
Tags: Machine Learning, Patterns, The Basics, Artificial Intelligence, Interaction Design

Generators & Optimizers

The newest and arguably most unsettling capability are those that output new things. It’s also the oddball category in this writeup that is not just about one kind of technology. These algorithms use some of the underpinnings of clusterers, classifiers, and regressors (and other things) to help people be creative or creatively solve some problem. (And, it must be said, some malefactors to spread disinformation.) Offer users generators as part of creativity tools. Offer users optimizers to achieve the best and unexpected results in complicated decision spaces, like architecture.

Practical examples of generators and optimizers

“Inception” Deepdream images : Those psychedelic images with, say, dog faces everywhere are neural nets that change images to maximize anything that looks like some target feature, like a dog face.

: Those psychedelic images with, say, dog faces everywhere are neural nets that change images to maximize anything that looks like some target feature, like a dog face. Deep style : Images in which the style from one image is applied to another, like, say, modifying a modern selfie to look like it was painted by J.C. Leyendecker or Raphael.

: Images in which the style from one image is applied to another, like, say, modifying a modern selfie to look like it was painted by J.C. Leyendecker or Raphael. Deepfakes : Images, audio, and video in which a person is replaced with another.

: Images, audio, and video in which a person is replaced with another. Mebots : These chatbots train on recorded speech or text by an individual, and thereafter can mimic them in chat or interactions, even responding to questions and prompts it hasn’t encountered before.

: These chatbots train on recorded speech or text by an individual, and thereafter can mimic them in chat or interactions, even responding to questions and prompts it hasn’t encountered before. This Person Does Not Exist : (https://thispersondoesnotexist.com/) Which composites features from different images of people to create new people. It’s mostly seamless. (And a fine place to go for persona images, if you only ever need one.) There are also “does not exist” networks for art, cats, horses, and even chemicals.

: (https://thispersondoesnotexist.com/) Which composites features from different images of people to create new people. It’s mostly seamless. (And a fine place to go for persona images, if you only ever need one.) There are also “does not exist” networks for art, cats, horses, and even chemicals. GPT-n : The Generative Pre-Trained Transformer algorithms from OpenAI can take a few words and generate a lot of sentences that sound related and reasonably convincing. People have used GPTs to “write” songs, poems, articles, blog posts (this is not one of them), and even movies. GPT-3 is the most recent version as of this writing, and is so sophisticated that OpenAI has not released it generally, for fear of its misuse.

: The Generative Pre-Trained Transformer algorithms from OpenAI can take a few words and generate a lot of sentences that sound related and reasonably convincing. People have used GPTs to “write” songs, poems, articles, blog posts (this is not one of them), and even movies. GPT-3 is the most recent version as of this writing, and is so sophisticated that OpenAI has not released it generally, for fear of its misuse. Text-to-image : Open AI’s CLIP can create surreal and gorgeous images from a short text prompt, like “a cityscape made of flowers” and “dogs playing poker”. (hat tip Gene Kogan.)

: Open AI’s CLIP can create surreal and gorgeous images from a short text prompt, like “a cityscape made of flowers” and “dogs playing poker”. (hat tip Gene Kogan.) Generative design: Given a set of criteria, generative algorithms explore a possibility space and suggest novel designs that meet the criteria. Spacemaker, for example, can present multiple configurations of buildings on a site to optimize for quiet, sun exposure, or wind control.

↑ Input: Most work from “seed” input like a few lines of text or an image, and all work with massive data.

↓ Output: New images, sounds, or texts

Basic interactions with generators and optimizers

13. Seed/Reseed: The source material, options, and criteria for success, if any.

14. Rerun: If users didn’t like the output.

15. Adjust “drift”: Many generators allow control of how “prim” or “wacky” the results are. A news agency way want prim headlines, but a poet may want the wackiest output they can get their hands on.

16. Explore options: Generative systems can produce a lot of results. As with any large result set, users may need help making sense of them all, or to design parameters that can reduce the number of results. (Or even to design systems that let end-users manage those parameters themselves.)

17. Lock or reweight criteria: If users are happy with some aspects of a generated result, but not others, they should be able to lock those aspects and re-run the rest for new results. For systems that are trying to optimize for certain goals, users may want to change the weightings and see what else results.

18. Save results (of course)

19. Export and extend: When users want to use the selected results as a starting point for doing more manual work, they need the ability to export in a way that lets them continue working.

Related concepts and jargon: neural nets (CNNs, RNNs), generative adversarial networks, deep learning, deep dream, reinforcement learning.

Universal Use Cases

Some basic interactions aren’t tied to any core technology. These aren’t derived from any particular AI capability, but they are important to consider as part of the basic interactions that need to be designed. Many are required by law.

20. Explain that to me: This isn’t always easy. Some AI technologies are so complicated that, even though they’re often right, the way it figured things out is too complicated for a human to follow along. But when decisions can be explained, they should be. And when they can’t, provide easy ways for users to request human intervention.

21. I need human intervention: This is vital because AI will not always reason like a human would. As AI systems are given more weighty responsibilities, users need the option of human recourse. This can often be in the form of objecting to an AI’s output.

22. Override: In systems where users are not beholden to follow an AI’s recommendations, users may need mechanisms letting them reject advice and enact a new course of action.

23. Annotate: Users may need to store their reasons for an override or correction for later review by themselves, their managers, auditors, or regulators. Alternatively, they may want to capture why they chose a particular option amongst several for stakeholders.

24. Let me help correct your recommendation for the future: If a recommendation was rejected or overridden, users may want to tell the AI to adjust its recommendations moving forward.

25. Let me help correct your recommendation for my group: One of the promises of AI is hyper-personalization. Its model may need to adjust to the idiosyncrasies of its user or group of users.

26. Show me what personal information of mine that you have.

27. Let me correct personal information of mine that you have.

28. Delete my personal information from your system.

29. Graceful degradation: If the AI components of a system should fail, users still have goals to accomplish. If the AI is not critical, the system should support the user’s shift into a more manual mode of interaction.

There are distinctions to these use cases when the AI is acting as an agent, an assistant, a friendly adversary, or a teammate. But the domain will determine how those distinctions play out.

Ethics

We’re fortunate that today many design discussions about AI involve ethical considerations. I don’t want to short-change these issues, but smarter people than me have written about ethics, and summaries risk dangerous oversimplification. For now, it is enough to remind you that understanding what AI can do is only part of understanding what it should and should not do.

This article is just about the likely interactions that can have good or bad effects depending on how they’re implemented. It is you and your team’s responsibility to work through the ethical implications of the things you design and to design with integrity — or to raise the alarms when something shouldn’t be designed at all. At the end of this article, you’ll find a list of links to better sources if you want to know more.

These basics are building blocks

The field of AI is changing all the time. New capabilities will come, but for now, these 29 are the use cases you are likely to run into when designing for narrow AI. But these also aren’t the end of it. These must be woven together and expanded upon given your particular domain and application. I expect most sophisticated products will use more than one category simultaneously. For example, the supply chain software that I work with at IBM, Sterling Supply Chain Software business assistant, built on Watson Assistant, uses all of these…

Clusterers: To analyze inputs for a given bot and identify common topics and questions that went unanswered. These become candidates for future development. Classifiers: To understand the user’s intents from their natural language inputs, as well as the “entities” (things) that have been mentioned. Regressors: To suggest playbooks based on ongoing conversations. Generators: For generating natural language responses.

And you look back over the lists of basic interactions, I hope you realize that these are use cases you can handle. Disambiguation, optimizer criteria, and scenario-izing may be the most novel interactions of the bunch. But things like managing groups, flagging bad results, and exploring multiple options are something you could design outside of AI.

These four capabilities are like building blocks. They need to be embedded in larger systems that support the humans using them, the environment in which they operate, and more to the point — supporting the user’s goals for using them in the first place. That’s why user-centered design is still key to its successful implementation. You got this.