Invisible Design: Co-designing with machines
By: Amber Cartwright
Tags: User Experience, Design, Leadership, UX, Product Design

The machine was, and still is, my constant partner. I need her in order to translate the creative thoughts in my head into tangible ideas I can share with the world. Transitioning to design from a modern dance career in my twenties, I never thought a machine would be my accomplice for innovation.

Machines have rapidly developed intelligence in this generation and their capabilities are changing the products we design. The process in which they are designed will also need to evolve. This article is the start of a conversation about co-designing with machines and what I’m calling Invisible Design — a process and design language for product designers working with artificial intelligence and technologies like machine learning. I believe these processes and tools are seeds for the future of product design.

Invisible Design — a process and design language for product designers working with artificial intelligence and technologies like machine learning.

Math and science are invisible forces that reveal themselves in more discernible ways when we take the time to observe and analyze them. Take, for example, an English gentleman strolling through his garden in the 18th century. He observes an apple fall from a tree and wonders why it didn’t fall sideways or upwards from the ground. How is this possible? What are the forces at play? What are they made of? Does the same effect apply to something as small as an apple and as large as a wagon? Sir Isaac Newton continued to grapple with these questions for over twenty years in what would become his law of universal gravitation. He was able to describe an invisible force that has tangible effects in our everyday lives.

The influence invisible forces have on our lives can be unexpected. I was recently perusing my Facebook feed when I noticed that several of my friends liked Simply Framed, a company that allows you to create and order custom frames for posters and artwork online. I started to think about all the unframed work in my closet and tapped through to check it out. What made me want to try that recommendation? What caught my eye? What kind of information was needed in order to personalize that post? The invisible forces of science and math here are not gravity, but Facebook’s algorithms. Advertising software is just a zygote when it comes to the power of machine learning and where these products are headed in the next five to ten years. Machines will increasingly be making decisions within user experiences, and co-designing with them is an essential partnership for the future of product design.

As in any craft, there are individual components to the creation process — understanding, tools and interpretation. I began to get the idea for developing Invisible Design while going through this process at Airbnb on a couple of data intensive product launches. I want to share some of my thoughts on what I have observed in my own work.

You have to truly understand a thing to design a thing.

You have to truly understand a thing to design a thing. Imagine trying to design a plane, but not knowing anything about aerodynamics, or designing a glove without knowing what environment it will be used in or the anatomy of a hand. You have to understand what something can do in order to design a product well.

Last year, my product partnership team was having a conversation about a machine learning model for a new pricing tool we wanted to build for our hosts. We were trying to create a model that would answer the question, “What will the booked price of a listing be on any given day in the future?” Answering this question was no small feat. I was trying to keep up as my data science partner described the regression model they were building. The words he was using were alphas and betas, and while he was showing me charts that I could follow, the language was foreign to my design background. I sat down with him afterwards and asked him to sketch a diagram of the model and talk me through it. This was an eye-opening experience. When he started speaking the language I knew — sketches and diagramming — I understood the model and what it was trying to achieve immediately. This was my light bulb moment. I understood what the machine could do for the product and how to integrate the information into the experience. We both were excited by this understanding, and once our language barrier was broken and we could speak fluidly about where the product could go, we could really begin to take the product thinking to the next level.

Smart Pricing regression model next to a visualization explaining the model is made of three parts that vary per host

I realized that this conversation did not have to be an isolated incident, but could have a larger impact on our teams. The discussion we had was a bite-size form of storytelling, just like what designers do when they quickly sketch out screens in a notebook. I learned from my colleague that the story of a product isn’t limited to the screens that the user can touch and see, it can also describe what’s happening behind the scenes. In the initial phases of product creation, an overarching story of how the experience will impact the end user is often created to help everyone understand what the product will look and feel like. These can take many forms, from storyboards to prototypes, strategy decks and diagrams. These presentations are created for many reasons, and one very important reason is to create a shared understanding of a product vision.

Understanding empowers teams. Building a shared knowledge allows innovation to happen as a step change instead of in micro steps.

Understanding empowers teams. Building a shared knowledge allows innovation to happen as a step change instead of in micro steps. Visualizing the roles that data and the machine play in the discovery process is the first part of Invisible Design.

I’m continuing to work with my teams to build data visualizations that tell stories along with the interfaces our users interact with. These visualizations tend to vary as much as the products we’re creating, but the outcome is always that they help to motivate, inspire and educate the broader product team.

After understanding what we’re designing and how it works, we can start building the product with a variety of tools. A carpenter has a hammer. A photographer, a camera. A product designer, sketch. A software engineer, code. What’s interesting about all of the examples above is only one of them has a tool with the ability to learn, change and grow over time. Most product designers today sculpt UI with reactive tools–shapes and pixels are drawn on screen input directly from a designer. We also use these tools for designing outputs that are controlled programmatically in systems like responsive platforms and components. Our data partners in product are adept with tools that evolve over time. Physical systems, economic models and algorithms organically grow as variables shape their outcomes. Technologies based on these factors can learn and determine their own paths. In conjunction, the tools that designers, data scientists and engineers use are advantageous to each other throughout the entire product process, not just in building the final user interface. This is the next step in the evolution of product design.

Invisible Design adds in data sets and algorithmic decisions into the initial stages of design– wireframing and user flows– to bring dimensionality into a typically flat and static part of the process.

Take, for example, a holiday campaign for pricing tips, which was the first iteration of our Smart Pricing product. We knew from past holiday seasons that there is typically low traveler demand during the last couple weeks of December and a spike around New Year’s when folks travel a lot for the festivities. We wanted to let our host community know that if they lowered their prices during December, they could attract more travelers. In our wireframe process, we had a one size fits all module to communicate this message. What we learned from the data model is markets have varying down seasons and need differing messages and visualizations. For example, Sydney’s low season starts in November and Miami doesn’t experience a low season due to the consistent demand from vacation travelers. Our user flows and wireframes could show how the market trends and data would have an impact on the product.