UX Designer: facilitator in the future of data design and Machine Learning
By: Liz Gilleran
Tags: Resources, Design Thinking, UX Design, Machine Learning

From Unsplash https://unsplash.com/photos/KgLtFCgfC28

Machine Learning (ML) has a firm mainstream hold in the technology industry, yet User Experience (UX) still has an uneasy relationship with it. The answer lies in not trying to learn each other’s skill sets, but learning to collaborate together. The UX specialist — is at heart — a facilitator in this relationship, which should be embraced.

What’s wrong with the way we do things now?

UX Designers don’t have to go so far as to become ML experts, but often don’t know where to start in forming this relationship. Also, trying to become an ML expert would take an exorbitant amount of time and there are more specific issues to solve. For instance, as far back as 2008 there have been calls for better integration of Human Computer Interaction (HCI, often regarded as UX’s precursor) and Artificial Intelligence (AI) with concerns that user’s mental models will not adapt to explanations. Designers and HCI are still on the backfoot — in a survey for UX practitioners of Machine Learning projects, there were reports that Designers often feel a step behind the ML practitioner on the project. For the purposes of this blog we will refer to Machine Learning, rather than AI.

Defining ML in terms of UX

The Interaction Design Foundation refers to User Experience Design as “[a] process design teams use to create products that provide meaningful and relevant experiences to users.”

The definition of Machine Learning that I believe is most helpful for User Experience designers is from Jason Mayes’ slide deck on Machine Learning 101 — which is that ML systems implemented today are a form of “narrow” AI — “a system that can do just one (or a few) defined things as well or better than humans”.

Essentially, ML is not necessarily creating a sentient artificially intelligent program but a set of instructions based on data that the system is given. This restriction can significantly limit what UX can and cannot do when working with ML, and help to isolate problems in terms of what is relevant for the user to know about a system, and what can ML provide in terms of a meaningful experience.

How does this differ from the UX and software developer relationship?

HCI and UX professionals are often seasoned to working with web developers or software engineers, but this is not a direct match for integrating another profession into this loop. The UX Designer is a facilitator to the ML process, not handing over design concepts for implementation like they might with a developer. The UX Designer is the go-to person to define the customer’s problem, a part of which might be for the ML engineer to solve, essentially boiling down to them answering a question with data (i.e. making the right thing to build).

What can stop a harmonious working process is believing that developers follow the exact same process as ML practitioners. Machine Learning calculates inferences and representations without regard for human comprehension and uses a lot of large and complex libraries that were probably coded by someone else and may or may not suit the data. As my friend in an adjacent area put it:

“The human in the loop must decide how well the algorithm has fitted a result to the data set. The human decides on the statistical measure to determine what is ‘good’, and may choose to adjust any or all parameters to the algorithm in an attempt to increase accuracy.”

To add more complexity here, deciding how much data is needed also seems to be an art unto itself, but the answer lies in the thousands of pieces of data and up. It is often suggested to me by peers and often eager entrepreneurs, but would be incredibly unwise, that an accurate model can be trained on very little data. However this would make a model overfitted to particular data, which would not be incredibly useful, more and better training data would be required for a decent model.

UX and ML follow almost the same process

UX and ML are both disciplines that require testing hypotheses, gathering feedback, and trying again. Yet too often they are both used as a band-aid for projects that may have more fundamental issues at play, particularly in trying to solve too many problems at once. This temptation often leads to a desire for just machine learning, without HCI or UX expertise.

It is at this critical point — where the processes are not working together — where time can be lost unnecessarily. Clients might get a basic understanding of ML, which then suddenly becomes the solution to every problem. Clients can then lose sight of the overall project and goal. It is difficult to then optimise two layers of a project if they are not in sync.

The UX Design process — From my 2018 talk at http://lizgilleran.com/?p=95

The ML process — From my 2018 talk at http://lizgilleran.com/?p=95

“I think there is an intersection in what the ML is trying to do and how it is presented to the user. I had a clear picture of how the algorithm would be in my head but the designing UI often has something different in mind. It is silly to optimise half your site for another goal when those two halves are highly connected.” - Renee Noble (ConnectEd Code Founder)

Approaches to explore in your UX/HCI and ML collaborations

Users need to understand the boundaries of the underlying technology embedded in their products and are able to form their own judgements in a verified way.

This sentiment was echoed in the ACM in 2017 by Lars Erik Holmquist detailing AI as a design material, to implement it in ways that create more open ended interfaces, but also have a locus of user control. The Good Data manifesto posited that we should respect people’s data more.

There is also scope for abuse and harm using this kind of technology — which is definitely where HCI/UX could help. Google Design’s human centred ML process expresses the view that practitioners should weigh up the cost of false positives and negatives. This is fair, but it is also difficult to attempt to out-predict the technology whose job it is to identify patterns in the data.

What we can get right

This is the beginning of understanding how best to start working with ML practitioners. Bringing humans closer to a technology that is open ended could be a lifetime’s work for those in this area. However — by and large — we will all be the ones who suffer if collaboration remains unlocked.

So, what next?

UX designers are typically effective facilitators of wicked problems in workshops, and this is where our skillset is leveraged to determine what we can actually build using data. When negotiating with clients who wish to have data-driven systems, I believe that there are fundamental questions to be answered:

What is the client’s data goal?

Does this even need ML? Machine learning will not fix everything, particularly if the problem is maybe more of an operational one within an organisation. What can ML realistically solve?

2. What data do we actually have? What are the disadvantages and advantages?

It could be that the data is very usable in its current form. Alternatively it could be completely unusable and not machine readable. This means we must brainstorm other ways to solve the problem without using ML. This could include the client owning, collecting and producing the data, or spending a good amount of time making it usable.

But, conversely, the client may not own the data or may have other roadblocks.

3. Is there enough data to action what the client wants to achieve?

This is key! Remember that 3 images is not enough. This must be in the 1000s and up.

4. Consider: what can the engineer realistically answer from this data?

The client needs to identify a question that can be answered by the data and possible predictions — provided questions 1–3 are positive!