How to build buy-in with users and validate product decisions during COVID-19
By: Ramis Khawaja
Tags: Covid 19, User Research, Civictech, Code For Canada, Fellowship

Hey everyone! Team PixelGram, from Measurement Canada checking in. It’s been a couple of months since our last update and we wanted to share the progress we have made so far and also talk about how we have conducted user testing during the COVID era.

We left off last time with having identified two problem statements, and we were getting ready to conduct deeper research into the first one:

Inspectors find it frustrating when attempting to do the logistical planning and coordination of inspections. It is a time consuming and repetitive experience, with very minimal infrastructure in place to help streamline the process.

Could we get a definition please?

We kicked off the definition phase by sending out four baseline studies, one for each role at Measurement Canada that we’re building for: Electricity & Gas Inspectors, Weights & Measures Inspectors, District Inspectors, District & Regional Managers. The purpose of the study was to widen our understanding of how inspectors currently conduct the logistical planning and coordination of inspections, and to identify other staff members involved in that process.

We wanted to get an understanding of this process from both a quantitative aspect and a qualitative aspect. For quantitative insights, we asked questions like:

How often do the conduct inspections?

How long does it take to do the inspections?

And for qualitative insights, we asked:

How do you measure success?

What is the most frustrating part of the planning process?

Once we got the responses back, we spent the next three weeks synthesizing the data. By conducting this synthesis we were able to better understand and articulate the problem. We created pivot tables to document the quantitative aspects of the study and created various artefacts such as workflows, journey maps and personas to distil the qualitative aspects of the study.

After synthesizing the baseline data, we hopped into ideation. As a team, we started sketching out and discussing possible directions that the solution could go based on the research insights. We then translated those ideas into proper wireframes that we could put in front of our users.

User panel, assemble!

A user panel is a group of subject matter experts that help validate design and functionality decisions of a product. They also serve as a sample size for the larger user population. For our purposes, the panel was made up of inspectors. We wanted the inspectors to become product champions, so we brought them into the decision making process as often as possible.

“We wanted the inspectors to become product champions, so we brought them into the decision making process as often as possible.”

Forming the panel also allowed us quick access to a group of subject matter experts on a bi-weekly basis, and let us validate our assumptions and ideas in quick iterations.

We used the baseline studies to gather a list of interested participants and created two different user panels, organized by the two disciplines that the Measurement Canada inspectors fall into Weights and Measures (W&M), and Electricity and Gas (E&G). When putting together our user panels. We wanted it to make sure that we had a diverse pool of both experience and locations. At the end we had a total of 29 participants across both panels, spanning multiple regions and districts across Canada and encapsulating all experience levels of inspectors and managers.

W&M User Panel Breakdown

E&G User Panel Breakdown

Building buy-in

The user panels have played a fundamental role in helping us design a solution that works for our end users. They have been tremendously helpful in pushing our project forward despite the challenges of COVID-19. Testing with staff user panels saved us the work of coordinating testers from afar during the pandemic, which would have been a time-consuming challenge. Instead, this approach enabled us to quickly move from low-fidelity wires to full UI mockups, and got us buy-in for our solution from our panelists at the same time.

“This approach enabled us to quickly move from low-fidelity wires to full UI mockups, and got us buy-in for our solution from our panelists at the same time.”

One group of inspectors, the E&G, haven’t had the same upgrades as their counterparts have over the years. There have been a few attempts to rectify this, but nothing has come to fruition. We were keen to help change this, and with the E&G panel’s feedback, we learned that our wireframes didn’t fully facilitate the functionality they needed to prepare for inspections.

Their input was essential. Before getting insight from the E&G panel, we designed the solution below to allow inspectors to freely move assigned inspections into different groups or as we call it ‘Batches’.

E&G interface Ver. 1.0

However, unlike their counterparts, the E&G planning process is precise, because it involves a lot coordination with external parties. Version 1.0 of the solution wouldn’t give them that ability.

So we went back to the drawing board and redesigned the UI. The end product still followed the high level flow of the original designs, but we added functionality that allowed E&G inspectors to be able to pick the date and time of the inspections and used that input to auto-group their inspections by months. NowE&G inspectors have more control over their planning, which improves efficiency.

E&G interface Ver. 2.0

The bottom line

For our team, forming user panels has been a successful way of conducting testing and research during COVID. An added benefit was the way it allowed us to connect with users that have been looking forward to streamlining their work process for a long time.

W&M Prototype

We were able to gain buy-in through collaborative iterations with the E&G inspectors in particular, which was an essential part of our ability to build something that catered to each group’s unique needs. Because of our user panelists’ time and attention, we have been able to refine the solution from low fidelity wires all the way to visual design in a matter of weeks.

E&G Prototype

So, if you’re working on a project that directly impacts a large group with varied needs, a user panel could be very beneficial to your project.

And to our user panellists: thank you! You’re all awesome, and our product is going to be more awesome because of your guidance.