Notes on Quantitative UX Research at Google
Chris Chapman
Staff Quantitative UX Researcher, Next Billion Users

Overview
This is a personal, unofficial view on Quantitative UX research at Google. I share opinions aboutthe job, who may enjoy it, reflections on hiring, and ways to prepare for a Quant UX career.

Who am I, and Why am I Writing This?
I've worked at Google for 10 years (Google Research profile). I am often asked about Quant UX
and it's efficient to compile the answers. Quant UX Research is a relatively new field and I hopethese notes help others to learn about it and, when relevant, apply for positions. I'm writing fromthe perspective of a social scientist, which I am but not all Quant UXRs are.

Quant UX Researchers
Quantitative User Experience Researchers (Quant UXRs) are part of User Experience (UX)
teams. UX teams define user interaction with Google products by understanding user behavior
and designing user interfaces. UX teams include designers, writers, interface engineers, andresearchers. UX works with Engineering to define and create the applications that we ship.
Quant UXRs apply research skills, together with statistics and programming, to define andanswer questions about user experience behaviors and attitudes. What do users do? What are
their goals? What frustrates them? Among product choices we might make, what would users
prefer? How do we measure success? Quant UXRs work with teams to define researchquestions, and use many methods and types of data to answer them. A question may betransient; after we answer it, engineering teams act and research moves to a new question. At
other times, research is foundational, leading to knowledge that guides a team for years.

Other and Related Positions
There are many other research, statistics, and data science positions at Google. Here are a few:●●●●

Quantitative Analysts are most similar to the typical industry definition of a "datascientist." There is no specific UX focus. Many have PhDs in Statistics.
UX Researchers (sometimes called "regular" or "qualitative" UXRs) conduct
human-computer interaction research such as user interface testing.
Business Analysts apply data science skills to customer-facing problems (where"customer" might be internal or external).
Product Analysts make business recommendations from data, but typically do not
conduct primary research.

Who Will Enjoy Quant UX
Here are a few dimensions that imperfectly distinguish Quant UXRs from other positions:
●
●

●
●
●
●
●User focus: you naturally think about problems from the user's point of view, and you aremotivated primarily to improve products for users.
Research design for ambiguous spaces: you have "consulting" skills to turn businessquestions into research questions. You enjoy developing research strategies forunexplored problems. You think deeply about reliability, validity, and reproducibility.
Generalist: you are good at many different things -- from programming to a variety ofstatistical methods -- and feel rewarded by solving applied problems.
Technical skill: you actually enjoy programming and statistical modeling. You aredemonstrably excellent at them relative to an "average" researcher.
Novelty: you enjoy tackling new problems on a regular basis, requiring different skills
and/or methods from project to project.
Engineering application: you want to help teams to build products. Your research goesbeyond basic science to directly influence product design and business strategy.
UX or HCI (human-computer interaction) experience: Quant UXR exists as a specialty
within User Experience, and specialized domain knowledge is valuable.

An approximate technical level of expertise is this: an applied researcher at a graduate schoollevel in social or natural sciences, with expertise in multivariable statistics and programming.

What Does a Month Look Like?
Activities vary day to day, so I'll talk about what might happen in a typical month. First of all,
Quant UXRs work on several projects at the same time, where the projects progress at different
speeds and require varying degrees of effort.
To scope research questions, I typically meet with several product managers, engineering
managers, and designers to learn about the business decisions to inform. We jointly prioritize
research according to importance and difficulty to answer, in alignment with the objectives forthe business. Then we draw up plans to address as many as possible, and assess how to use
Quant UXR with other resources such as qualitative research to inform the business needs.
Quant UXRs choose tools according to personal experience and product area. I commonly
import data from online systems using SQL or SQL-like query tools, or from surveys (e.g., CSVfiles). I do almost all statistical modeling and programming using R, running under Linux andMac OSX. Other Quant UXRs might use R, Python, Matlab, and other languages. We rely
heavily on Google-built internal tools that are capable of processing vast amounts of data.
In the course of a month, I'll present new research findings and recommendations to UX teams
and Engineering stakeholders. Reports take the form of a Google Docs slide deck or document,presented via videoconference to a mix of local and remote teams. I may also teach (or attend)
an internal class on research methods. Quant UXRs often consult with one another; no one isexpert in everything, and Googlers are usually quite willing to help one another. For my part, I
often consult internally about discrete choice models and psychometrics.

Skills and Interviewing for Quant UX
Quant UXRs differ in individual skills and need not be expert in every area. We look forgeneralists, and the following describe approximate minimum levels of skill to be successful.
Beyond these minima, we look for a Quant UXR to have some area of particular expertise.
Research Design. This is the most important skill. In an interview, you will be asked to plan
research to address one or more business questions. Here's one example: how would you
assess product usage patterns from logs data? The interviewer will consider issues such as:
How do you tackle it? Will your research be valid and reliable? How will it be affected bysampling and biases? How do you ensure statistical power? You should be able to design A/Bexperiments that are appropriate for real-world situations. Often there is no single best answer;
you should clarify a project's goals and talk through the tradeoffs that affect a research plan.
Programming. Perhaps the most unique skill for Quant UX at Google is programming. Can you
program in a general-purpose language to munge data, interact with data systems, and performbasic algorithmic processes? If not, you won't succeed as a Quant UXR. Quant UXRs must beable to work independently to extract, merge, shape, and analyze data from various sources.
The language is up to you, if it is a relatively general-purpose language. Python, R, and Matlab
are most common; C, C++, Java, Ruby, Go, and others also work. (Non-general purposelanguages such as SQL and SPSS are not sufficient, although SQL is a useful skill to extract
data.) Expect a whiteboard exercise to write code for a basic algorithm or data handling.
Here's an example problem -- AFAIK not actually used in interviews -- that shows theapproximate level of programming skill for an onsite interview:(Problem) Suppose you have two vectors of numbers in sorted order. You need tocombine them to form a single vector that is sorted from lowest to highest value. Youcannot use an existing sort() routine. Write a program to interleave the vectors andpreserve sorted order.
(Potential follow-up questions) Make the program a reusable function. What if thenumbers are of different types? What if one or both vectors have zero length? What if
the vectors are too large to fit into memory? What is the time complexity (Big O) of yoursolution? How can you make the function faster? How would you generalize the functionto combine an arbitrary number of vectors?

Note: this is just an example that demonstrates relevant skills. Solving this problem itself
won't get you very far, but mastering the related skills will be valuable!
Statistics. Quant UXRs must understand statistics with technical depth and applied practicalskills. At a minimum, you should be fluent in descriptive statistics and linear regression models:the various kinds, common pitfalls, and how they relate to other models. Expect to be asked
how to select an appropriate model for an applied scenario, and what issues might arise withthe data or analysis. Above the minimum, Quant UXRs vary in skills, and some have experiencein machine learning, psychometrics, or Bayesian methods.
User Experience (UX). Finally, Quant UXR involves technical skills applied to User Experience.
Experience and background related to UX -- or more generally, human subjects research -- isessential. This might include knowledge of specific areas such as human-computer interaction
and usability research, or more general experience such as social science research. In theinterview, you can expect scenarios about working with human subjects data and interactingwith other UX professionals such as designers.

Preparing for a Quant UXR Career
We're often asked what an ideal Quant UXR looks like. Quant UXRs differ in backgrounds,
including degrees in psychology, human factors, brain sciences, computer science, economics,statistics, political science, and other fields. Many have PhDs, although that is not required. The
benefit of a PhD is that it involves training to identify research needs, manage research frominception to completion, and explain and defend the results. This is what Quant UXRs do all thetime. But if you match the "Who Will Enjoy Quant UX" dimensions above, you're a candidate.
The "R" matters more than anything else: a Quant UXR is an applied researcher. The mostimportant background is deep, hands-on experience in research definition. This means not justcontributing to research that someone else defines, but leading it from start to finish. Gainknowledge of, and direct experience with, a variety of statistical methods and data sets.
Check yourself for interest and skills in programming. You do not need to be a computerscientist, but you will need to write mostly correct code or pseudocode in real time for routine
algorithms such as loops and list structures.
Finally, if you can gain experience with UX or closely related areas -- such as website usability,interaction design, online logs analysis, behavioral modeling, or survey research -- you'll
understand much more about the unique position of a Quant UX Researcher.

Why Google?
It's a great place to work! In my experience, people treat each other well, pay is good, theworkplaces are nice, and there are great office locations around the world. Most of all, there aremany, many interesting problems with resources to tackle them, and the work has real impact
on millions, and often billions, of users.

Applying
Quant UXR jobs can be found at https://careers.google.com/ by searching "quantitative user
experience researcher".
⇒ Tip: if you know someone at Google, ask her or him for a referral before applying
online. The Googler can add a note about your skills and potential fit.
Official details about the interview process are available at the Google Careers site.

Want to Read More? Some Publications
As a former academic, I feel like a reference section is required!
● A model that is often used at Google for user-centered assessment is the HEART
framework (Rodden, Hutchinson, and Fu, 2010).
● For common programming and statistical analysis tasks, I've described some common
models applied to consumer research using R (Chapman & Feit, 2019). There is acomplementary text covering these topics in Python (Schwartz, Chapman, & Feit, 2020).
● Many publications from Google UX Researchers (both Quant UXRs and general UXRs)
are available in the Google HCI Research archive.
● Tim Hesterberg has written about Statisticians at Google, with many links to Google
projects, positions, and publications.

Thanks!
First, many Quant UXR colleagues at Google contributed ideas to this document, especially
ex-Googler Sören Preibusch, who provided many notes and reflections over years of working
together. Although this document reflects my opinions and observations rather than official
policy, these suggestions made it much better.
Finally, thank you, the reader, for your interest in Quant UX and Google! Quant UX is an
exciting, growing field where enthusiastic and excellent researchers are always needed.