  Data analysis requires translating higher level questions and hypotheses into
computable statistical models. We present a mixed-methods study aimed at
identifying the steps, considerations, and challenges involved in
operationalizing hypotheses into statistical models, a process we refer to as
hypothesis formalization. In a formative content analysis of research papers,
we find that researchers highlight decomposing a hypothesis into
sub-hypotheses, selecting proxy variables, and formulating statistical models
based on data collection design as key steps. In a lab study, we find that
analysts fixated on implementation and shaped their analysis to fit familiar
approaches, even if sub-optimal. In an analysis of software tools, we find that
tools provide inconsistent, low-level abstractions that may limit the
statistical models analysts use to formalize hypotheses. Based on these
observations, we characterize hypothesis formalization as a dual-search process
balancing conceptual and statistical considerations constrained by data and
computation, and discuss implications for future tools.
