Data Science Manager Experimentation at Reddit

“The front page of the internet," Reddit brings millions and millions of people together each month through their common interests, inviting them to share, vote, comment, and create across thousands of communities. 
Reddit is continuing to grow our teams with the best talent. We're completely remote friendly and will continue to be after the pandemic. Learn more about how we're evolving Reddit's workforce here.
We are looking for a Data Science Manager to lead our Data Science horizontal on Experimentation and to partner with our Engineering team to drive and implement the vision of Experimentation at Reddit.
Our Experimentation Platform has been used by our Product and Engineering teams to run thousands of experiments in order to ensure that we fully understand our users and create the best possible experiences, as well as to roll out thousands of features in a safe, appropriate way. In 2021, we have rapidly scaled our Experimentation team and are quickly turning the platform into a world-class framework, capable of even greater scale and effectiveness. 
As the Data Science Manager for Experimentation you will lead a small but growing team and will be pivotal in driving company-wide innovation and rigor, shaping our metrics strategy, and building a strong experimentation culture. The opportunity for impact will be great as you enhance the success and understanding of our products across our diverse and growing user base.
How You’ll Have Impact:
Reddit’s mission is to bring community and belonging to everyone in the world, and you will be pivotal in making that happen. As the 6th largest site on the internet Reddit has a dramatic scale but at 1000+ employees you won’t be stuck making small incremental gains on enterprise systems. You will create organization-wide capabilities to run more advanced experiments and create publication quality research, leverage complex testing and optimization algorithms to grow revenue, and produce more robust results faster to help teams increase their velocity and effectiveness.
What You’ll Learn:
You will partner closely with a world-class org of Engineering, ML, Search, Data Science, Data Engineering, and Analytics practitioners who work together to create a variety of capabilities for Reddit to be more successful in its decision-making and product creation. In particular, you’ll be exposed to and drive adoption of a variety of new techniques in modeling, experimentation, statistical frameworks, and causal inference.
What You’ll Do:

Be the main point of contact for Data Science / statistics of experimentation, creating and implementing the roadmap for the stats engine of Reddit’s Experimentation Platform, providing guidance on experiment design, analysis, and metrics 
Apply modeling (advanced statistical methods, machine learning) to improve the accuracy of our experimental results (e.g. variance reduction techniques such as CUPED)
Be responsible for platform-wide Data Science process & analyses, defining company-wide experiment processes (delegation of authority when certain metrics are impacted, etc.), communicating good experimentation best practices and setting the standard for experiment design for the entire organization
Ensure high-quality and trustworthy experiment results and ensure that experimental designs lead to action
Collaborate with multiple engineers/product teams across the whole company to find technical solutions to complex experiment design challenges
Develop and adopt statistical methods that help Reddit teams better measure their feature areas
Perform meta-analysis on experiments to understand macro trends in our product ecosystem and to understand the health of experiments for the company as a whole

Who You Might Be:

5+ years of experience in quantitative or data science roles, preferably for a consumer-facing service/app
1+ years of experience focused on experiment design and/or A/B testing
Demonstrated application of the statistical underpinnings of experimentation: hypothesis testing, p-values, causal inference, etc.
Familiarity with statistical analysis and programming languages (e.g., R / Python)
Proficiency with querying relational databases (e.g., SQL)
Results-oriented with a strong customer and business focus
Entrepreneurial and self-directed, demonstrated ability to innovate and bias toward action in fast-paced environments
Ability to communicate and discuss complex topics with technical and non-technical audiences
Ability to tackle ambiguous and undefined problems