An Innovation Funnel for Machine Learning Projects
Vatche Isahagian, Yara Rizk, Vinod Muthusamy, Merve Unuvar, Ayhan Sebin
IBM ResearchABSTRACT
The innovation funnel is a popular tool in the project management
world that allows a team to track numerous projects at various
levels of maturity and imposes a well-structured operational model
to integrate them into the main product. It is characterized by stages
and gates [1]. Projects can only move to the next phase if they havepassed the criteria defined by the gates. These stages and gates canbe defined based on where the funnel concept is being applied.
For instance, one funnel can have incubation, inception, productization, and transition as the stages and identification, business,market and engineering gates. When a new idea is identified (passes
the identification gate), it enters an incubation stage where a proofof concept is implemented. If there is a business need for this idea,
then the idea can be turned into a project and moved into the inception stage where more experiments can be performed to verify
viability, mature the implementation, etc. Next, the project must
pass the market gate where the interest of possible buyers is gauged
and a market is defined for the project. At that point, the project
enters the productization stage where more emphasis is placed onturning the proof of concept into a fully functioning product (orintegrated within an existing product) that can be released to business users for beta testing. Finally, if the beta release goes well, theproject passes the engineering gate where the technology is transitioned to a engineering support team that takes over the feature
which is now integrated into the product.
Applying the innovation funnel to machine learning projects
may not be as straight forward. Many challenges arise when attempting to marry traditional software development operationalmodels with machine learning development. One such challenge isestimating the time it would take to implement a proof of concept in
the first stage. Machine learning development tends to be iterativeand can take anywhere from weeks to months depending on how
well the model performance with the initial parameters. Anotherchallenge relates to data scientist or machine learning engineers
and software developers not seeing eye to eye or being able to see
the other person’s perspective when it comes to collaborations.
Machine learning is a probabilistic software system that doesn’t
behave predictably; software developers expect predictability andreliability, spending a lot of time crafting test to ensure the systembehaves as expected. If not designed carefully to account for the
probabilistic nature of machine learning, these tests may fail frequently causing software developers a lot of anguish. In this talk,we discuss some of these challenges and possible ways to mitigate
them or resolve them to create a more machine learning friendly
innovation funnel.
ACM Reference Format:
Vatche Isahagian, Yara Rizk, Vinod Muthusamy, Merve Unuvar, Ayhan Sebin.
2022. An Innovation Funnel for Machine Learning Projects. In Proceedingsof The Workshop on Applied Machine Learning Management at The 28th
ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD
WAMLM ’22). ACM, New York, NY, USA, 1 page. https://doi.org/XXXXXXX.
XXXXXXX

1

APPENDIX. Relevance Our talk relates to “Collaboration with product development" under “Managing Machine Learning Projects" fromthe CfP. Specifically, we discuss how to structure the collaboration
between research and product business units to more organicallydrive innovation into the product especially when that innovation
is based on machine learning technology.
Discussion Points. There are many points to discuss including but
not limited to: how to structure research and product collaborations;
how to align research processes with product processes; challengesof fitting AI into traditional software processes and others.
Company and Project Portrait. IBM Research is an industrial research lab with over 3000 researchers in 17+ labs around the world
developing advancements in AI, quantum and many other fields
within IBM, a multinational technology corporation. The talk draws
on the authors’ experience collaborating with product teams on
multiple AI for automation in business enterprises.
Main Presenter Bio. Vatche Isahagian’s research spans a broad setof disciplines across distributed systems, AI, and business processes.
As a research staff member at IBM Research, he worked on several
projects including business process insight and service composition,
OpenWhisk large scale Serverless Computing framework, and theIBM Deep Learning as a Service. He currently co-leads a team ofresearchers focused on improving business processes using AI capabilities such as planning and conversational interfaces. Vatche has
published in top-tier conferences such as CACM, BPM, AAAI, KDD,VLDB, SDM, and INFOCOM. He co-organized several workshops,and served as a program committee member, demo co-chair, andpublicity chair on several conferences.

ACKNOWLEDGMENTS
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.
KDD WAMLM ’22, Aug. 14–18, 2022, Washington D.C.
© 2022 Association for Computing Machinery.
ACM ISBN 978-1-4503-XXXX-X/18/06. . . $15.00
https://doi.org/XXXXXXX.XXXXXXX

The authors would like to thank Stephane Mery and Rania Khalaffor inspiring this paper by creating and defining the original operational model for the Research-Product collaboration based on theinnovation funnel concept.

REFERENCES[1] Niek D Du Preez and Louis Louw. 2008. A framework for managing the innovation
process. In PICMET’08-2008 Portland International Conference on Management ofEngineering & Technology. IEEE, 546–558.