Proposal for a talk in the Applied ML Management in KDD
https://wamlm-kdd.github.io/wamlm/index.html
Title:
Who writes prompts? Who creates training data? Who does model-fine-tuning? Setting adurable partnership structure between AI and partner teams in the world of GenAI
Abstract:
GenAI has blurred the roles and responsibilities between AI and partner teams. In traditional
ML efforts, AI trains models and PD teams integrate those models; everyone has clear roles
and responsibilities. In GenAI, prompt engineering and fine-tuning models - at the surface appear highly accessible to non-ML experts. Training datasets are easy to curate and many
vendors are providing simple fine-tuning endpoints which are accessible to software
engineers. It is now easy for software engineers to create an AI POC. When the decision ismade to go from POC to production we need to figure out where and how to bring AI
specialists into the project.
At Intuit, we have defined Squads, a partnership model between a centralized AI organizationand our product and engineering partners. The Squad model defines the roles andresponsibilities of each function in the cross org AI projects, as well as meeting structure andhow to share reporting responsibilities to separate sets of leaders. By defining a durable
model which we can apply to new projects, we are ensuring that the pain points weexperienced in early GenAI efforts are not repeated in subsequent efforts. In this talk we planon describing the Squad model and the motivations behind decisions made in the model.

Potential discussion points:
Should prompt engineering be done by just data scientists?
How do we evaluate end-to-end applications so everyone is aligned on performance
metrics? When fine-tuning models, who collects the data?
Who trains the model?
How do we set up projects for success, so everyone has clear alignment on their roles?
Should Data Scientists join daily standups? Should we track DS work in the same way SWE
work is tracked?
How do we handle written communication? What conversations happen in shared channels
and what conversations happen in function-specific channels?
Relevance to workshop
GenAI’s popularity and accessibility has upended the traditional working relationships
between AI, Software Engineers, and PMs. Producing demo-ready AI code used to requiresignificant AI expertise. Now, anyone can create a working demo with a well-engineeredprompt. As a result, the traditional roles and responsibilities across AI projects require
re-addressing in the context of GenAI. Figuring out how to navigate these complex andchanging dynamics is directly relevant to the topic of the workshop.

Info about the presenter:
Conrad De Peuter is a Senior Staff Data Scientist and Manager at Intuit. He has worked on
deep learning models in the document understanding space, as the AI lead for specialprojects on delivering faster tax experiences, and most recently as the lead for a portfolio of
product-focused R&D projects at Intuit.

Joy Rimchala is a Principal Data Scientist leading AI research and innovation in domains
including document services and AI safety. Joy’s current focus is to scale and enhance AI
native capabilities – enabling scalable, secure, and compliant document intelligent service. In
addition to document understanding, Joy has served as one of the lead architects in AI safety.
Joy has presented her works at WiDS, NVIDIA GTC, and O'reilly AI Conference and NeurIPSworkshops. Joy has served as a program committee and a reviewer at leading conferences
including ACL, EACL, EMNLP, and AACL-IJCNLP. Joy holds a PhD in Biological Engineeringfrom MIT with a focus on parameter estimation methods in cell decision processes.
Info about the company

Intuit is a leading financial software company that helps millions of individuals, small
businesses, and accountants manage their finances. Their innovative tools andservices, such as QuickBooks and TurboTax, simplify complex financial tasks andimprove productivity. Intuit is focused on creating intuitive, accessible, and reliable
solutions to drive financial success for their customers.