Case Study

INSTACART.COM

Instacart Taps Confluent to Put(and Keep) Its Data in Motion During
Unprecedented Growth
Instacart is the leading online grocery platform in North America, partnering with nearly
700 national, regional, and local retailers to offer delivery and pickup services from morethan 65,000 stores.
At the onset of the COVID-19 pandemic in early 2020, Instacart experienced its highest
Headquarters
San Francisco, CA
Industry
Grocery
Challenge
Design and maintain a resilient
IT infrastructure that can ensure
continued seamless grocery delivery
during a period of unprecedentedgrowth.

customer demand in history. Its customer-facing applications, backend systems, andbusiness processes were all put to the test as millions of people turned to Instacart as anessential service to safely get the groceries and goods they needed.
“We experienced about 10 years of growth in six weeks,” said Dusty Pearce, Vice Presidentof Engineering at Instacart. “How do we help these users who’ve maybe never used an app
or done e-commerce before? How do we rapidly change our product to help them get what
they need? How do we onboard 500,000 shoppers across the United States in six weeks?
All of these challenges wrapped up into one was just a really wild ride.”
Instacart required a true cloud-native service that could elastically scale and deliver theresilience needed for an always-on service to support its growing business.

Solution
Confluent Cloud for a real-time data
platform that unlocks the full value of
streaming data and empowers data
visibility, agility, and flexibility acrossa rapidly growing organization.

They chose Confluent’s platform for data in motion to connect the disparate systems andResults• Better inventory management via
real-time data

innovating on core products and offerings for customers, shoppers, and partners.

applications required to update inventory in real time and support Instacart’s end-to-end
order fulfillment process from the time an item is added to a cart to when it’s delivered to acustomer.
By leveraging Confluent’s fully managed cloud service to support its underlying data
infrastructure, Instacart is now able to better allocate valuable technical resources toward• Reduced TCO• Improved fraud detection• Faster execution“Ultimately it comes down to support. When I call Confluent, I know I’m alreadygoing to be talking to a Kafka expert. They don’t have to go find someone.
And that matters when minutes or even seconds are on the line for uptime.”
Dusty Pearce
Vice President of Engineering, Instacart© 2022 Confluent, Inc. | Confluent.io1

Case Study | Instacart

Building the infrastructure to support millions of customers
Few companies grow as fast as Instacart did in 2020, going fromtheir table at a time when physically going to the grocery store

being available in about 1,500 stores in 30 states to more thanhad, for many, become a scary prospect.

28,000 stores nationwide and in Canada, and serving more than
85% of U.S. households.

“It was really a nightly exercise of, ‘What can we do to create
headroom for the wave, the bigger wave of people coming

With essentially no roadmap for managing such a swift and large

tomorrow?’” Pearce said. “And that was a big part of what mytransformation, the engineering team at Instacart had to think

infrastructure team was working on, primarily with data systems.

both fast and in real time to keep their data discoverable andData systems are the ones that are the most brittle, but at theaccurate, and to uphold Instacart’s mission of making it as easysame time, our product team and our executive team were looking

as possible for American and Canadian families to put food on

at all of these new users.”

Going beyond Kafka for massive scale
Already familiar with Apache Kafka® because he’d used it atexpertise, so we really needed to find a partner who could help usprevious jobs, Pearce knew Kafka was the technology Instacart

push this platform forward at the speed that we needed to moveneeded to adopt to keep their massive inventory system runningwithout me waiting a year or two years to assemble all of thesmoothly during this unprecedented time.

staff in a really competitive market. It just felt like it was the right“I’m very familiar with (Kafka) and understand the platform, and Itime.”

knew that it was a platform I could trust,” Pearce said. “However,Pearce and his team chose Confluent Cloud and immediately

I didn’t have the time that I had before to build a team and build

began to weave Confluent into its IT infrastructure to support key

this from the ground up. I had to jump in and just start leveraging

inventory management and other use cases.

streaming systems almost immediately.”“The famous line I’ve always used is when you call your service

Instacart had begun to use Kafka for a few initial use cases,

provider, you don’t want the support desk to go find their Kafka

including Customer 360, relying on their cloud provider for Kafka-expert,” Pearce said. “So Confluent was a natural choice for ourrelated managed services. “It worked to some degree, but we had

Kafka infrastructure.”

struggles with support,” Pearce said. “We also lacked the internal

The results:
The right product at the right time for every customer
Better inventory management via real-time data. “Confluent

Improved fraud detection. “When I think of our first few wins thataugments the experience in a way that allows us to use data towe’ve got with Confluent, the one that stands out to me is fraud,”

more accurately track inventory,” Pearce said. “By using data in

Kupp said. “We plugged in, spun up Kafka and Confluent, spun up

motion and real-time data, the likelihood of you getting not only

the team around this, and in a couple of months we were able tothe apple that you asked for, but a really good apple, goes up

put a system in production that’s solving this for the team. That

tremendously.”

was awesome.”

Reduced TCO. “It’s funny to say this because we’re a big

Faster execution. “Confluent has really been an accelerant toengineering team, but we’re always under-resourced,” said Natethe work that my team does,” Kupp said. “I talked earlier about

Kupp, Director of Engineering, Data at Instacart. “For me to go

taking the ops burden off my team, but I think actually, Confluent

hire a bunch of engineers to babysit Kafka, I don’t have the ability

really unlocks a faster execution for my team on a number ofto go do that. And being able to offload those concerns is justdimensions. Just things like spinning up Kafka clusters and getting

such a relief for us and lets us focus on delivering value to theprototypes up and running very quickly, Confluent has been reallyorganization and not worrying about ops and other overhead.”

helpful there. Those sort of exercises might take a long time formy team to do if we were doing this on vanilla open source Kafka.
With Confluent, we can turn that around very quickly.”© 2022 Confluent, Inc. | Confluent.io2

Doctype
Case Study| Document| Instacart
Title

What’s next for Instacart’s data infrastructure?
For the future use of Confluent, Pearce and Kupp said they plan

For Pearce, unlocking the full value of Confluent within Instacart

to focus on making Confluent the backbone of Instacart’s data

really comes down to tapping into the full potential of Instacart’s

architecture to further enable the current use cases and alsotalented engineers to improve the customer experience. “You will

unlock the full value of data in motion across the organization.

start to see more and more software development teams thinking“We’ve been thinking a lot about things like fraud, where weabout reasoning and how we can leverage real-time data to build

want to be able to have real-time fraud detection and be able
to build that into the product,” Kupp said. “Or our availability
algorithms—how do we understand if a particular item is available
at a particular store? Those sorts of things. There’s so many

features and to return value back to the customer in ways that I
think weren’t possible in the older batch data model,” he said. “Ithink Confluent will unlock the creativity of a lot of really smart
people who really understand their customers, and that can’t beopportunities like that across the company, and I think a lot of it

anything but good.”

comes with moving to streaming from these batch systems that

Ultimately, Pearce said, Confluent will become the backbone forwe had before.”keeping data in motion through Instacart’s continued growth.

Ultimately, Kupp said, the vision for Confluent is to use it as“It was really about how do we partner with someone who

a data mesh and central nervous system for all of Instacart.
“As we look across our distributed organization, there are data
ownership concerns associated with our data mesh strategy
and the Confluent features that enable and empower all of ourorganizations to own their own data,” Pearce said. “And thenthere are specific ways that we can incorporate real-time data

knows a lot about this space, has a lot of exposure to keeping
data in motion, and understands the data mesh strategy we’vebeen working on,” he said. “Now we have to operate and run
the software and company at this much bigger scale; Confluent
has been and will be critical to us succeeding as this much larger
enterprise.”

into software development. And then there are other parts of ourbusiness, like our ads business, that want to move quickly, wantto understand user behavior, and try to inspire users with content
they value.”© 2022 Confluent, Inc. | Confluent.io

Learn More About Instacart
https://instacart.com

3