Adapting Retail Analytics to the Pandemic
By: Loyalty Science Lab
Tags: Analytics, Retail, Customer Experience, Marketing, Business

An in-depth conversation with two Teradata retail analytics experts

Image by Loyalty Science Lab

The COVID-19 pandemic has brought significant disruptions to the retail industry. Store shutdowns, supply chain issues, and changing consumer behaviors are all creating challenges for retailers. What has really changed? How should retailers adapt their analytics to the new reality?

To find the answers to these questions, our Lab Director, Dr. Yuping Liu-Thompkins, spoke with two retail analytics experts from Teradata:

Dr. Bonnie Holub , Managing Partner, Data Science, Americas. Dr. Holub has a Ph.D. in artificial intelligence. She has spent her career correlating different sets of big data for actionable result. Dr. Holub is a member of the Loyalty Science Lab’s Advisory Board.

Dr. Holub has a Ph.D. in artificial intelligence. She has spent her career correlating different sets of big data for actionable result. Dr. Holub is a member of the Loyalty Science Lab’s Advisory Board. David King, Industry Consulting Principal for Retail, CPG, and Communications, Media, & Entertainment. In his role, Mr. King works with Teradata sales leadership across the Americas to identify solutions and answers to problems facing specific industry verticals, including retail.

In this article, we bring to you these experts’ detailed insights on:

major changes in the retail industry due to the pandemic;

adapting retail analytics to the new environment;

current retail success cases;

leveraging analytics to create a frictionless customer experience; and

managing changes in the customer journey.

Pandemic-Induced Changes in the Retail Industry

Yuping Liu-Thompkins: Bonnie and David, thank you for doing this interview. I’d like to start by asking what are some of the most significant changes among retailers and retail customers that you have seen over the last six months or so.

David King: I think the impact of the pandemic has been really wide-ranging. The COVID has brought winners and losers. The winners are obvious. The grocery stores have done phenomenally well; do-it-yourself home improvement stores like Home Depot and Lowe’s have reported record earnings; and take-out restaurants are also doing great. There are also losers. If you are based in a mall, you’re probably getting to know your bankruptcy attorneys at this point. Formal clothing stores are also not doing so great, because nobody really needs a new suit right now. We’re all working in pretty casual stuff from home. Pretty much anybody without an e-commerce platform is also suffering.

Of course we see an e-commerce explosion, as Amazon continues to set new records. With that, we’ve seen an acceleration of a trend that was already starting toward omni-channel, whether buying online picking up at store, or having it delivered to your home, or curbside pickup.

Finally consumers have seen shortages and have been hunting for things. We’ve all heard of jokes about toilet paper or paper towels. Even now, for somebody who likes diet drinks, an aluminum can shortage is causing Coke and Pepsi to make far fewer diet drinks than they were before. They are pushing out a set of SKUs that are much narrower, and maybe using COVID as an excuse to change their product line.

Bonnie Holub: Absolutely. I think that we have seen an exponential increase in the power of data and analytics. People were certainly doing clickstream, bounce rates, and slot rates before. But slot rate as a key performance indicator has become even more important in COVID-19 as we saw the system get overloaded. I have been a ship-shopper for several years. I have ordered my groceries that way, and I am used to ordering groceries and frankly getting it delivered in two hours. But during COVID, I would find what I think would be a delivery time that works for me. I would order my groceries, and by the time I was done ordering, that opportunity would no longer be there. I would have a full grocery store cart and be told now I have to wait two weeks to get it delivered. Well, If I were planning on making something for a family dinner this weekend, that would be a big problem.

So I think we have seen a bunch of new analytics come forward, and analytics have become even more important. It used to be something nice to have, and people kind of looked at their online site. When that becomes your primary portal for engaging with your customers, all of a sudden the analytics become super-heated with not only the higher volume but also new analytics that are emerging.

Adapting Retail Analytics to the Pandemic

Yuping Liu-Thompkins: I have observed similar behavioral changes myself. Bonnie talked a little bit about the increasing role of analytics. Given the current changes, how should a retailer’s customer analytics strategy adapt? Should anything be done differently from an analytics perspective?

David King: Well, if you think about it, COVID for some businesses may have cut what their 2021 budgets look like. But I think a disproportionate amount of the analytics budget will go to the e-commerce side of the house, disproportionate to maybe where it was when people started 2020. Bonnie said that clickstream was important but is now exponentially important. People are going to want to understand conversion in micro slices and really be able to get at what’s driving improved conversion, etc.

More broadly than that, I think one of the things that people are being asked to do that is really tough is to connect the data from stores and from e-commerce, whether it’s from the desktop, a mobile app, or call center data. A number of places are making it available so you can call and order something that way or report issues. Connecting all your data across silos is crucial now, particularly in a world where maybe somebody started their order online but then they picked it up in the store and they finished it there. We’ve seen an example analytically where people order something that they can see is available in that store right now; but when they say “great, I’ll take it,” even though it’s sitting in that store right now, they’re going to have to wait 11 days to ship it to the store for them to pick up a different version of the exact same SKU. Things that made no sense. Retailers need to analyze the supply chain even more to understand and deal with shortages.

I think one challenge that everybody is going to wrestle with is forecasts. I mean there’s always uncertainty in a forecast, but maybe now more than ever. Forecasts start by looking in the rearview mirror at what has happened, like normally in February we see this type of trend. Now you’re going to have this challenge of understanding right now what’s going to happen. Also when you try to look forward to 2021–2022 and you say “we’re going to base that on what happened in 2020,” are you really? You probably need different proxies. So there’re going to be new algorithms and proxies needed, not only for now but also for when things get back to “normal”.

Yuping Liu-Thompkins: David, you raised a really good point about connecting data from different sources. Do you see retailers adopt the idea that they have to better connect different data sources? Are they already moving toward that direction, or would you say a lot of retailers are still kind of scrambling right now?

David King: Both. I think they’re scrambling, but I think they really are trying to get connected. In something that I was in, people would say, “how many people who came online then called within, say, 24–72 hours?” The challenge is that people would like to do it, but it’s often difficult to have the data sets available where they can be connected. You have to find that common identifier and connecting on that. So that’s the challenge. I don’t think it’s a lack of want or interest. Sometimes it’s the lack of ability, and the challenge I think is “Gosh, it’s going to be expensive to figure out how to do that.”

But I think now people are realizing that it’s essential. Often companies are in functional silos. People use that in a pejorative way, but it was just that they are in their own business unit. Customers don’t care. They kind of assume that you’re going to know them whether they’re in the store, on the phone, or on the website.

Bonnie Holub: I think that’s absolutely the case. We see in our best-of-breed customers and what they really do to transform their large complex data sets into the high-value retail experiences. They focus on the outcomes they want to do, whether it’s selecting the store’s slotting, searching or browsing, pricing to promote, recommending substitutes, managing the inventory, or fulfilling picking up or delivery.

Whether it’s the analyst, the developer, the data scientist, or the marketing professional, you need to be able to do that on a persona-based user experience, determining that I need a $10 coupon to lure me back to close out my abandoned cart. Of course these professionals are all working with a variety of tools such as APIs, SQL, Python, R, or Java. All of those tools have native environments whether it’s RStudio for R or other kinds of environments and tools. For folks developing a product, they want to bring these tools together, look at all the data all the time, and integrate them. The data come from very different sources, whether it’s CSV, JSON, PARQUET, or any number of different sources. Depending on the client, they might be doing this in the cloud, on premise, or on a hybrid system. There is a lot of preparation to be done so that the data sets are available for people to be as productive as they need to with the tools they are using, so that they can be out there creating the customer experience. There’s a whole connected ecosystem, and analytics play an important part of it.

Image by Bonnie Holub

Retail Success Cases

Yuping Liu-Thompkins: As companies adapt to these changes, have you noticed any retailers that stand out during this time? Are there retailers that have been doing a great job handling the disruption and have been using their analytics well to help transition the business to this special time?

David King: When I think about the ones that have done well in this, a couple of them you would expect them to do well. Amazon has just rocked on, and that’s almost such a standard answer that people don’t even pay attention to it any more. But with the enormous growth they’ve had, they still have the warehouse capacity and the delivery capacity to find ways to execute. That’s why they’re having record sales. Although with all the delivery cost and earnings pressure earlier, overall they’ve moved their way to that and are doing well.

Bonnie Holub: Importantly, Amazon has been standing a fleet of delivery trucks on the ground very quietly, kind of under the radar for the last three years.

David King: So that played really nicely for them. Walmart has had some growing pains from this. Obviously people like going in Walmart, even though at times they have been caught in the middle of some of the political things that have happened, like people asking “can I go in without a mask” and all that kind of stuff. Despite the visibility of things not going perfectly, they’ve done really well with their curbside pickup that may be of use. Have things always gone perfectly? No. We’ve seen plenty of examples where they had to substitute or they were out of stock on things. But what I would say for Walmart and for some other retailers is that, because they were positioned to work at it, even if it was imperfect, they will have learned so much that they’re going to have a big edge going forward.

I’ll tell you two others. One is Instacart that I think has been incredible through this, because they were able to ramp up their staffing, by a hundred thousand or two hundred thousand people. They can adapt when nobody else can, which is great. But they’ve got an interesting combination of doing things in supply-chain combined with a really smooth app experience that they do for a lot of different grocers. I think in people’s mind they are realizing this is Instacart, whether it’s Kroger or Loblaws or somebody else. They built a lot of brand equity through this.

One other who has done well because they had e-commerce before is William Sonoma. You would think William Sonoma would get really hurt in a situation like this. But because they had invested already heavily in e-commerce, they were able to swing and adapt. Because people are making their own sourdough bread, they had just the right thing to cook it in and different things for people to use when they are home. So they have thrived because they were able to keep the experience going well.

Bonnie Holub: William Sonoma is really strong from the data analytics perspective because they were gearing up for this for years in advance. They had optimized their online experience to near the high-touch, high-feel experience that you get in stores. No, you don’t smell the bread baking as you’re walking into the store. But when you go on the website, you find recipes and serving ideas, and you get a richer experience for having been there. That’s part of their curation and value-add that they bring to it. They’ve figured out how to do that efficiently online at speed and at scale, and had done the homework necessary to prepare them to succeed in that. So they are doing well in this.

Photo by Markus Spiske on Unsplash

Leveraging Analytics to Create a Frictionless Customer Experience

Yuping Liu-Thompkins: Given all the issues caused by the pandemic and the importance of frictionless, how can customer analytics be used to empower retailers to create a more frictionless experience? Is there anything that retailers can do on the analytics side to help with that process?

David King: Sure. As always we have to start by thinking about the customers’ end-to-end journey and identifying the pain points that are there. We talked a lot about grocery. Let’s keep the groceries theme going even though I think this applies to a lot of other parts of retail. When we think about groceries, buying online and curbside pickup, their analytics to look deeply into what’s creating the friction should look at the conversion rates, the bounce rates which are when someone comes to the site and immediately leaves, or other places of exit where people find things difficult and get out.

The other thing I’ll say is to read and analyze open-ended comments, because customers are often quite generous and telling you what you’re doing wrong. Companies often struggle to figure out how to digest the gift of feedback that they get. 4 or 5 quick examples. One will be slot capacity. It was mentioned before, if I’m going to have to wait a week, that is pretty annoying. What do I need to do to create more slots? Do I need to hire people? Do I need to create more parking spaces? Do I need to get more capacity to go pick that order? Analyze each of those pieces to figure out what’s the bottleneck and how to solve it.

Going online and thinking about browsing and search. When Bonnie talked about her shopping, she probably spent a long time looking for all those items. If you are just buying one thing on Amazon, that’s not too bad. If you suddenly ordered 30 or 40 items of groceries and in the end you find out that something goes wrong, you’re going to be angry. So how can you check the path that people take and whether they are able to find items through browsing or using a search function? In our observation, after being spoiled by Google, search capabilities are terrible on a lot of retailers’ sites. So trying to get search capacity working and using some tech analytics would be good to see what’s going on there.

Then with recommendations, whether when someone is on the site or through emails, you hope that those are personalized and that they’d better be right. They better have something to do with your own interests and behaviors. I had one retailer who was convinced that I was a teenage girl. I am not! So pretty important for those recommendations to be relevant.

Substitution is another huge pain point. There’s nothing worse. You’ve waited. Then when you get there, and they say, “We didn’t have this, but we brought you that.” Try to get really targeted analytically about what it is that this customer cares about, thinking about segments. Do they care about having the same brand? Do they want the same value? Do they need the same size? If there’s no 2% milk, would they accept 1%? Or do they prefer to go organic? Getting to know something as simple as “we don’t have this, but I can offer you that” is important and takes a lot of analytics. Or maybe even just know the customers who say “Don’t substitute! Don’t even try.” You want to arm somebody with some analytics to help them get to that.

Finally you can use analytics on an abandoned cart. Let’s say Bonnie went on the site and spent a lot of time. She got annoyed because the delivery slot wasn’t there. Maybe a general email with a coupon for $10 off might get her to go back in and finish that order.

Managing Changes in the Customer Journey

Yuping Liu-Thompkins: I would like to follow up on something David said earlier about the customer journey and how it might be changing during this time. For companies who have already mapped out their customer journey in the pre-pandemic environment, do you think their customer journey should be remapped to see what the new customer journey looks like in the special situation now?

David King: I think there are a couple of things with the journey. Some elements of the journey may stay the same. Other parts may be different. For example, you can’t walk into the store. That has to change. But I think there’s another part of the analysis. Let’s say there’s a cool website that is used to customers who are a lot younger than I am. Let’s say a 25-year-old sipping a craft IPA while streaming and sitting there ordering with one hand on the phone and getting all set up. Now suddenly his or her parents or grandparents are brought into this. Not to say that the older folks aren’t often digitally literate. Many are. But as a percentage, it’s a chance to get to people who would normally not have tried this mechanism. Back to the point of the journey and the persona idea, I think it’s important to recognize that there may be some different ways that carts can be created. Maybe there’re some standard ones saying, “There are four standard order packages. Would you like to start with that?” So you don’t have to go hunt for tomatoes, milk, cereal, or whatever. Then you can still add a few things to it. I think there’s a chance to really revisit that customer journey to try to bring in some people who may not be as familiar and comfortable with the process.

The other part that gets tricky is that we are often dealing with families. Maybe mom and dad don’t really care about this, but maybe there’s an influential teenager who does. So you create you offers based on that, but it does get tricky sometimes trying to figure out who to send that offer to, who’s the decision maker, who’s the influencer, all the things that go into understanding the activities of a household versus the activities of an individual.

Bonnie Holub: There are extensions because of the COVID too, with the number of people who are buying for elderly parents and delivering food to them. So all of a sudden the shopping cart is getting more complex.

Yuping Liu-Thompkins: This issue of social customer journey is an important topic in academic research as well. I’d like to ask one final question. Our previous research shows that when people are feeling negative or uncertain about what’s going to happen next, they often start to search for things and try new things. That psychology has played out during the pandemic, where we see a lot more consumers trying out new brands. Some of that is because their old brand is no longer available, but some of it is also people proactively looking for new things to try. I came across a recent study that shows these new customers acquired during the pandemic may be quite difficult to retain. From your perspective, if a retailer was able to acquire all these new customers, what can they do in terms of marketing and analytics that can help them keep these newly acquired customers, so that after all of this is over, the consumers will still want to stay?

David King: I find this question really interesting. I believe that some customers may have moved from their normal grocery store to a nearby competitor because that competitor suddenly had more slots available now instead of having to wait. Or maybe they had something I wanted, like they had toilet paper. But if the new store does not deliver on that promise, the customer might switch right back. Think of a restaurant chain. I decided to pick it up to go, and I hadn’t done that before. When I got there, I had to wait an hour and 20 minutes. I’m never going back. I’m laughing bringing this up. Again Coke and Pepsi have cut back on their brand. So here is some generic Kroger Big K Diet Citrus Drop. I’m drinking this right now, and it’s actually pretty good. It’s also very inexpensive. So I’m probably going to stay with it.

This brings up the other point that companies are thrilled to have an opportunity to demonstrate their offerings to some new customers. So often their pipeline is primarily their returning customers, and they don’t get that many new customers, which makes growth tough. So this is a rare blessing of new faces walking in the door or clicking on the site for the first time. Even though they may not stay, a lot of brands are excited to get a chance to go to bat.

Bonnie Holub: From the analytics perspective, when I read the report, one of my takeaways was that millennials and other cohorts dominate many factors related to loyalty, including social and environmental responsibilities. I think brands that are able to integrate vast amounts of data sets and look at all of that to get a more holistic view of their customers are going to do better. It’s not just that they bought this organic cotton shirt that was developed and done in a fair trade environment with fair market labor. It might have been that I was looking at the website and learning more about the program where every dollar from the shirt will get donated to a school in Angola that I care about. There’s a whole context about why I’m making a buying and perch decision now. It could have to do with the “statability”; it could have to do with environmental goals; it could have to do with diversity and inclusion goals. Many factors go into a product selection that maybe weren’t as important as even a few months ago.

Brands need to be able to tell that story effectively, to connect you with the people who are benefiting socially from doing business with them, to the causes that are being promoted, to the responsible environmental and sociological components that are connected to the business. If you can tell that full story, I may be more predisposed to work with you. Again, it’s more and more data coming. Some of those data may be third-party data from some NGOs that provide us metrics on how things are getting better or worse, and the results of my business transaction would be related to that.

David King: Beyond retail, Bonnie and I are in a business where we’re normally getting on airplanes a lot. So we are normally in hotel chains, on airplanes, in Uber’s, and all of that. I’ve done virtually none of that since March. It’s interesting watching the communication from some of the folks we know. We are frequent fliers. Some companies tell us, “Well, you know you’re going to lose your miles next year if we haven’t seen you in a while.” Then you got others who are not like that. Their message is: “We know that you’d love to be flying with us right now, and that you’re working from your basement. How do we stay in touch?” It is a really interesting loyalty challenge. How do you deal with us? It seems like we’re not being loyal right now, but you assume when things get back to normal we will be loyal again. How do you plan to keep reminding us that you care about us, so that when we can travel again, hopefully we will be staying at your hotel?