Inside the Game: Unlocking the Consumer Issues Surrounding Loot Boxes – An FTC Workshop

August 7, 2019

Segment 2

Transcript

PATRICK: We'll present recent academic research about loot boxes. This afternoon's panellists 

include David Zendle from York St. John University, Andrey Simonov from Columbia 

University, Adam Elmachtoub, also from Columbia University, and Sarah Domoff from Central 

Michigan University. As with the prior panels, each of you will have the floor for about 15 

minutes, and then we will move straight on to a moderated discussion. 

We're running a little bit behind, so please try and keep to your allowed time, and don't get 

offended if during the moderated discussion session if I have to cut you off. As a reminder, there 

are comment cards. If you'd like to submit a question to any of the panellists, we have a 

paralegal. You can raise your hands, and the paralegal, Emily, will come over, and she can take 

them, and we'll submit them. OK, without further ado, David, let me turn the podium over to 

you. 

DAVID ZENDLE: OK, thank you. OK, hello there. My name is Dr. David Zendle, and I'm one 

the world's leading experts in the potential for harm present in loot boxes, something that hasn't 

really been covered extensively so far today. And if you're wondering what it looks like to be 

one of the world's leading experts on this, this is a subset of the journal articles that I've 

published on the topic this year. 

They are in What you call Q1 journals, which only take the most scientifically valid research. 

Please, believe me what I say I'm an expert about speaking from a position of authority. The stuff 

I'm saying today is real, and it is very worthy of your consideration. 

There's one clear message that I want to get across today, and it stands in stark contrast to 

everything we've heard so far, or mostly everything we've heard so far. The message is this, 

spending money on loot boxes is linked to problem gambling. The more money people spend on 

loot boxes, the more severe that problem gambling is. 

This isn't just my research. This is an effect that has been replicated numerous times across the 

world by multiple independent labs. This is something that the games industry does not engage 

with. 

Today, I want to talk about this link, and I want to talk about why you should care about it. The 

reason you should care about it, are the two words, problem gambling. Problem gambling refers 

to an excessive a disordered engagement with gambling activities that is typically outside of the 

gamblers volitional control. 

It is incredibly harmful. It's linked to depression and anxiety. It causes financial distress, the 

destruction of families, and most importantly perhaps, it leads to people taking their own lives. 
Problem gambling is the reason why some parents come home at night to find that children are 

not there. This is so important. It's not something we should trivialize, or laugh at, or compare to 

baseball cards. This is life or death. 

That's what I'm here to talk about today. So the reason why problem gambling is such a big topic 

when it comes to loot boxes, and why people care about gambling and loot boxes, is because loot 

boxes look so much like gambling. Both when you're playing on a roulette wheel or while you're 

opening a loot box, your wagering something that you have in your hand of value now on the 

uncertain hope of getting something of greater value later on. It's that reason that loot boxes have 

tripped gambling regulations in a couple of countries within Europe, because of those formal 

similarities, and because of those formal similarities, people are a bit worried for a very long 

time that loot boxes by act as a gateway to problem gambling, particularly amongst young and 

vulnerable populations. 

We set out to find out more information about this. We went to-- we start out go to large samples 

of gamers, big, big samples of gamers, and we found out how much they were spending on loot 

boxes. And then we measured their problem gambling severity using a standard instrument from 

the gambling literature, called the Problem Gambling Severity Index. it's very commonly used, 

very well known for predicting real world things. 

And what we found, was this. So this is the first time we found it. Again, this has been replicated 

many times across the world. This isn't a new effect. We've known about this for a while. 

You've got people who have no gambling problems. And they tend to not spend much money on 

loot boxes. And then as people's gambling problems get more and more severe, they spend more 

and more money on loot boxes. And the effect associated with this is something that we call 

clinically significant, meaning that it's large enough that people should maybe take notice of it. 

And you see it again, and again, and again, and again. 

Now, some people might say, OK, you've got a link between how much people are spending 

more money or boxes and they're more severe problem gamblers, does that show that loot boxes 

are a gateway to problem gambling? That is one interpretation to this effect. I'm going to go 

through others. 

But it's the interpretation that fits with the theory of how problem gambling develops. We know 

that one of the main pathways to problem gambling is a process of conditioning, whereby the 

gambler comes to need and expect the excitement associated with the gambling win. So what we 

think-- one of the possible explanations for this effect, is a situation in which people are buying a 

loot box, getting excitement, buying a loot box, getting excitement, buying a loot box, getting 

that reward, getting that hit, going out into the real world, seeing something that has many of the 

formal characteristics of the loot box, like a slot machine, and that conditioning transfers over. 

So therefore, spending money on loot boxes, literally causes people to engage in gambling, 

leading to problem gambling. 

However, one thing to note about this, is that it's a correlation. And because of that, we can't 

determine the direction of causality from it. We won't know this director of causality for many 
years, by which point, one could argue, damage has been done. This might also indicate a 

relationship that flows the opposite way, where people who already have severe gambling 

problems spend more money on loot boxes because of what is driving those gambling problems. 

And that's a theoretically valid explanation as well, because after all, problem gambling is a state 

of affairs in which the individual is engaging in excessive and disordered spending or gambling 

activities that are beyond their volitional control. 

They can't control this. So you could be out spending, spending, spending, spending, spending on 

slot machines uncontrolled. And then you go home, you switch on your favorite game, and you 

see something that looks a lot like a slot machine, so you start spending on that too. 

We don't know which of the cases is true. We don't know which of these is right. But we think 

that in either case, it's a clear cause for concern, and it's not something to be trivialized. 

In one case, you have a mechanism in games that many children do play, that is literally causing 

a state of affairs which is enormously destructive. And if loot boxes do cause problem gambling, 

we're looking at an epidemic of problem gambling the state of which the world has never seen. 

But at the other case, if that's not true, and I'm totally open to that not being true, totally open the 

other state of affairs, be the case, if that's true then you've got a system in which games 

companies are differentially profiting from the most vulnerable of their consumers. Problem 

gamblers already have enormous going on in their lives. They don't have their body taken away 

from them through this as well. 

So that is-- you'll notice at the top, there are URL links. Those will link you to each of these 

studies. But more than that, they'll link you to the data for each of these studies. 

I adopted a process called Open Science. It's a set of practices, one of which is that I always 

openly share all of my data. If you want to have a look at the data for these studies, if you want 

to see what it's like, it's out there. It's free for you. Go have it. 

And other people have been having it, and they've been finding similarly worrisome things. This 

is a study by Aaron Drummond and Jim Sauer out of New Zealand, where they sort of looked at 

how much money problem gamblers were spending by reanalyzing our data, which is available 

freely to anyone who wants it, and they found that problem gamblers were spending enormous 

amounts of money on loot boxes and suggested that maybe setting limits might be a good idea. 

But beyond those effects I've talked about, there's one very important topic, which is children 

and adolescents. 

So contrary to what you may have heard, a recent study conducted by the government in country, 

in the UK, the UK Gambling Commission found that as many as a third of children aged 11 to 15 

had opened a loot box, which was alarming. It's alarming because we know that engagement in 

gambling activities in childhood and adolescence is a key driver of gambling problems in 

adulthood. We don't know why children and adolescents are particularly vulnerable or 

susceptible to the development to gambling problems. 
Some people say it's because it's a turbulent social time, and they find gambling as a means to 

cope with that. Other people say, oh, there are neurodevelopmental explanations. It's to do with 

brain plasticity. But for whatever reason they're very vulnerable. 

So we set out to find out if the same link between problem gambling and loot box spending 

existed in adolescence as well. And guess what? It does. 

In fact, it's much, much stronger than in adults. This isn't new research. This has been out for a 

while. So you have the same relationship replicated in adolescence, but that's no surprise because 

this replicates everywhere. 

Another interesting thing we did, was we asked those adolescents why are you buying loot 

boxes? And they gave us a number of answers, some of which you might have predicted. Others 

of which, you might not have predicted. 

When I read out these things bear in mind, these are older adolescents. They're aged 16 to 18. 

These are teenagers. 

So some of them say things like gameplay advantages, which you might have seen coming. I feel 

pressured to get new gear, continue to compete with ever changing boundaries of what's classed 

as good gear. New gear is constantly-- is added constantly, and thus, gear quickly becomes 

outdated. 

Or I enjoy the game. I compete with friends. I don't want fall behind them. You cannot be 

competitive at NBA 2K19 or FIFA 19 without them. This is the sort of stuff we might have 

predicted. 

But we had other responses as well. And the papers are out there. All these papers are freely 

openly available. You can get access to them if you want to. 

Lots of the adolescents said that they were opening boxes for the fun, excitement, and thrills of 

opening the box itself. Here's a quote, bear in mind, this is a minor. "Shit, just feels good, man. 

Seeing other people opening hundreds and you get a few of that, feels good and keeps me 

going." 

Here's another one. "Because it's addicting and thrilling reaching into the unknown." Some of 

them talked about the gambling feeling associated with loot boxes. 

So this brings me to the subject of Kinder Eggs, if you're at the UK and EU, or baseball cards, if 

you are in the US. I used to have a lot of sympathy with the argument that loot boxes were just 

like baseball cards. I don't have any sympathy anymore, and that's because whilst you might be 

able to point to similarities between loot boxes and baseball cards, you can also point to 

differences. And I'm increasingly coming to see the baseball card line as a method by which the 

industry, like a stage magician, draws our attention towards something, whilst distracting it from 

something else. 
So sure there are similarities. But here's a difference for you. Loot box spending is linked to 

problem gambling. Here are other differences for you. Can you imagine anybody saying that's 

about Kinder Eggs? 

It's a ludicrous argument. We've never determined what's safe before by looking at the 

similarities between it and something else. Say I run a cinema, and I serve Coca-Cola to all my 

customers. Coca-Cola is a thick, black, viscous liquid, full of energy, and I sort of got a great 

deal of engine oil. And I said to you, I know what, I'm just going to swap out the Coca-Cola in 

people's cups with engine oil because it's similar in that it's also a thick, black, viscous liquid. 

You'd have me arrested. That's never been how things are done. You can't say something is safe 

because it's similar to something else. 

Let me move on and talk about the features of loot boxes. There's some people who would tell 

you that loot boxes that are only cosmetic contain only cosmetic items and are in some way 

harmless. That's a common opinion you'll hear from people. 

We were asked by the Australian government-- I present this research to governments quite 

regularly. I've presented to the Australians. I've presented to the UK Select Committee, in our 

own parliament. And the Australian's said, are there any particular types of loot boxes that are 

harmless? 

So we went out and did something called a moderation. And by the way, this is the basic effect. 

This is the problem gambling severity people who engage early in unpaid openings versus who 

pay to open, and people who pay to open the boxes have more severe problem gambling. But 

that's what you'd expect, because that's what everyone sees whenever they try and run this 

analysis. 

We tried to do something else, where we tried to see if certain features of loot boxes 

strengthened relationships between spending on them and problem gambling. And we essentially 

found that no matter what kind of loot box you had, it was linked to problem gambling, whether 

it was cosmetic or paid to when. It's an early exploratory analysis. There were some small effects 

in there. 

But generally, what we seem to be seeing is that there's something specific about the loot boxes. 

That brings me to my final slide, which is about the prevalence of loot boxes. Loot boxes are an 

extraordinary popular way of making money in games. 

They're in a lot of games. It's quite hard to find out what games have loot boxes, because they're 

not sort of labeled very clearly. So we went through, we just tried to find out how many games 

they were in. 

We went through the highest grossing Google Play games in the UK. I imagine this is analogous 

to a situation you have in the US. We found that 54% of those top 100-- 54% of that top 100 had 

loot boxes in, the top grossing games. But sort of, perhaps worryingly, 94% of those games with 
loot boxes in were PEGI rated, which is our rating system, it's an analogous to the rating system 

you have over here, were rated as being suitable for 12 or above. 

So there are lots of loot boxes in games that are played by children. And there's clear evidence 

that there's the potential for harm in them. Again, we don't know if it's that the loot boxes are 

driving problem gambling, or if it's that problem gamblers simply are drawn to spend more on 

the loot boxes because something to do with them. But in either case, it's extremely worrying. 

I'm here today, I'm aware that there's a lot of industry people. We could get this wrapped up. We 

could find out which of these is the case if you would work with us. 

There are people out here. We're not mobsters. We want to work with industry. But in order to 

answer these questions, in order to find out which way the relationship goes, people need to 

share that data with us. 

You have terabytes of data on users interaction and user spending. That stuff is what we need to 

find out which is happening, and it's just not being shared. So please, share it before someone 

decides that's the enormous, self-regulatory powers that you've been given are something that 

you're no longer worthy of holding, which is a very real scenario. That's me. Thank you for 

listening. 

[APPLAUSE] 

PATRICK: Thank you, David. Next we'll hear from Andrey Simonov. 

ANDREY SIMONOV: All right, thanks Patrick. And hello, everyone. It's a pleasure to be here. 

So I'm Andrey. I'm in faculty of Columbia Business School and-- where is clicker? This one. 

PATRICK: Yeah, that's it. 

ANDREY SIMONOV: Oh, wow. OK, let's try it. All right, here we go. 

All right, so I'm Andrey at Columbia School. And the work I'll talk about today with Tom 

Romano from Harvard. So Tom is also in the audience here. 

In some ways, this paper relates a lot to the kind of the topic of this panel, and the question the 

panel asks. And I'm really glad that David had a talk about his paper or his work before us, 

because it really highlights how many different perspectives on loot boxes there are out. So 

where this paper is starting is, we're trying to summarize all the different views we heard about 

loot boxes in two separate buckets. 

One is that loot boxes are really useful for gaming, and they enhance gaming. And this is that, it's 

voluntary to use items, and people can choose them, because they'll go to the game, and this 

items help you to make progress in the game. So it's part of the video game developers to work 

on. 
For companies, it's a great way to monetize the games. And companies have been struggling with 

this a lot. We heard about fixed cost involved, and it is one of the forms of bundling that this 

ways you can do pricing. So Adam will talk more about this after me. 

For consumers, it's a great chance to obtain useful items in the game. So if I'm stuck on a 

particular stage, I want to make progress, it's somewhat useful to open a lottery, and with a large 

probability I'm staying in the same spot, but a small chance I'm just jumping to a very different 

productivity curve. So there's the same economic arguments which were about lotteries in the 

60s and 70s in the US. 

So that's one view on loot boxes. A different view is that while loot boxes look a lot like 

gambling in a lot of different features, so you have to buy some currency. It's in-game currency, 

but it's often purchased. It's a chance to obtain the item. So even though we're sad that consumers 

could get direct utility, so in the first story it was utility for play the game, maybe consumers just 

play little boxes, because they get some utility from a risk. Utility like indirect, like utility from 

getting excited about the risk. 

And this is really problematic, because that is the same as casinos. And it can lead to problem 

gambling, to addiction, and to all stories like this. Particularly, this is concerning for minors, 

because kids play video games a lot. So what we are trying to do in this paper, is to really 

separate out these two views on loot boxes. 

One is-- our question is, how much people play loot boxes, because of the in-game functional 

value. That like the items you get will enhance this gaming utility. And how much they play just 

for the sake of playing the loot box and getting this maybe like risk preferences, like some draw 

of behavioral utility from opening a risk. 

So that's where we start. And a second way, so if we show there is some preference for loot box 

which is direct, now we might ask, OK, how much addiction there is? How much problem 

gambling we can detect about this in the data? So is there some habit formation? 

I play more today. I play today and I play more tomorrow because of this. How much of is 

moderate by the variance of loot boxes related to variable schedule reinforcements? And then, do 

people open loot boxes in certain conditions, like [INAUDIBLE] consumption. 

OK, so this will be empirical paper. We actually have data from a large video game company in 

Japan. And it will be able to separate this out in some way. But to get us started, to get us 

thinking on how-- what it means to separate this question, let me show you a very simplified 

version of the theory model just to get intuition or to imagine. Given its video game conference, 

we'll call it a toy model. 

So there was a consumer who wants to play a game and also open good boxes. One period, he 

makes two binary decisions. Do I play the game and do I open the loot box? If consumer plays 

the game, he gets the utility from the game. One is alpha j, which is I just enjoy playing the game 

whatever. Another is beta, I also enjoy the game more if I win a particular stage in the game. 
OK, so this probability win. Note that it also depends on yl, which is do they open the loot box or 

not. So that's my utility of plane. If the consumer opens a loot box, he or she also gets utility. So 

it was direct utility from a loot box, alpha l, which is potential as a risk preference. I need to pay 

a price, p and i don't like to pay money, so it is marginal disutility of money. 

That's one part of why open loot boxes. But also, if I open a loot box, there is a chance I get an 

item which helps me to advance in the game. So with some probability, my probability to win 

becomes higher because I have this new item. So the goal of the paper is really to separate out 

those two stories. 

And from the model, we want to show do people play loot boxes because it enhances the 

probability of winning, so it was a beta factor. And if that's the story, we should see that people 

will open boxes at the moment when the marginal return of having this extra item is the highest. 

In other words, if I really don't play so well at this stage, and an item in the loot box will help 

me, that's the moment I want to open the loot box and get the item. 

OK, if the second story is saying it's alpha l, it's just my preference for gambling, then it 

shouldn't really be correlated with do I lose in the game, do I win in the game? I should just be 

opening loot boxes quite a lot in general and get this utility from loot boxes. OK, so the basic 

intuition what we'll be looking for in the data. 

So the data we get is from a Japanese mobile video game. And to kind of simplify it which 

simplifies our story in some way is that you think about it as some version of Candy Crush. So 

there is not much social interactions. People open loot boxes mainly for functional value of the 

items, so it's not-- it's not about skins. It's really about these characters will help to progress in 

the game. 

We can extend analysis to social interactions, but this helps us to simplify. And so we'll be able--

having the data, we'll be able to measure what is the return of having each character to make 

progress at each stages of the game. OK, so let me tell you a bit more about the game description 

to get a sense of what is the context. 

It's a popular mobile game in Japan. It's a puzzle game where you have to-- there's a lot of stages 

you make progress in this game. Every stage is what's called a battle. And when you do this 

battle, you need to use your skill to play, but you also need to use some kind of in-game 

characters. To acquire in-game characters, you can play a lot and get them, or you can open loot 

boxes and have a chance to get a good character. 

And then characters differ in the quality. Some characters are better than others, but also in how 

they specialize. So which helps us as the game progresses, stages becomes increasingly hard, and 

it requires often different quality of characters, but also different specialization. So we can easily 

see a moment where I play amazingly at stage 50, but at stage 51, suddenly I get a very different 

set of characters. And if it's a functional value of loot boxes, that's the moment I would want to 

open them together as this new thing. 
So covers-- will give us some indication for how it matters and different things. OK, so a bit 

more about the data. We got a subset of individual level, very detailed data from this mobile 

game company on how people play, and how people open loot boxes. We have a sample of 

around 800,000 different users. 

To highlight some of things we have the data, we have different metrics of play success. So one, 

I'll show you in a bit is what a score of the player, and how many stars each score gets in a 

round. That's a very important metric for these players. 

We can see which characters they used once they played the game. So from this, we can see, 

well, if I use a particular set of characters, how successful it was. So we can measure production 

function of having an extra character, how much extra scores will I will I get. It also allow us to 

identify the moments in the game where the set of characters I have now is not really 

satisfactory. That's the moment I would want to open loot boxes for functional value. 

We see loot box realizations and distribution of outcomes of loot boxes. And in this game 

though, I think the regulation in Japan requires you to post probabilities. So for all the loot boxes 

there was probability for every character which is in the game. 

So we see whether the probability is. We see which characters who you get. We know what is 

the actual realized distribution. 

From this, we can compute what is the expected value for a player to open a loot box. So because 

we know how I play the game, how good the score is, I know what is all distribution of 

characters in a loot box, I can measure if I open it with which chance he'll be better in the game 

or not. So that's how we match the functionality of this thing. 

And then we have data on actual spending in the game, both in-game currency and how they will 

purchase this currency with real money. OK, so let me-- basically, I'll share at this stage, the 

main results we're still working on. So I'm not sure we're ready to share the main kind of analysis 

of the full model and everything else. 

But let me show you our results on the [INAUDIBLE] in question which we started with. So do 

people tend to play more-- open loot boxes more at the moment when they start losing the game? 

So as we would treat it, we have different metrics of success. 

So maybe the success is you advanced to the next stage. Maybe the success is you have to pay 

something to keep playing the stage and finally win it. Maybe success is what is the score and 

how many stars you get at this level. For different metrics we use, almost always-- and this is 

something we're still testing to be a bit careful-- we find that there is strong correlation between 

how well you've done as a stage. 

So the worse you do at particular stage par to the game, the more you'll be able to loot boxes. 

And even though-- so we have a control bunch of fixed facts, we try to be careful that this is the 

right variation to use in the data, I want to highlight this is still correlation evidence. We're still--
there is some instruments in the data we can use, but we are not ready to present results with our 

correlation. 

However, to highlight how this is looking at this on one metric, here is a plot which shows how 

much people open loot boxes as they play the game, and how well they did in this game. So here 

on the y-axis, on the vertical one, you have transform probability of people actually opening, 

engaging in a loot box. And on the x, you have different scores. 

The three vertical lines are thresholds to get one star, two stars, and three stars in this game. So 

the people who play really not so well, they also don't open loot boxes. Those are the guys on the 

left bottom corner. 

As you make slightly more progress and you're close to the one star threshold, that's where you 

open loot boxes a lot. And that's-- one of the attrition is, you want to make progress and get a 

better score in this game. And then as people play better, and better you can see downward trend 

in how much you open loot boxes. So what this shows to you is-- and this is consistent with all 

descriptive evidence we saw-- there is definitely some functional value in how people open. 

So people do open loot boxes a lot in this game for the sake of getting a character which will 

help you to progress. The slide I wanted to finish with, and kind of wanted to-- and also actually 

save some time for discussion later, this slide I wanted to kind of highlight here, is even though 

we show some suggestive evidence for this functional value, and in this we believe that 

functional value is quite important, it doesn't mean that people open these loot boxes is only for 

functional value. So what we're really trying to get out of this paper and results-- hopefully we 

will share soon-- is which share of amount of loot boxes which are opened to people, are really 

open because of functional value, or opened because of persistent preferences, this preference for 

loot boxes directly. 

Because if it's like 95% of loot boxes because of functional value, well, that's actually part of the 

video game. But if 95% is really this extra component where it doesn't have to do with how 

people play the game, maybe we should think about it more as casinos as gambling. So that's our 

takeaway for that, from the work so far. Right, thank you. 

[APPLAUSE] 

PATRICK: Thank you, Andrey. Now we will hear from Adam Elmachtoub. 

ADAM ELMACHTOUB: Hi. Good morning. So my name's Adam Elmachtoub. I'm from 

Columbia Engineering. Thank you to Patrick for organizing this session, and for all the great 

speakers today for their well-informed talks. 

Today, I'm going to be talking about a slightly different topic, very different angle. We're 

actually going to take a perspective on how would I actually design a loot box if I'm the 

publisher, developer, or the gaming company. And before I really get into it, I want to emphasize 

that we have no connection to the industry at all. So this is like a neutral perspective. 
And we also have a neutral perspective on the entire issue as well. So although we're just we're 

telling you how one would design it, we're not advocating or vise versa, OK? The reason why we 

think it's important, is that how can one design regulations without understanding the actual 

economic mindset of a gaming company, what they're trying to accomplish, which is, make 

money, right? They're all for profit companies for the most part. 

So this paper is telling us how would one optimally design and price loot boxes, and what are the 

actual outcomes for sellers and consumers. So this is work with [INAUDIBLE] at the University 

of Toronto, Michael Hamilton at the University of Pittsburgh, and Shao Lei, who is the audience. 

He's was my PhD student who convinced me to turn his PhD into studying video games. So this 

is a large credit to him. 

All right, and the paper is online if you want to see it. So here the research questions, why do 

video game companies even use loot boxes? Can we quantify the optimality of such a strategy? 

One doesn't need to use low boxes to have a video game, so why does this even exist? What is 

the actual optimal way to design a loot box? We consider many design aspects of loot boxes. 

Someone earlier today was telling us that there's millions of ways to come up with a loot box. 

Some of those tactical decisions includes, do you allow the gamers to have duplicates of items or 

not? What are the actual allocation probabilities do you use? And do you allow items to be resold 

to other gamers or back to the platform? These are all actually very important things to consider 

in loot boxes, and affect both the seller and consumer happiness. 

And finally, how do all these things affect the consumers? So given that the seller is going to do 

something to make them the most money, how does it affect consumers? Their overall 

happiness? So we'll quantify how much they end up purchasing, and what's the surplus they 

actually get. 

So the framework we're going to use is a mathematical model to answer these questions. I'm 

going to try to avoid math for today, and just tell you what the results are. You can see all those 

details in the paper online. 

But the core idea is that what we try to model is at the really most granular level, each consumer 

has some specific willingness to pay for each item. And there might be thousands of items. And 

these willingness to pays are going to vary by consumers and also by the items. 

So some things you value $1. Maybe some things you value $5. Some things you value nothing. 

They're random, but the seller somehow has a good understanding of how much these items are 

worth. 

So let me give you an actual example. So here two different people. The person in blue has 

different valuations for these six items than the person in black, and you can see those six 

numbers above their heads correspond to the six items on the right hand side. And each customer 

is sort of having different valuations for all the items. And this is the core principle behind our 

model that we're trying to capture, this heterogeneity across consumers and across the items. 
All right, so what is the seller doing in our model? They're trying to maximize revenue. That's 

always their goal. 

And there's two ways to think about loot boxes that we look at. One is called the unique loot box, 

where consumers are always allocated a new item. And the other one is a traditional loot box, 

where customers can actually potentially receive duplicates. We call it traditional because it kind 

of dates back to the idea of baseball cards where you can end up with duplicates. And unique,

again, is called unique, because you always get a unique item. 

And the way we model consumers, is that they essentially just keep purchasing loot boxes until 

they no longer perceive any value from them. And that's the central core principle behind our 

model. So I think it's best to now just show you some examples. So here's just two games, one 

where there's unique box, one does traditional box. 

And here's an actual picture of how the model works. So back to this person over here, he has 

these-- there's these six items that they're willing to pay for, and you can see how much you're 

going to pay for these six items above his head. And let's say we're selling loot boxes for $2.99 

each. So the way this model works is, the customer thinks that they're going to get one of these 

items at random. So with probability one out of six, they'll get each of the items. 

So on average, they value this loot box at $4.50 by taking the average of those six items. And 

that's more than $2.99. So in their head, they think, yes, I'm going to buy this because I value the 

loot box more than $2.99. 

So they buy the loot box. They receive an item. And now because they have the item, now the 

item is worth zero. They don't want to get it again. 

So now the value of the next loot box decreases becomes $3, which is still bigger than $2.99, so 

they purchase another one. But guess what? They got the same thing. So they still value that 

same loot box again at $3, so they end up buying again because they just received something that 

was worthless. 

So they buy again. Now they have two items and their value goes down. And now their are value 

for loot box is $2, which is less than $2.99. So now they stopped buying. And this is just one 

example of our model. So in this case, the customer bought three loot boxes and stopped. 

All right, so our goal is basically theoretically describe which strategies are best, and how does 

affect seller and consumer behavior? So actually, I'm going to take a quick poll. If you're a 

gaming company and maybe people here representing the gaming industry, do you think it's 

better to sell unique boxes or traditional boxes? 

So anyone can actually take a vote here. So if you think it's a unique box makes more money, 

raise your hand. If you think traditional box makes more money, raise your hand. 

So around 80% of people's the traditional box. Now, let's think about it from the consumer side. 

Do you think if you're a consumer, do you prefer a unique box or traditional box? So if you're a 
consumer, raise your hand if you prefer the unique box. OK, raise your hand if you prefer a 

traditional box if you're a consumer. 

So it actually went the other way. So around 80% of people think consumers would prefer the 

unique box. Actually, our research is going to show the opposite of both those things. So what 

we end up with something very counterintuitive. So first of all, we show that unique boxes are 

actually optimal for the seller, and traditional boxes only make around a 1/3 of much revenue as 

unique boxes. 

And what's happening is that basically, you can charge more for unique box, because it's more 

valuable. You're guaranteed a new item. And this in turn leads to more revenue. And in both 

cases, the customers end up purchasing roughly the same amount of number of loot boxes, 

except that a unique box is going to earn more revenue than traditional box because it has a 

higher price. 

But in both cases, customers end up buying the same number of boxes. Now what that means is, 

for the customer, if you're buying traditional boxes, you're going to end up with less items 

overall, because you have all these duplicates. Now the flip side is though, that given that 

traditional boxes actually had a lower price, consumers are actually happier in the long run 

because it turns out, the prices are a lot lower when you sell traditional boxes. 

So we find that consumer surplus, which is the sort of classical economic notion of consumer 

happiness, is a lot higher when you sell traditional boxes than unique boxes. Actually, you can 

show when you sell unique boxes, consumers are left with almost no surplus. And again, the 

driving force here is that the prices are so high when you sell unique boxes, that essentially 

leaves nothing for the consumers. But for traditional boxes, you price low enough where some 

value goes to the company and some value goes to the seller. So we end up with this sort of very 

counterintuitive situation where, actually, it's better for customers to receive duplicates, because 

the prices come down a lot. 

Now, what happens if I allow resale market, which is the gaming company can completely 

control this. They can allow you to resell or not. It depends. 

And naturally, you might think that if I'm a gamer, I want to be able to resell stuff. That gives me 

more flexibility. But once again, actually we find that it doesn't really help. We actually see that 

at most it can increase surplus by 1%, and most of the time it decreases surplus. 

Which is again surprising, and again, the driving force is that companies will actually end up 

charging higher prices, which hurts you overall. So although resale sounds like a good idea, it 

also allows prices to go up, which is not good for the gamer. Good for the seller. 

So let's talk about allocation probabilities. So this is a very important topic. And I think a lot of 

people talked about it this morning, about being transparent. 

So here's just an example of one company being explicit about the probabilities, and we saw 

several examples this morning as well. So if you're the seller, what is actually the optimal 
strategy for you? It turns out, actually the optimal thing to do, is to do the simplest thing, which 

is just allocate uniformly at random. Which is also in this case may be good for consumers, 

because they can understand this strategy. 

Simply, if there is 1,000 items, allocate each item with probability 1/1000 regardless of how 

much these items vary in quality. So for example, let's say 1% of your items are the best kind, 

legendary, which is the example I was looking at before. Then you should allocate legendary 

items with 1% probability. 

So if 10 out of the 1,000 items are legendary, then you should allocate a legendary item with a 

chance of 10 in 1000. So this is the simplest possible policy, and it turns out to be the optimal 

one. So this is in some sense good news for both the seller and the gamer. Simplest strategy is 

not only best for revenue, but it's also the easiest for the customers to understand. 

Now here's the caveat, and this is where regulation becomes really important. What happens if 

the seller lies about the probabilities? This is something that's impossible for any one user to 

keep track of, especially if you have a 1% chance of winning something, it's reasonable to not get 

it 15 times and think that's just bad luck. There's no way for an individual to monitor if their 

allocation probabilities are really being true or not. 

So it turns out, if the seller publishes some list of probabilities and lies about them, the seller can 

actually make a significant more-- significantly more amount of money, more revenue. So there 

is benefit to lying. 

Since there is benefit to lying, there must be regulation around this. Otherwise, people will make 

money. They're for profit companies. 

And this is why Apple and Google have already made such rules in their platforms to make sure 

that these allocation probably these are announced. And we already had the news this morning, 

that these allocation probabilities should be announced. But what I'm saying is that, in addition to 

them being public, they all should also be monitored actually make sure you're following these 

probabilities. 

So we need to keep track of this. And not only on the aggregate level, but also on the consumer 

individual level. So it's even possible to make more money where in the aggregate you're 

following the probabilities. If you see what everyone's getting, the probably all look correct. 

But for individuals, the probabilities may not be correct. So it's possible to gain more revenue by 

extorting specific individuals. So even monitoring at the individual level is necessary over time. 

So to wrap up, I don't think one can design regulations and policies without really understanding 

the economic or business mindset of a company. So we're proposing, basically, the first model to 

do this, and there's many sort of caveats, of course, but I think there's an important direction to 

study. So we show that unique boxes are best for companies, but actually, traditional boxes are 

possibly better for consumers. 
Allowing a resale market may seem like a good idea for consumers, but actually we show it has 

minimal value. In most cases negative value. So that's important to understand. 

We show that allocation probabilities don't have to be over thought. Actually, the simplest thing 

to do, is just allocate randomly. And that's actually best for everyone. 

And finally, be sure that there is benefit from lying about these probabilities. So this is a specific 

case where regulation is needed. If loot boxes are allowed to stay as a legal sort of way of selling 

items, then at least we need to regulate these kind of allocation probabilities. Thank you very 

much. 

[APPLAUSE] 

PATRICK: Thank you, Adam. Now we will hear from Sarah Domoff. 

SARAH DOMOFF: All right, thank you for having me. And I'm glad to be presenting along 

such esteemed researchers on this panel. I'll be speaking about children and gaming, and some 

current issues that I have encountered clinically, and then also in the research. 

So I will be presenting some gaming trends among children, parent child interactions around 

gaming, unique concerns related to current games, and problematic gaming, defining it for you, 

and helping you understand when does gaming interfere with a child's functioning. So in terms 

of gaming trends, we know that screen time is really high for children, adolescents. And actually 

the amount of time children use mobile devices has tripled in the past few years. At the forefront 

of many parents and children minds, this past year has been Fortnite. And it remains a popular 

game, with 45% of children and 61% of teens ever playing it. 

When we consider how games impact children, it's important to consider the content and the 

context of gaming or other types of screen media use. So when I speak of context, I'm talking 

about when children can play different games. And with mobile games, children can play games 

at any time at any place. And indeed, a quarter of teens indoors playing Fortnite in class. 

But besides nationwide studies on screen and media use among children, there really has been 

limited research conducted on preteens and younger children, and their gaming experiences. And 

so there's definitely a gap in the research that my team and I hope to address. What I can tell you 

a little bit more about, are parent child interactions around gaming. 

Now unfortunately, approximately 3/4 of parents and children have never played Fortnite with 

each other. Although they may played on their own, there isn't interaction around different 

mobile games, for example, one being Fortnite. In my research, we conduct a naturalistic study 

looking at how do parents and children interact around different types of screen media. So 

children wore audio recording devices, and we transcribed and described parent child 

communication interactions around different types of screen media. 

In this study, we found that there's very limited interaction between parents and children around 

media and mobile devices. So we've heard earlier today that parents have a lot of interaction and 
power to kind of control some of these concerns related to games, but right now things are 

getting in the way. There are barriers to parents and children interacting around gaming. 

And this is really problematic, because recent research supports setting limits around gaming. 

And that parent child communication about gaming can be really important for older children 

and adolescents. So we definitely want more of this, but for some reason I think one of the big 

things, being the rise of mobile games, makes it very challenging for that to happen. And indeed, 

parents beliefs about games associate with parenting around gaming. So that definitely is an area 

that we want to pursue more and really research into, what can we do to help parents engage with 

their children more around gaming. 

So in terms of unique concerns, at my clinic at Central Michigan University, we really focus on 

helping clinicians, teachers, other individuals important in children's lives, communicate and 

make decisions around media use, screen time. And one thing that we hear time and again, is that 

gaming is embedded in social interactions among children. So sometimes this can be really good. 

You connect with your friends and peers on games. 

And other times it can be conflictual. And we're seeing it kind of trickle into the school-- the 

schools zone. Children are developmentally vulnerable to gaming risks, and we heard about this 

today. And then finally, one thing that I've encountered is parent child conflict around gaming 

and problematic gaming. 

I wanted to find for you what problematic gaming is, and tell you what the current status is on it 

being a diagnosable condition. Here in the States, we have DSM-5. In section three, which is 

conditions for future study, internet gaming disorder is listed. These symptoms include 

preoccupation, withdrawal, unsuccessful attempts to cut back, loss of interest in other activities, 

continued excessive use despite psychosocial problems, among others. 

In order to get-- let me put it back for people so they can write it down. In order to be considered 

to have a problem with gaming or disordered gaming, there has to be dysfunction in someone's 

life. So it's not just enjoying playing video games, or being really enthusiastic, or wanting to be a 

professional gamer. It interferes with the child or adolescents functioning. 

Recently, the World Health Organization has made gaming disorder an actual diagnosis with the 

ICD-11 code up there. Symptoms include impaired control over gaming, increased priority given 

to gaming to the extent that gaming takes precedence over other life interests and daily activities, 

and then continuation or escalation of gaming despite the occurrence of negative consequences. 

So it's very important to point out here that it's not about the number of hours that you game. It's 

not about frequency or passion. It's about actual dysfunction. 

This must be severe enough to lead to significant impairment in important areas of functioning. 

So, for example, not sleeping, not going to work, not going to school, losing relationships. So it's 

really important to clarify what's a clinical issue with gaming versus enjoyment. 
Now back several years ago when I was doing my clinical postdoc, I'm a clinical psychologist 

and work with children and families, there weren't measures out there to capture concerns related 

to media use. And so that led to the development of the Problematic Media Use Measure using 

DSM-5 criteria to identify children at risk for problematic media use including gaming. And 

what's really important to point out here, is that this measure predicts psychosocial functioning 

over and above the amount of screen media use. 

And so again, I'm trying to shift this conversation away from just tell me how many hours is 

allowed, to what are some symptoms or engagement with different types of screen media that 

would indicate that there's a problem that should be addressed. And so that's a very different 

approach to understanding when does it become problematic. We don't necessarily care as much 

about the number of hours, although that will correspond or correlate with problematic gaming. 

So this measure worked equally well for boys and girls, and it's been tested in children ages 4 to 

13 years of age. It's a parent report. We just create a self report version of this. And really, this 

came out of the need from clinicians to have a screener so they can address concerns related to 

media use or gaming in well child visits or in other clinical arenas. 

I want to spend the rest of my time talking about some of the work that I pursue at the Center For 

Children, Families, and Communities at CMU. And this really stems from a need from providers, 

school personnel, and parents in communities around the country regarding, how do I manage 

parents around screen media use? How do we handle or make systemic change when screens are 

in schools and so forth? 

So at this center, we seek to address growing media related concerns, including problematic 

gaming, cyber victimization, media parenting skills. But importantly, a large part of what we do, 

is we provide training to providers to help them screen for problematic media use, and give them 

tools for managing conflict in the home. Again, with mobile gaming and mobile devices, they 

can go anywhere. And so when it comes to school policy, there is not one consistent school 

policy across all schools in a state, for example. 

And so a lot of times the schools are seeking guidance around should we set limits on access to 

mobile devices during the school day. With mobile games it may be embedded into interactions 

during the school day, and so if there are conflicts related to performance on a mobile game, that 

may trickle over into the real life, in real world. And then additionally, what we also focus on, is

developing interventions that treat screen media related concerns. So helping parents manage 

screen time using harm reduction approaches. 

So if you'd like more information, I have my contact information up here. I wanted to leave 

enough time for discussion, but then also, there's a lot of research that I wasn't able to talk about 

today related to other types of screen media and concerns, and I'd be happy to share that with you 

if you're interested. 

[APPLAUSE] 
PATRICK: Thank you, Sarah. OK, now we have time for a Q&A. We'll have about 15 minutes 

for this. So I'll start off with the first question, is-- this will be posed to as many people as I feel 

comfortable answering it. How can you determine if a person is not just a risk taker, meaning 

they are more likely to buy loot boxes, excessively gamble, and/or use drugs? Is this getting at 

some correlation with risk aversion and just give them a definitional issue of can you separate 

out risk aversion from boxes? 

ANDREY SIMONOV: I mean, I guess that in the framework which I talked about, it's really all 

part of his alpha l is a persistent preference for loot box. And I guess one way to separate it out 

is, just ideally what you want to find is some shifters which will affect your-- will not affect the 

risk preference characteristic of risk of the loot box, but will affect how much people are exposed 

to the loot boxes, in which environments they use them? It required a lot of data and the right 

variation in the data, I guess. 

And then it's often-- I think in-game, if you have-- if you can separate it out, was it in model, 

economic model list of things. That's great. It's harder to have extra data on how consumers--

what happens to them later in their lives, because it's really hard to attribute what happens to 

people later on to a particular thing which happened to them in this game. 

PATRICK: This is a question from Twitter for doctors Zendle and Elmachtoub. Did you 

consider aesthetic design in your research? 

DAVID ZENDLE: So to some extent, yes. So what are the interest-- there are lots of-- we've got 

a paper that's just sort of coming out in a journal called Computer and Human Behavior where 

we looked at different features of loot boxes, and whether any of those loot boxes were 

particularly strongly linked to problem gambling. When it comes to aesthetics, the one thing we 

did look at was near miss effects. So near misses are a common thing you get in gambling 

devices, and loot boxes mirror gambling devices in lots of important ways. 

So there's the fact where, say you're playing fruit machine or something, and you almost get a 

win. You're just one fruit out. You may be more likely to play again, because you've got that near 

miss, or perhaps in the gamblers mind, a near win. Many loot boxes have a similar mechanism. 

It's not clear if they're imitating slot machines directly, or whether it's some sort of convergent 

evolution, where you've got a sort of rotating disc of options go round. And then you might just 

miss out on something really good that you like, then you get something less good. So we look to 

see if those types of loot boxes were both strongly linked to problem gambling. 

We found tiny effects. Like little tiny things. Nothing that we consider important or trustworthy. 

Generally, it seems that regardless of these features, that link to problem gambling exists. 

ADAM ELMACHTOUB: Well, I guess it's good that I let you answer for us, because we 

ignored that effect, and now you justified it for us. 

[LAUGHTER] 
DAVID ZENDLE: Yes. 

PATRICK: Another question from the audience. Does the literature support the idea that loot 

boxes are different from baseball cards or Kinder Eggs, or has that simply not been studied? 

DAVID ZENDLE: Oh, that's actually a good question. So I was wondering if I would get this, 

because-- so we've got this link between problem gambling in loot box banding. But you might 

engage in an argument where you say, ah-ha, but perhaps buying Kinder eggs is also linked to 

problem gambling. Now logically, that doesn't sort of fly as well as loot boxes does, because loot 

boxes look so much more like gambling, and there's this sort of distribution of value in them 

which you just don't find in a Kinder Egg. 

There are many formal distinctions. We thought, just ironclad things. Well, we'll go and run that 

study. 

So we went and we asked about 900 people about collectible card game spending to see if that 

was linked to problem gambling? And it just wasn't. There's something special about loot boxes. 

We haven't published that study yet, but if anybody would like to have access to the data from it, 

I'm very willing to share any of the data from this. And since it's a question that people are 

asking, I'll make it a priority to get it published. I do think we're pre-printing, where as soon as I 

finish the manuscript, I make it publicly available so people don't have to wait for it to go 

through the general process. So if this is something people care about, they can have that data 

within a week if they like and the paper. 

Yeah, one thing though it's important to recognize, is there's no friction costs for buying loot 

boxes. There's a huge friction cost for buying a physical item. And that's why we don't have that 

cost in our model. 

So when you buy something-- even if you buy it from Amazon, you still have to wait to receive 

it. And by that point, you're thrill may have disappeared a little bit. 

DAVID ZENDLE: Yeah, that's a really neat point. I remember when we were talking to 

Australian Senate about this, they sort of said, what are the differences between loot boxes and 

trading card games in the real world. We said, well, there are loads, and we don't really know 

which and the important ones. But certainly one of the things that seems important, is the 

velocity and the volume with which you can make loot box purchases. I mean, you can't go to a 

shop and just buy Kinder Egg, Kinder Egg, Kinder Egg, Kinder Egg, Kinder Egg, Kinder Egg, 

Kinder Egg, Kinder Egg, but that's what we see people do with box boxes. 

PATRICK: This question's from the audience and potentially to everybody. Are there common 

graphical or audio elements presented during the opening of a loot box, such as a flashing screen, 

or lights, or louder music, that increases the potential for spending on loot boxes? I stumped 

them. 

[CHUCKLING] 
DAVID ZENDLE: I'm not aware of any research, which shows that that's the case or not. 

PATRICK: This question's for Andrey Simonov. What control variables were in your study? Is 

there something to control for the popularity of the game, or the rarity of the items, or any unique 

traditional style boxes? 

ANDREY SIMONOV: Yes. So all the analysis done within the game, so there is no need for 

control for the game. There is-- so for the descriptive items I showed, there was controls for 

stages, for player fixed effects, what kind of items people had. So at this stage, what we-- this is 

all against [INAUDIBLE] relational. We have the right variance, we just didn't clean enough 

results to be ready to share the results of causal estimates, but well basically, does correlations 

hold whatever the fixed effects will include basically? 

PATRICK: Do you see any difference between purely cosmetic loot boxes, or some of the more 

pay-to-win loot boxes for any of the research you presented for implications for addiction, or for 

the pricing, or for usage? That's open to everyone. 

ANDREY SIMONOV: And I mean, for what we presented, one, there's two things to keep in 

mind here. One is, we kind of-- in this particular game, there is no value for having cosmetic 

value of items. There's little social interaction, so we can't focus on this functionality as 

probability of winning. About pay-to-win, so this is also a pay-to-win game in a sense of you 

need to get those items to progress. 

But important fact is, you compete with a machine. You don't really compete with other players. 

And I think that's the role of play to win items it's extremely different if I am paying to get an 

items that other guy can get as well, and now it's like a prisoner's dilemma, because I want to 

play the game well, but also this guy has a game item. 

And I think in any games was interaction with between players, even items without functional 

value, it can get very problematic because gaming companies have a desire to do it. In our case, 

we don't have this problem, because you really compete with the machine. But I would keep this 

distinction in mind for different games. 

DAVID ZENDLE: I think you can see also, different motivations for purchasing-- it might seem 

obvious-- items from the boxes that you give you some sort of advantage in meeting the ludic 

challenges of the games and ones that give you some cosmetic value. When we look to our data 

we asked people why, these adolescents, why are you buying these loot boxes, lots of people 

said, I just want to fit in with my friends. And so I think one thing that we've all got to be aware 

of, is that games aren't coin operated arcade machines anymore. 

They're vibrant social worlds, and just because your motivation for purchasing something isn't to 

do with literally winning the game, doesn't mean that it's valueless. There's a lot of value in 

looking a certain way in many of these games. And lots of times, people play these games not to 

win them, but to hang out with people. 
PATRICK: So there'd been some mention early today of video game developers potentially 

using dynamic odds for loot boxes. Does that have any implication for addiction and variable 

reinforcements, or for the optimal pricing of that, or usage? 

So I think that with regard to dynamic odds, I think that would be a nightmare to regulate. 

Because as the odds are changing, you can never-- with just a couple samples see if you're truly 

adhering to such odds. So that's something that I think would really be something to worry about 

in terms of-- just in terms of making sure that people are sticking to these odds, even if they are 

dynamic. And the unique thing-- another unique thing about loot boxes versus baseball cards, is 

that companies can see your inventory. That's a fundamental difference. 

So being able to take advantage of that would obviously be beneficial for the seller and allow 

them to exploit more. But also be bad for consumers, because they would be very, very difficult 

for them to understand their optimal purchasing strategies in the long run of the game. It would 

be very hard to anticipate how much money they will need to succeed in the game if everything 

is updating dynamically. 

DAVID ZENDLE: I'd like to agree with Adam's comments there. They're well taken. I'd also 

like to point out that this isn't a theoretical future. These are things that being patented by 

companies as we speak. 

There's a new paper that's out in this journal Computers And Human Behavior, which is one of 

the best journals for this stuff, by a research called Dan King. You can find it online if you like. 

And he approaches this issue at an interesting way, in that he just does a Google patent search. 

And the things that people are patenting are unusual and might surprise you, or they might worry 

you. 

PATRICK: And you, Sarah? 

SARAH DOMOFF: What was that? 

PATRICK: Did you have any comments? 

SARAH DOMOFF: I just think it's-- I'm going to bring a point that may not be entirely related to 

this, but it's really challenging for parents to navigate all of the details of a variety of games that 

kids are playing. And it takes a lot of effort, and time, and I find myself, and other clinicians, and 

other individuals who work with children, have to spend a lot of time to figure out what are these 

protections that we should tell parents about, because it's just not clearly labeled. And regardless 

of whether regulations are coming forth, I think we definitely need better documentation about 

what parents should consider, whether from within the industry or from consumer groups, such 

as Commonsense Media, because it's just really complicated, and there are just so many games 

for parents to keep up with it. It's a real challenge. 

DAVID ZENDLE: I'd like to follow up on what was just said by saying that, in fact, many of the 

games for many of the companies for which we see these sort of patterns for these new types of 

loot boxes being registered are companies that make mobile games for children. So I know 
Kabam was mentioned again, and again, and again, during this paper. That's the company that 

makes Marvel's Contest of Champions. That's a game where you can play with your favorite 

Marvel superheroes against each other. 

I think the industry needs to take a really long look at itself and see what is it doing. I hear these 

internal discussions by the industry say, of course, we behaving totally ethically. But from the 

outside, it really doesn't look like that, and it really doesn't look like that to researchers, and it 

doesn't look like that to policy makers, and it doesn't look like that's regulators. And in my home 

country, the UK, we're talking very seriously about should you be able to self-regulate? Have 

you demonstrated that responsibility, and lots of people think that you haven't. 

PATRICK: OK, that concludes the second panel. Thank you all very much. I think we'll take a 

10 minute break. 

Yeah, we'll shoot for a 10 minute break. And it's 2:35 now, so let's aim to be back here at 2:45, 

please. Thank you all. 

[APPLAUSE] 

[MUSIC PLAYING] 

[SIDE CONVERSATION] 

MARY JOHNSON: Good afternoon. I am still Mary Johnson, and this is William Ducklow. And 

together, we're going to be moderating today's final panel on self-regulatory initiatives and 

consumer education. 

WILLIAM DUCKLOW: So joining Mary and I on the final panel today are the following, Pat 

Vance, president of the Entertainment Software Rating Board, Keith Whyte, the executive 

director of the National Council On Problem Gambling, Anna Laitin, director of financial policy 

with Consumer Reports, and finally, Ariel Fox Johnson, senior counsel for policy and privacy 

with Common Sense Media. Please feel free to refer to the speaker bios that are available outside 

for more background information. 

MARY JOHNSON: So thank all of you for being here. We're in the homestretch here of today. 

And as with the prior panels, you'll each have 15 minutes at the podium. And thank you to 

everyone for staying on time. This has been moving very smoothly, and we really appreciate 

that. 

So after the formal presentations, then we'll move to a moderated discussion. Please feel free to 

take the podium, Pat. Thank you so much. 

PATRICIA VANCE: Well, thank you very much, Mary and Will. It's great to be here. We're 

going to-- I'm going to start my presentation just talking a little bit about who the ESRB is. 
We are celebrating our 25th anniversary this year. We were established by the industry as a selfregulatory body. You're probably most familiar with our rating system. We assign ratings to 

video games and mobile apps to ensure that consumers, but especially parents, have the 

information they need to make an informed purchase decision. 

We also enforce a very robust set of marketing guidelines that the industry has adopted that 

relate to how ratings are displayed across boxes, in ads, as well as how product can be marketed, 

particularly mature rated product. And then, last but definitely not least, we have our ESRB 

Privacy Certified Program, which is a privacy seal certification that is one of the first COPPA 

safe harbors sanctioned by the FTC. So those are our key activities. 

When it comes to our ratings, we have a three part rating system. We have age rating categories, 

that suggest age appropriateness. We have content descriptors. We use approximately 30 

different content descriptors that indicate why a particular age rating was assigned to that 

particular game or app. And then our newest and third component of the rating, is what we call 

interactive elements, I'll talk a little bit more about them as we get into the presentation. 

ESRB ratings are available for games and apps across a variety of different devices and 

platforms. They are available for all boxed games sold in the United States. They're available on 

many digitally delivered games, mobile apps, virtual reality, augmented reality, and mixed 

reality games and apps as well. So we've had universal adoption among major retailers, as well 

as the major game platforms from virtually the beginning of the ESRB rating system. 

In part thanks to being around for 25 years, 87% of parents of kids who play video games, say 

that they're aware of the ESRB rating system. And of that 87%, 77% say that they regularly use 

the ratings. In other words, that they're checking the most, if not all the time. 

Now, despite the fact that interactive elements is the newest part of our system, 70% of parents 

say that they're aware of the interactive elements. And of those parents, 79% say they regularly 

check them. Now, all of the interactive elements are important, but this is a ranking, and in the 

context of the conversation today, I thought it was important that although in-game purchases is 

an important component to parents, 75% of parents say they're that the in-game purchase notice 

is either extremely or very important in helping them decide which games are appropriate for the 

kids to play. 

If you look at this chart, our shares location interactive element, 82% of parents indicated that 

that was extremely or very important to them, followed by users interact, and followed by 

unrestricted internet access. So all are important. But this is a general ranking. So it's important 

in the context of the conversation today to appreciate that there are many different aspects of 

disclosures that are important to parents. 

Now, parents consult many different sources when they're trying to make a decision about what's 

appropriate for their children and families, Not Just the ESRB rating information, but they're 

playing the game themselves. And I think we're finding that increasingly to be the case as new 

generations of parents have kids, and they grew up with playing games. Parents are also 
checking out the genre as a good indicator of whether or not a particular game is appropriate for 

their children. 

They're conducting internet searches. They're looking at the descriptions on the game boxes and 

on the detail page when they download a game. And they're also consulting user reviews. 

Now, the Family Online Safety institute released a study last year that suggests that parents are 

actively engaged in having conversations with their kids about online safety and the use of 

technology in the home. 91% of parents set household rules. That's a really important statistic for 

us to understand, because it's not just about one solution or one tool. It's about parents being 

actively engaged. 

The ESRB provides a family discussion guide on our website to actually start that conversation. 

64% of parents indicate that they frequently discuss online safety with their children. And our 

own research suggests that 7 out of 10 parents have actually prevented their child from playing a 

game because of one of the interactive elements that were assigned to the game. 

Now, ESRB ratings are integrated with the parental controls that are available across different 

game devices. So here you'll see the Xbox, Playstation, and Nintendo parental controls, which 

allow you to block games by their ESRB rating. And you can do the same thing in the Google 

Play Store. 

Now, we put together this little video to give you a brief glimpse of how parental controls enable 

parents to manage the gameplay in their home. 

SPEAKER: Playing video games as a family is a great way to spend time together. And it's never 

been easier for parents to manage what, how, when, and with whom their children play. Even 

when they're not around. 

Parental controls are available for all current game consoles, handhelds, PCs, smartphones, and 

tablets. Each device has settings that can limit and manage the experiences that your children 

have, and they take just a few minutes to set up. On some devices, you can remotely set controls 

from your mobile phone or your computer. So whenever you want to add a restriction or change 

a setting, you can be assured that your children enjoy playing games within the parameters that 

you approve. 

There are four important ways to control how your kids play video games. You can control ingame purchases or block them altogether. And the account holder will always be notified 

whenever a purchase has been made. Parents can also limit play and screen time. Some devices 

allow you to set specific time limits for every day of the week. You can also block games based 

on their age rating, and you can restrict online communication. 

Some devices allow you to approve with whom your child plays online or block other players, 

even by specific game. Above all, remember to keep on having fun playing video games with 

your kids, and talk to your kids about the games they like to play. There is no better way to make 

sure your child has the best experience possible playing video games then staying involved. Visit 
parentaltools.org to access parental control guides and a family discussion guide to help start the 

conversation with your kids. 

PATRICIA VANCE: So there are obviously a lot of functionality in parental controls, but are 

parents using them? And based on our latest research, 72% of parents have indicated that they 

have activated or enabled parental controls on their computer at home, their mobile device, or 

their game console. And not surprisingly, the propensity for them to do that is higher for the for 

parents with kids of younger ages than older ages. 

So again, all of the functions and parental controls are important to parents. But if you look at 

what they're actually doing with parental controls, our research says that the number one function 

they're actually enabling is the manage in-game spending function. So 2/3 of parents indicated--

2/3 of parents who were using parental controls indicated that they had activated the manage the 

in-game spending limits. 

64% indicated that they were restricting access to social media using parental controls. 61% had 

indicated that they were-- they had blocked games based on ESRB ratings. And 52% indicated 

that they had set time restrictions, followed by 50% indicating that they had managed online 

communications. 

So parents are familiar with parental controls. They're using them. And they're using them in a 

variety of different ways. 

Last year, we began looking at loot boxes specifically. We conducted research among parents, 

and we discovered that a large majority of parents don't know what a loot box is. In fact, only 

32% of parents indicated that they knew what a loot box was, but when we presented several 

different options for them in terms of a definition for loot boxes, they were able to select the 

correct definition only less than a 1/3 of the time. 

Once we told them what a loot box was, by far the biggest concern that they expressed they 

would have would be the ability for their children to spend money much more so than the 

randomized nature of loot boxes or the impact that they may have on the amount of time that 

their child plays games. We repeated this research earlier this year and found very similar results. 

Although there is slightly higher awareness in use, awareness and understanding of what loot 

boxes are among parents, the actual concern they expressed about spending was even higher than 

the first time we surveyed parents. 

So this is important to understand in terms of how we're presenting disclosures. Our rating 

systems primary target audience are parents. Parents need to understand what it is that we're 

providing, and we need to provide it in a way that they understand, and that's concise, and that 

they can digest at a fairly quick glance. 

So we started assigning in-game purchases to physical video games back in April 2018. We had 

begun-- we had already begun doing something similar in mobile and digital games, but it wasn't 

until early last year where we began actually assigning the in-game purchase descriptor to 
physical video games. And today, 18% of all rating assignments for physical video games 

include that notice. 

Now that notice spans not just loot boxes. It spans all types of in-game spending. As our research 

indicated, parents are concerned about in-game spending of all kinds, not just loot boxes. 

And so the when you see an in-game purchase notice on a game, it indicates that there are--

there's the ability to make a purchase using cash, whether you're buying virtual currency, or 

whether you're buying a subscription, or a season pass, or a loot box, or some other in-game 

transaction. 

But disclosures aren't enough. We want to make sure that parents know that when they see that 

in-game purchase notice, if they want it limit-- if they want to limit their child's ability to spend 

money, they know how to do it. So we launched parentaltools.org last year, which gives very 

easy access to parents to instructions on how to set up parental controls depending on which 

device they have in the home. And we created an animated video that's very simple way to sort 

of describe what parental controls can do. And to date, almost a half a million views have been 

generated for the video, almost 100,000 page views, and we've been writing articles, and blogs, 

and making an effort to make sure that parents understand what in-game purchases enable, as 

well as the parental controls that are available on different devices. 

Now we've just recently refreshed parentaltools.org and added an additional functionality to the 

website, which allows parents to first check, well, what do I want to do with parental controls, 

and then get specifically to that place in the instructions for that device that they have in their 

home so that they know how to control spending, control time, control by age rating, or restrict 

communication. We also have just partnered with GameStop and are talking to other retailers, so 

that this holiday season whenever a parent purchase, or any consumer for that matter, purchases 

a new console, that it comes with an insert that reminds them to set parental controls and directs 

their attention to parentaltools.org so that they know how to do that. And we're going to be 

complementing that program with an online ad banner campaign targeting parents. 

We also just released a new blog on our website, esrb.org, that is entitled What Parents Need To 

Know About Loot Boxes And Other In-game Purchases which really tries to break down what's 

a very complicated concept. And as you heard earlier today, loot boxes come in all different 

forms in all different contexts. And so we tried to really simplify it for parents, and also make 

sure that they understand what other types of in-game purchases can be-- are available. And we'll 

continue to create new articles and new blogs that help parents navigate games. 

So in summary, I just want to tell you that you can be confident that any game that gets 

published, regardless of the device, will have some descriptor that will indicate that there are ingame purchases, that parents have very low awareness of what a loot box is, but that their main 

concern is spending, and we are addressing that by not just having disclosures available for 

games, but also making sure that they have the tools and parental controls to manage the money 

and time that their child spend playing video games. We'll continue to support parents like we 

have for the last 25 years with disclosures, with making-- with enhancing the system whenever 
it's warranted, having educational materials available to parents, and addressing their concerns as 

we go. So thank you very much. 

[APPLAUSE] 

WILLIAM DUCKLOW: Thank you, Pat. Next we have Anna Laitin. I can't believe no one's 

fallen. 

ANNA LAITIN: Hi. Good afternoon, everybody. My name's Anna Laitin. I'm the director of 

financial policy at Consumer Reports. 

First, a little bit about Consumer Reports and why I'm here. We're an 80-year-old independent, 

nonprofit member organization. We work side by side with consumers for truth, transparency, 

and fairness in the marketplace. Most of you know us for rating cars and mattresses, but we also 

do work on a wide range of issues. 

We approached this workshop-- we don't have-- we haven't historically done a lot of work on 

video games, so we approached this workshop from a perspective of looking at larger 

marketplace trends first. And a couple of things we've been spending a lot of time on lately, drip 

pricing, hidden fees, the obfuscation of the true cost of a product or service. This is something 

the Federal Trade Commission has spent a lot of time on from a 2012 workshop on drip pricing, 

to the workshop earlier this summer on online event ticketing. 

It's very hard for consumers to know what they're getting, what it's going to cost. Shopping has 

become more complicated and more confusing. And then the manipulation of consumer 

psychology, monetizing user experiences, and dark patterns on websites that get people to do 

things that maybe wasn't exactly what they intended to do when they started. 

So how does that apply to the gaming marketplace? Well obviously, the increased monetization 

of play. That's what we're talking about here. 

You've got the downloadable content, those one time purchases, and then the microtransactions, 

the loot boxes we're all talking about today, repeat purchases, consumables, often very quick 

purchases made often in quick succession, and manipulative user experiences. So subtle tactics 

that influence consumer behavior, and nudge them to purchase these loot boxes. What are loot 

boxes? I put together these slides not knowing exactly what was to be talked about in the 

morning, so there's a little bit of repetition here. I'll move pretty quickly. 

In the FTC announcement, the description is here. We see it. The key things for us, the rewards 

are seemingly random, paid for with real money or in-game currency, sometimes impact 

gameplay, and the contents are generally not transferable. 

So in terms of transparency, this is a screenshot from Counterstrike Global Offensive. 

Consumers are unaware of what they're actually purchasing. The odds of winning a specific item 

are not disclosed. 
This particular loot box, there's a lot of options, including one full, surprise, rare, special item. 

In-game currency and pricing can really hide the true cost. This one from Fire Emblem Heroes, 

it's very hard to see on these slides, but it costs five orbs to summon a character, four each to 

summon the next three, and three orbs to summon the last. So 20 orbs spent on loot boxes. 

But how much is an orb? That price isn't linear. So to figure out how much it costs to summon 20 

characters, is not a simple matter to figure out. This is very much finding ways to hide the fact 

you're spending real money on these characters. 

In the ratings, we applaud ESRB for the work they've done, but as the previous presentation 

showed, there's a label for in-game purchases and that can mean a huge range of things. That's 

everything from, you can buy a new character when it's released, to we have surprise loot boxes, 

and a whole wide range. And I know when I look at a game, there's a lot more detail that 

consumers need to really understand how they might be presented with the option to spend 

money. 

And similarly, labeling for mobile games provides limited information. This one says in-app 

purchases. And then in information, you actually can get some good information about how 

much things cost. But again, you've got that same obfuscation. 

This one for Clash Royale, you can get a-- can't even read it on my own piece of paper, a pouch 

of gems for $4.99 or a wagon of gems for $49.99. Is a pouch 10 times smaller than a wagon? 

What are you actually buying? And it's again, taking the money away from the actual what 

you're buying. 

So loot boxes, just in conclusion, are not transparent. Consumers aren't sure of what they're 

getting. Odds of winning items, although the announcement this morning may move to change 

that, that cost is hidden, and those loot marks mechanics are insufficiently labeled. And another 

thing I'll mention that came up a lot today, is how quickly the decision to buy a loot box can be 

made. This is a speedy process, very different from going back to the store and buying another 

set of baseball cards if you didn't get what you wanted. 

Then on this issue of dark patterns. This is something that Consumer Reports has looked at quite 

a bit. And it's interesting. Its tactics to nudge consumer starts taking actions. Grinding, making 

the alternative to buying a loot box, doing a lot of relatively pointless work for a very, very, very 

long time, making it extremely costly on a personal level to not spend that money. 

Appointment dynamics, dynamics that build the habit of playing. Using loss aversion and getting 

people to keep going, keep going, keep going. And get bonuses for playing every day. And payto-win, as was discussed quite a lot this morning. Playable without microtransactions, but if 

you're not willing to spend money, you're not going to do as well as your opponents, or your 

friends, or you're not going to help out your group. 

This problem is both deep and broad. There have been a lot of press reports about gamers 

spending far more than they intend on loot boxes. People spending thousands of dollars. We've 
all heard about the parents whose kids racked up huge charges before they figured out what's 

going on. 

These are actually two separate stories of people who discovered they'd spent more than $10,000 

on microtransactions. I raised this story with my son who plays FIFA 19 the hard way without 

any allowance to spend any money, and I showed him the cost of the loot boxes and his eyes 

bugged out. It is-- people can spend obviously a lot a lot of money. 

And then a growing population of game players exposed to loot boxes and manipulative content. 

So I think this was discussed earlier today, we're not just talking about young gamers who play 

all the time. We are now a society of gamers, whether you're playing on your mobile phone 

while commuting, whether you're sitting in your house playing games for hours, it's a lot of 

people. 

65% of American adults now play video games, and 75% of households have at least one gamer. 

So this is not an isolated problem set to those young, sort of stereotypical gamers. And then this 

is intentional in these games. 

There's a column written by the CEO of Tribeflame called Let's Go Whaling, A Guide To 

Monetization Through In-app Purchases. The whole idea of this column is about getting people 

to spend as much money as they can, and make it so that you're accustomed to it, you're-- this 

last line to me, you're just a tap away from spending. This is how the games are constructed. This 

is where the money is made, and consumers aren't necessarily aware of that. And I'm relatively 

short, because so much of what I talked about came up earlier. Thank you. 

[APPLAUSE] 

MARY JOHNSON: Thank you, Anna. And now we'll hear from Keith Whyte. 

KEITH WHYTE: Hi, everyone. And thanks to FTC, and my fellow panellists, and everyone 

today for this really informative discussion. I'm Keith Whyte, the executive director of the 

National Council On Problem Gambling. I've been working on gambling addiction issues for 25 

years now. 

And as a brief note, I'd like to wish my son, Ian, a happy 14th birthday to day. When he's not 

playing drums, guitar, bass, or piano, he's often gaming. And he wanted me to tell you, that loot 

boxes in a Fortnite are occasionally annoying, mainly because he doesn't get the gun he wants. 

Nothing to do with gambling. 

It's so just a little bit about us and why we're here, because we do have a unique perspective on 

this, I think. We're the national advocates for programs and services for problem gamblers and 

their families. We were founded in 1972 and are neutral on legalized gambling. And that is very 

important, because it allows us to work in partnership with government, gaming industry, 

counselors, regulators, researchers, and recovering gamblers. And we're happy to work with 

groups like ESA and their member companies as well if they're interested. 
Many of the world's largest casino and slot machine companies are members of the National 

Council, and again, we're not anti-gambling, nor are we anti-loot boxes. However, we're here to 

share our experience because many features of loot boxes are similar to those of slot machines, 

and we've got about five decades of experience working on consumer protection issues in the 

gambling space. Both our experience and the evidence, show that some features of loot boxes are 

absolutely associated gambling problems among players. My presentation was just going to be 

everything that David said, but I think I have to do a little bit more than that. 

But yes, many of the panellists have discussed the issues we've been looking at as well. And 

indeed, a number of countries do regulate loot boxes as gambling, or certain types of loot boxes 

as gambling. But it's clear that whether or not loot boxes meet criteria for a gambling device in a 

particular jurisdiction, and whether or not parents recognize or understand the risks, additional 

consumer protection issues-- protection features must be put in place to protect vulnerable 

players from developing gambling problems. Loot boxes and slots can powerfully influence 

player behavior in ways that lead to entertainment for most, great excitement for some, and 

excessive play, and even addiction for a few. Players with gambling problems likely provide a 

disproportionate percentage of the, quite frankly, massive profits from slot machines and from 

loot boxes. 

To my knowledge-- oops, I think I'm going the wrong way. To my knowledge, every study 

published to date on the connection between boxes and gambling has found an association. 

You've heard from Dr. Zendle and others on that today. And in fact, given all everything we 

know about the similarities between boxes and slot machines, it would actually be astounding 

and surprising were there not such a connection. They are, in many ways, so closely related. 

We know that one of the reasons that, of course, as Dr. Zendle said, problem gambling is an 

issue, is because it can lead to massive and significant negative impacts. And I'd like to focus a 

little bit, as we've talked today, about the types of groups that we're most concerned with. 

Obviously, anyone who plays a slot machine, or anyone who pays to play loot box, may be at 

some risk, but we know that there are groups with higher risk. 

And those certainly include males, youth, and some groups not been talked about a lot today, 

veterans. We know veterans have much higher rates of gambling problems. And we believe, 

again, there is likely a bi-directional effect. People who are vulnerable for gambling addiction or 

who have gambling problems may be more likely to pay-and-play, and develop problems with 

loot boxes. And those who play loot boxes, may well be on their way to developing gambling 

problems due to their loot box play. 

These are very, very complex associations. Obviously, a lot more research needs to be done. And 

again, the industry can play an enormously helpful role in providing data to help all of us make 

more informed decisions about some of these risks. 

So based on our experience working with government and the gambling industry to protect 

players, we've got sort of four of solutions if you will, a number of which have already been 

discussed, so I won't spend a lot of time on them. And we have much more detailed information 
in or written submission, which is, of course, available on our website, and we have we have 

some copies here as well. 

So first, we look at, in the gambling industry, we look a lot about creating informed consumers. 

And we've talked a lot about-- today, about making and building transparency. And I think one 

of the challenges to this industry, and one of the ways that you can actually do much better than 

the gambling side, is if you're spending $250 million to develop a game, and you've got some of 

the world's best, most creative talent, let's find a way to make this information in disclosure 

entertaining, and interactive, and exciting. 

Build it into gameplay. Reward players for doing some pro-social behavior, like finding out what 

really the odds are in this game. I would hate to see it look like what a pay table looks like for a 

slot machine, which is you know 2.5, zillions of numbers in there, and without a degree in higher 

math, you're utterly unable to understand this. 

But there are ways to make this transparency quite effective, especially when you're trying to 

communicate with younger customers, or parents who are not technically well equipped. And 

obviously, we talk a lot about, in the gambling space, about consumer education protection. I 

think, last but not least, we would suggest a rating of most games with loot boxes is M for 

mature, because, ironically, if many of the parental controls are based on existing ESRB ratings, 

then most games with loot boxes, including some of the ones we're most concerned about, are 

rated as T for teen. And so if you're a parent who's based near parental controls on what the 

ESRB rating is, if the ESRB rating is as we would think artificially low, then that might not 

trigger the appropriate level of parental controls. 

We also, in the addiction prevention world, or in the gambling world, we know that some 

addiction in some people you can never prevent from developing a problem, right? We must 

make all the efforts we can to prevent, but just as we have learned from decades of experience 

with drug, and alcohol abuse, and other things, so while parental controls are important, we need 

to go beyond that. And one of the things that we do a lot in the gambling industry, is we 

recognize the role of parents, we recognize the role of industry self-verification, but we 

absolutely believe that there has to be third party objective regulation. 

Sometimes that could take the role of the-- sometimes that could be the role of the FTC. Other 

times it can be the role of third party groups, like ourselves or others, perhaps some of these 

panellists. And one other thing on this that's I think important when we talk about certification 

and verification, nobody in the gambling industry would ever trust a slot machine manufacturer 

to self-certify that their machines, the odds and randomness of their machines is as-- their 

machines perform as they say. 

So we use independent testing labs. That's what the state of Nevada and New Jersey, that's what 

everybody uses to verify that the odds are as they are stated. And they often find machines that 

don't perform adequately. It's an important consumer protection feature, and so if the industry is 

going to provide us information on odds and randomness, take a lesson from the gambling side, 

you got to get it done independently. It's not going to be effective if you're just telling us, oh, 
trust me, this game, these items drop at this rate, especially without any means to independently 

verify it. 

So we try and prevent as many problems as we can. Those who slip through the net are going to 

need help. And so one of the things that we will be launching very soon, is responsibleplay.org to 

help people who have questions, and perhaps problems, find a place to go. As Dr. Domoff talked 

about, there's a number of flavors, if you will, of addiction that are implicated in this discussion. 

There's straight up gambling addiction. There's, of course, gaming addiction, which she went 

through with the clinical criteria. And there's internet addiction. These are all separate, they're 

distinct, but quite closely related issues. And what we'd like to do with responsibleplay.org, is 

help people come take these various self-tests, and then find where they perhaps need to be. 

Some people who have problems with loot boxes are probably people have gambling addiction. 

Some people that have problems of loot boxes may well be gaming addicts. Some people may be 

internet addicts. Some people may have other problems, and so we want to be sort of a gateway, 

an information referral resource, where folks can come and then get steered to the appropriate 

help for their condition or issue. 

And again, another tip from the gambling side is self-exclusion. So one of the most effective 

ways to help someone who may have a problem with their gambling, or with their gaming use, is 

to allow them to self-exclude themselves. And in an environment where transactions are 

monitored, you can you can use self-exclusion through payment mechanisms, because while 

people may have many different accounts and play many different games across many different 

providers and platforms, they're probably using that one credit card, or at least a common bank 

account. And so payment level blocking can be very effective. It buttressing and adding to 

existing platform level controls and others. 

Self-exclusion also places a priority, or that places the emphasis on the gambler, or the gamer, 

and not necessarily the operator. But operators must have an affirmative duty to honor selfexclusion. So self-exclusion is not effective at all when you can walk right through it. So there's 

got to be, again, that partnership between people who exclude and companies that are going to 

participate in that program, because the worst thing you can do is set up an exclusion program 

and then not honor it. And that will bring the worst of both worlds. 

So last but not least, we talk a lot about evidence, and we believe it's incumbent upon the 

industry to help. If-- sorry, if by providing identified data to independent objective researchers to 

help all stakeholders validate concerns and develop solutions. If video game industry disputes 

our concerns, they should make publicly available the massive amounts of data they have on 

player participation and spend on loot boxes that they collect. We, as Dr. Zendle said, we'd be 

happy to be wrong. We don't think we are. 

Again, we've looked at looking through the gambling lens, and with 50 years of experience on 

this issue, we think there's clearly both cause for alarm and a link between people who pay-toplay loot boxes and people who developed gambling problems. But the only way we're going to 

really find out who exactly is at risk, and thus how we can create solutions, is to really dig into 
this information. So we don't want to identify people by name, so deidentification of data is 

critical. But we think there are ways to help provide that information to qualified third party 

researchers, which will help all of us figure out their true-- some of the true concerns. 

So in conclusion, with great profits come great responsibility, right? We call on the video game 

industry to dedicate a portion of loot box revenues to a public health trust fund that supports 

independent prevention, education, treatment, recovery, and research initiatives. ESA and its 

member companies can play a constructive and productive role, just as some casinos and 

lotteries embrace responsive gambling as the most ethical and economical way to address the 

harm their products cause. It's clear that paying for loot boxes is linked with gambling problems, 

and that some gamers are at higher risk for addiction due to their age, gender, or even military 

service. 

We know from decades of experience with slot machines and gambling companies, that 

educational awareness campaigns, a couple of strong responsible gambling or consumer 

protection policies and programs can help reduce, but never eliminate, the risk of problems. But 

for these measures to be effective, it will take true commitment of leadership from ESA, ESRB, 

and every developer and publisher worldwide, because if you have even one company that 

chooses not to participate, that opts out, doesn't comply with standards, the whole system, the 

foundation of the entire system is undermined. 

In the five decades in gambling, we've learned that self-regulation alone is never enough. It must 

have an enforceable consumer protection framework, and be companied by external oversight, 

research, monitoring, and verification by independent groups. So there's a three legged stool, 

there's a room for industry, for regulators, and for advocates. We look forward to working with 

anyone who has a sincere interest in preventing addiction and protecting players. Thank you very 

much. 

[APPLAUSE] 

WILLIAM DUCKLOW: Thank you, Keith. Ariel, you have the podium. 

ARIEL FOX JOHNSON: So good afternoon. I think I'm the last presentation, so thanks all, for 

still being here, and awake. And also, thank you to the FTC for hosting this workshop today. I'm 

Ariel Fox Johnson, senior counsel for policy and privacy at Common Sense Media. 

So as you've hopefully already heard this morning when my colleague Jeff Haynes was on the 

stage, Common Sense Media is committed to helping kids and families navigate an ever 

changing world of media and technology. And over the years, that's grown from helping parents 

pick out what TV shows might be appropriate for their children, to now helping them understand 

how to protect their privacy and their pocketbooks as kids are discovering new opportunities, and 

facing new risks online and in games. 

A lot has been said so far today about how children and youth are particularly vulnerable. And 

I'm pleased about that, that I won't be the only one. I want to just talk a little bit more about what 
kids and families know with respect to loot boxes and in-app purchases. And spoiler alert, they 

don't know a lot. So let's also talk about how we can improve the situation. 

So as you've heard today, this isn't just a kid's issue, but kids are particularly vulnerable. There 

are a number of different reasons why. First, kids can have trouble distinguishing play money 

versus real money, and games do not make it easy for them by, as discussed, not always listening 

things in real dollar terms. It's hard for kids and adults to figure out that things can cost real 

money. 

Second, even if information is listed providing real dollar amounts, digital transactions can make 

it difficult for people to understand that they're spending money. There is very little friction. It's 

hard to comprehend if you're making a purchase, if you're just clicking online or talking to a 

smart device, a lot harder than if you were handing over cash. Additionally, the use of 

microtransactions can compound this problem. To a kid $0.99 to $2.99 doesn't sound like a lot, 

and they don't think about the fact that they're going to make that purchase 70 times. 

When talking about teens, they're neuroscience and other issues to consider. Neuroscientists have 

looked at how teens brains are different. They're still developing, and this has been talked about 

a little bit earlier today. Their prefrontal cortex is not in the same shape as in adults, and they're 

more likely to do the immediate and risky thing to get a reward, and less likely to consider long 

term consequences. In this space, that means that they're more likely to spend money, and as 

we've heard, some of them think it feels good. This is just an example of even how in dollars are 

presented to a purchaser, it can be very hard to read how much they cost, and hard for a child to 

make a smart decision. 

In addition to cognitive and comprehension issues, there are also social emotional issues at play. 

And companies can take advantage of this. Older kids want to compete with their friends. And 

we've seen that some gaming companies are filing patents that would take advantage of this 

desire to compete, contemplating pitting a junior player with a more senior player in an effort to 

get the junior player to spend more money. 

Younger kids, they want to make people happy. They don't want to disappoint their friends, their 

family, and this also includes their favorite characters. And what I now hope is an infamous 

example, that you've all heard of, and if not you'll hear of it now, is Strawberry Shortcake 

insisting that children who are trying to play her game and make treats, purchase certain costly 

kitchen tool items. And if they don't she'll berate the player. 

There are other children's games in which a character will cry if the child doesn't make the 

purchases recommended. I think it bears noting here, that host selling like this is prohibited on 

TV. And here you have not only the host selling, but the host getting angry or upset with a child 

if they're not making a purchase immediately. 

Kids can also fall prey to the same type of selling techniques that adults can. So here, if they can 

read they may go for the best evaluate them. If they're pre-literate, they may just like the pink 

color. 
I think it's important to consider kids purchasing techniques in the broader context of their 

gaming experience. From our common sense research, we know that some kids are spending a 

good chunk of time playing video games each day. While a minority of teens and tweens, 

according to our research, are playing video console games on a given day, I think the numbers 

are much higher for mobile probably, only 27% of teens play console games. Those who do 

average over two hours. 

Among gamer teens and tweens, they're spending over two hours a day on video games. And 

those who consider themselves mobile gamers, spend almost as much time. As you've heard 

today, there are real and growing concerns about addiction on games and via other techniques, 

not just in-app purchases, but buzzing, and digging, and randomized notifications, snap streaks, 

autoplay, and other features that tech companies are using to keep us hooked, and Common 

Sense is focused on these broader issues as well. 

If you add that to the thrill and excitement of not knowing what you're going to get when you 

open up a loot box, and it's really no surprise that, as you heard, the American Psychological 

Association, the World Health Organization, have identified internet gaming disorder and 

hazardous gaming as public health issues that merit further attention. So we do hear a lot of 

concerns from parents at Common Sense about their kids spending too much time online and on 

games. 

And we also hear questions about in-app purchasing. We hear less, as you heard from the ESRB, 

about loot boxes. I think a large reason for this, is that parents are in the dark. They barely 

understand in-app purchases in general, let alone specific mechanisms like loot boxes. 

Why are they so in the dark? Well, one reason, again, disclosures can be ineffective, in small and 

tiny print, and you have to click further to see the costs of actual purchases and items. 

Additionally, parents might not realize that a kid has access to their credit card. They don't know 

that it's already linked to their online account because they used it for unrelated purchase. They 

don't know that their kid can access it in their purse, and they might not know to worry about that 

if they don't realize that the game has in-app purchases to begin with. 

While they may get statements, in some instances, such as in Facebook gaming, a number of 

parents weren't getting any statements at all. And when they do get statements, they'll have sort 

of vague descriptors, like Facebook or Amazon, in amounts of cents. It could be hard for a parent 

to tell that that's an in-app purchase or if it's maybe your monthly iCloud storage. 

When we talked to parents, they have very basic questions. How do we turn off in-app 

purchases? How do you find out if a game has them? How do you find out if a game requires 

them to play? They feel lost. 

And for kids-- for parents of kids who've already racked up huge in-app purchases, they feel 

angry. I think at this point it's sort of-- you've heard multiple times about people spending 

thousands of dollars in these games whether they liked it or not. And I have family members who 

was telling me this weekend about spending thousands of dollars on in-app purchases. 
One example here, is four kids, all under 10, spent 550 pounds trying to get their favorite 

football, or soccer player. The parents only realized when their bank card was declined. They 

were playing a game that was recommended as appropriate for ages 3 and up. 

When parents find out that their kids have made these purchases, they respond in sort of a variety 

of predictable ways. One, they try to get their money back, and a lot of these games have 

chargeback rates that would be considered fraudulent in other industries. Also, some of these 

parents turned to the courts. And indeed, in a lot of the examples I mentioned in the last slide, 

there are about parents who've sued. 

Almost all of the major platforms have faced FTC actions in settlement due to deceptive in-app 

purchases and disclosures. So Google and Apple settled with the FTC, and then Amazon, who 

first went to court. They're now required to disclose in-app purchases in games. Though, as 

noted, these purchase these disclosures can be woefully inadequate for parents. 

Common Sense and other advocates have also asked the FTC to hold Facebook to the same 

standard more recently. That issue has not been resolved, and we're concerned that the claim has 

been extinguished by the recent settlement. So current practices don't seem to be serving kids 

and families, and we've heard that there are some steps to improve those today, but we think 

there need to be more. 

Thus far in the US, most efforts have focused on transparency. App stores, self-regulatory groups 

are indicating in-app purchases. Now there's also Google and Apple, and today others who will 

be disclosing odds, drop rates on loot boxes. Common Sense in our reviews tries to give 

information to parents about in app purchases, as well as the prices of those purchases when we 

can. We also note when in-app purchases are so pervasive or manipulative that they might 

disrupt gameplay or a child's experience, and that's what we call commercialism in a game. 

We think companies can and should do more, however. One useful guide here is a UNICEF 

paper on child rights and online gaming that just came out this spring. It sets out key principles 

that all game companies could follow to serve children. Companies should help understand--

should help children understand the commercial aspects of games, and speak to them in a voice 

that they and their parents can understand. And I love the idea of companies and their developers 

being creative and making this something that players and parents want to read and spend time 

on. 

Companies should clearly label advertising and other commercial content. And they should make 

sure that children fully understand all purchases before they pay for them, and not later when 

their mom asks how they racked up huge charges. Companies should also be inclusive in their 

game design. They should make them so that all children can play them, and understand, and 

have the expectation that children maze the games in a way that they did not intend, and they 

may play games, even if the product was not designed for them. 

Common Sense would like to see companies eliminate features that manipulate kids into 

spending more money or time than they or their parents were intending. We also think that 
platforms should take more responsibility. They are the gateways, and they can do more to flag 

particularly problematic titles, as well as help ensure that parental controls are the default setting. 

As you've heard today, loot boxes are a global concern. They are a concern for many regulators, 

though there's not always consensus on what to do about this. Belgium and the Netherlands have 

said that some types of loot boxes constitute gambling, and game studios have responded by 

pulling out or modifying their games. 

The UK has found that some similar practices were not gambling. In China and South Korea, 

there are rules that require game companies to disclose the odds. This seems like a positive step 

forward, but some of the odd disclosures that we have seen include ranges of winning certain 

items that are so broad, like say 5% to 60%, that seems to barely constitute a disclosure at all. So 

as we move forward and talk about disclosing drop rates and odds of winning, I think it's 

important that we maybe look closely at what's being disclosed. 

In addition, European regulators, Washington State Gambling Commission have also signed 

joint statement expressing concern. In the US, obviously, legislators and regulators are also 

taking notice. Senator Haslam asked for the FTC to investigate loot boxes, and make efforts like 

today's workshop, to educate parents and the public about potential addiction and other negative 

impacts. And senator Holly, along with Senators Blumenthal and Markey, has introduced 

legislation to ban loot boxes and pay-to-win monetization practices for those under 18. 

There are also broader bipartisan efforts, like the Camera Research Bill, that would study the 

effects of technology and media, including video games on kids. And at the state level, we've 

seen a number of efforts, including one Hawaii legislator who's made repeated efforts to ban loot 

boxes for those under 18 and require odd disclosures, and recently gotten past a commission to 

study this aspect of the gaming industry in his state. 

This is still an emerging issue for many parents, and it's one which many are still unfortunately 

not aware of. But as it faces increased attention, we're hopeful that efforts from all sides will lead 

to a better experience for kids and families in the future. Thank you. 

[APPLAUSE] 

WILLIAM DUCKLOW: Thank you, Ariel. So we're going to jump into some Q&A now, and 

none of the following questions are directed at any particular panellists, so please feel free to 

jump in. The first question, that I think we're curious to hear the panellists response to is, this 

issue of disclosure of odds. 

This has come up throughout the entire day. In fact, we actually had announcement by ESA 

earlier this morning, that additional platforms are bringing this online, joining Google and Apple 

to kind of make this certainly a trend. What are our panel's reaction to this idea of disclosure of 

odds as kind of being the way forward here? 

KEITH WHYTE: I'll start out and say, we've got a lot of experience with this in the gambling 

world, and it is not harmful, but unlikely to be effective. Most people don't understand odds and 
randomness in the most simple dimensions, especially when you're talking about dynamic odds. 

It's almost impossible for people to figure that out. 

And you have to look at the people you're disclosing to. If it's a young person or someone who's 

variable the gambling addiction, they're going to understand that information completely 

differently than a rational, or well-informed, or non-addictive consumer. So again, from the 

gambling addiction space, there's been few studies that have found much impact on odds and 

randomness disclosure around slot machines. 

It doesn't hurt. It doesn't lead to negative perception, except in some ways if you-- there are ways 

to talk about odds and randomness within gambling that can actually encourage or lead people 

into false beliefs. But by and large, I think that information is OK. 

It's valuable. It's true. It's factual, and should be disclosed. But I think the next step is to make--

is to find ways to make it sticky and entertaining for consumers, and to make such disclosures 

impactful. And so I think there's a whole lot we can do to try and find ways to communicate 

those odds to people in ways that they're going to understand and be able to make more informed 

decisions. That's the ultimate point. 

The point of disclosure is to help improve and change consumer behavior. The disclosure itself is 

not the point. It needs to lead to something. And that, of course, can be measured and evaluated, 

and there can be a feedback loop to find better and better ways to do it. 

ARIEL FOX JOHNSON: Yeah, I would second that. Say, I don't think a kid is going to make a 

significantly better decision with certain odd disclosures. And while it's a good step, it can't be a 

step that replaces more meaningful change. 

KEITH WHYTE: Well, another thing I'll say just real quick, is that look at Powerball. Your odds 

are 246 million to 1. Does that stop anybody from buying Powerball tickets? 

No. Some people love to chase long odds. That's part of the thrill. 

That's, frankly, part of the addiction for some people. So again, not a magic bullet. A good first 

step, but it's a first step towards a lot of changing behavior, and that's a much bigger challenge. 

ANNA LAITIN: Yeah, I'll jump in and agree with all of that and say, from first inclination it's a 

good step, but there's a lot of questions about when that disclosure happens and how. If the 

disclosure only happens at the time you buy the game, and then it's weeks, months later when 

you're actually playing and encountering the loot box, and how do you have any recollection of 

that at that time? Is that a meaningful disclosure? 

And it's something, Keith was talking about different ways of doing disclosures. I think finding a 

way that people can really understand what's going on, and that creativity is great, but we have to 

remember that these games are looking to have people play these loot boxes. And so finding that 

line, you can be creative in the disclosure, but the reality is they want people to buy these, and so 

is there a need to disclose in a way that creates some friction, slows people down, makes them 
think, and that may, of course, stop them from playing the little boxes, which takes away 

revenue. 

PATRICIA VANCE: I think you have to trust that the industry is serious about making the 

commitment that they announced this morning. They have their own customers to serve. And 

they've made a commitment to make disclosures easy to access and to be understandable. And as 

we learned earlier this morning, loot boxes vary game to game, loot box to loot box, and if there 

is no one silver bullet for disclosures, there's no one standard, I think we have to leave it to 

individual game developers to develop the right type of disclosures for their game and for their 

customers. 

I also think it's different disclosures for different audiences. I mean, what you would disclose for 

a parent, like we just created a blog that helps parents understand what drop rates mean, but it's 

complicated. And instead of-- I think if-- to make a parent comfortable, we're better off focusing 

on a generalized disclosure up front that this game enables in-game spending, and then point to 

parental controls that allow them to limit the amount of spending that their child can enable. 

MARY JOHNSON: Just picking up on that point about the disclosure of in-game purchases. We 

heard today, that some feel that that disclosure isn't prominent or detailed enough. Are their 

thoughts for ways to-- how would you respond to that? And then also, are their thoughts of other 

ways to improve upon that to make that more impactful for consumers? 

PATRICIA VANCE: You're talking about the in-game purchases notice. I mean, I walked 

everybody through the rationale for making that decision, and that was the right decision based 

on our research. So it was informed based on what we were hearing from parents. 

Our rating system, at least the upfront information that we provide prior to purchase, needs to be 

really easy to digest. Otherwise, it's going to get ignored. And so we really try to make our 

information concise, easy to digest. 

We cannot throw a lot of information at parents. They just-- their eyes glaze over. But if they 

want to go deeper, for an example, we offer rating summaries on our website and through our 

mobile app. So that if a parent wants to know what do we mean by suggestive themes, or OK, it's 

teen rated, but I have a 10-year-old, and I really want to make a decision. I want to understand 

more about the context of the content that we've called out. They can go to our rating summary 

and read a paragraph or two giving them far more information, far more examples, but we can't 

expect-- a, we can expect that to be on the box. 

We can't expect that information to be up on a mobile screen when you have this amount of real 

estate. We have to give different layers of information depending on how parents want to digest 

it, and what parents are looking for. But look, the drop rates is really to serve the gamers. The 

drop rates are really to provide clarity about the relative rarity and probability of getting certain 

items in a loot box. 

And I think whether-- I think most gamers would understand what that means if they've been 

playing a game for years. They know what those stars mean. They know what the different terms 
are that are being used, whether it's legendary, or rare, epic. And the drop rates, I think, they 

understand based on the context of the game that they're very familiar with. So I don't think we 

should underestimate gamers ability to figure it out. 

KEITH WHYTE: I think that-- just to add to that a little bit-- I think sort of implicit in the 

question is focusing on point of sale. And that's akin to when someone sits down to a slot 

machine or blackjack table saying, oh hey, here's a plaque. You may lose you may lose your 

money, and this is a random game, or if you go to a bar, you expect your bottle of beer. It's not 

going-- it can't tell you, point of sale is a very, very limited time for all sorts of reasons to 

provide this kind of information. 

And it really-- the true way to approach this, as with any other public health issue, is through 

large based awareness campaigns. ESA is starting in that direction. I think there needs to be 

others. 

It's almost impossible. We can't push all consumer edge protection at point of sale or point of 

purchase. That is there is a time. 

I mean there's things we can do there, but it's got to be throughout the lifespan. I mean if we're 

not talking to kids in schools about this, those measures are not going to be-- again, they're not 

going to hurt, but they're likely-- they're not likely to be very effective. It's got to be 

multilayered, multifaceted, multi-year approaches all across the lifespan. 

ANNA LAITIN: And I'll add to that analogy. I think it's not sitting down at the slot machine. I 

think it's entering the hotel in Vegas, and it says, you may end up spending money here. Possibly 

at a restaurant, maybe at a show, or maybe you'll be at the slot machines. 

And that's what's so hard about that contains in-app purchases. As a parent, I look at that. I don't 

know if that's loot boxes, or you can buy another world sometime later. It could be anything in 

between. And so to get that level of specificity when consumers are at the point, or when players 

are at a point where they're accessing that stuff, that's a different moment. The point of purchase 

is really, really important, and I'm thrilled that that's there, but it's not doing enough of the job. 

WILLIAM DUCKLOW: So Keith had mentioned the idea of dynamic odds. And one of the 

specific types of dynamic guides that we saw come up in comments, is the idea that you could 

actually guarantee a specific item or a specific rare item to pop up after a certain number of loot 

boxes. I'm curious what the panel's reaction is to that idea? Does that increase clarity for gamers, 

or could it actually counterintuitively increase the number of boxes that they purchase? 

KEITH WHYTE: I think it absolutely depends on the gamer, right? If you're a kid, if you're, 

again, vulnerable to addiction, there's lots of people that can perceive those as absolutely 

exhortations to play. The gambling world, you only have to admit you've lost when you stop 

playing. So anything you can beg, borrow, or steal to stay in action, you're always one bet away 

from winning everything back. 
If you believe, or you know, or you think you know that additional play or additional spend is 

going to guarantee you an item, and why would you stop anyway, but especially if you're 

addicted, especially if you're risk for addiction, especially XXXX. Other players, recreational 

players, non--problematic players, adults, may be able to see-- maybe able understand the 

dynamic odds better. And again, that's one of things that makes this hard. You're talking 165 

million people, but some of them are people who either have problems or likely to develop 

problems, and for them, their judgment is by definition impaired. 

They are worse at understanding odds and randomness than others. They have cognitive 

distortions. They have illusions of control. 

And providing information dynamic odds in such a way to make them think that persistence is 

going to allow them to win that epically rare item can be disastrous. That can absolutely be a 

pathway to gambling problems if the problem is not already there. So again, it's hard to answer, I 

mean a lot of it depends on the player, in our opinion, because we've seen this happen in the 

gambling addiction space with devastating consequences for some. And we predict that would be 

the same within this loot box space. 

ARIEL FOX JOHNSON: I think this also just sort of speaks to the recurring theme you've heard 

today about the need for more research, and different individuals respond differently, and how 

can we support that. 

MARY JOHNSON: So--

PATRICIA VANCE: Can I just-- can I just add one thing? There's is a theme that you know 

incentives are bad. Rewards are bad. 

Games are all about rewards and incentives, and that's what makes them fun, and that's what 

makes them compelling forms of entertainment. So I want us to be careful about how we frame 

the conversation. Providing an incentive isn't on its own a bad thing. 

It provides challenge. It provides progression, encouragement to progress through a game. So I 

think it's important to not tar all types of incentives as somehow bad. 

MARY JOHNSON: Oh, that's a good--

ANNA LAITIN: Go ahead. 

MARY JOHNSON: I think that's a good segue into sort of another question, which is, what 

about just offering randomized loot boxes for free, essentially? And only available through 

gameplay. And then instead having specific virtual items, or bundles, or passes available for 

purchase, sort of á la carte. Why not-- would that solve the problem? 

KEITH WHYTE: Not all of it. Not from a psychological model of addiction, no. Whether or not 

a reward is monetary or not, whether or not how you pay for it, those things or not are slightly 

salient in addiction, but not entirely. So it wouldn't-- making me boxes free would not remove 
the risk that some people will become habituated and conditioned to, and will play them 

obsessively. 

We see this in the social casino space all the time. Free to play social casinos have quite high 

rates of people who will play til extinction, and get a billion chips, and then they'll spend days 

and hours playing all those chips to extinction again, so they can go buy more free chips to 

continue to play, even though they know they will never win anything of value. So no, it 

wouldn't. 

It won't solve-- it might solve some people, but it won't solve the truly vulnerable people for 

gambling problems. It'd be great if it was that easy. I'd be out of a job, but gambling addiction is 

a little bit more sticky than simply price. 

PATRICIA VANCE: I mean, I do think that would obviously change a lot of the economics. 

You may find the upfront cost of games to be higher. You may find it has a huge impact on the 

free to play market, particularly in the mobile market. These independent developers in particular 

need revenue streams to monetize, to cover the cost of development. 

So I think it obviously would have huge impact on the economics of the business, which I think 

you need to be careful about. Plus, many loot boxes are free and they're optional. So you don't 

have to buy a loot box to play through a game. 

ANNA LAITIN: I think if-- it's interesting that you say how much it would change the 

economics if people could buy the things they're currently winning in loot boxes. I don't-- I have 

no studies, no knowledge, but it would be interesting to see how that played out, and how much 

the reliance on low boxes is necessary for the economic viability of these, because again, if we're 

talking about warnings, and disclosure, and making sure people are aware, and if paid loot boxes 

are necessary for these games to continue to exist, that's something-- that alone is something that 

parents and others should be very aware of, that these games can't exist without these, and that 

the games are relying on people taking the chance here in order for them to continue to exist. 

MARY JOHNSON: A follow up, do you think there should be any kind of cap on the amount of 

in-app purchases for children and adolescents? So for example, you know you may see bundles 

that are $99.99, or range anywhere from like $2.99 to almost $100. Would it make any sense to 

limit the price cap for loot box-- sorry, for in-app purchases for games that are marketed to 

children and adolescents? 

ARIEL FOX JOHNSON: So I think it would make sense to limit or eliminate potentials for 

spending in games that are marketed to children and adolescents. I do worry that if we say, if you 

have a loot box, or you have an in-app purchases, you have to rate your game for adults that that 

might be seen as a get out of jail free card for people who have games that are, in fact, really 

appealing to 10-year-olds. And so I think you have to sort of consider it carefully, but I think that 

parents would feel a lot more comfort if they knew that there were limits for their kids spending 

for a certain age range games. 
PATRICIA VANCE: Based on our research, almost the 7 out of 10 parents have rules that their 

kids can't make any in-game spends. So we believe that parents need to be parents, and set their 

parental controls, and be informed with the disclosures that we're making, including the in-game 

purchase disclosures, but other information that we're putting on the box and on the product 

detail page. I think parents need to be informed, which is why we're doing a lot to try to educate 

parents, and make sure that they're aware, not just of the disclosures, but also the parental 

controls, and leave it up to them to set the parental controls at whatever levels that they think is 

appropriate. But based on our research, the majority of parents are not allowing their children to 

make any expenditures. 

MARY JOHNSON: And then we've gotten a couple of questions from the audience. This one is 

for ESRB. And the question is, has ESRB found that there's any effect of household income or 

socioeconomic status on use of parental controls by parents? 

PATRICIA VANCE: We've not studied that issue, but I don't know if anybody else here has. 

KEITH WHYTE: No. 

ARIEL FOX JOHNSON: No. 

MARY JOHNSON: And let's see, another one. This one is, why not define the type of purchase 

consumers do care about whether they will be hit with constant money grabs versus rare or 

occasional full game add-ons or subscriptions to pay online. So I think this goes to the issue of 

adding more detail to the type of purchase upfront. I don't know if you have anything more to 

add than what we spoke about earlier with regard to that. 

PATRICIA VANCE: I mean, like I already said, refer you back to the comments I made earlier. 

But I would also just say that I think that you have to look at what problem you're solving. I don't 

think the choice to purchase a game, it would be dependent on that information. I think that 

information is really relevant. Once you're in there playing the game, the most relevant 

information, at least that we know, for parents from our research with parents, is that they want 

to know that there's some ability to spend money inside. And then once they bring that game 

home, hopefully, they'll likely set their parental controls. But that level of detail, I think might be 

helpful. 

But it's helpful only after somebody has made a decision to purchase a particular game. The 

reason why they're purchasing that game is because there's great word of mouth. It's great game 

design, really based on a brand that I know my kid loves. There are a whole host of reasons, 

price point, I mean there's a whole host of reasons why a parent might make a decision to 

purchase a video game. So I'm not sure that particular information is relevant for that upfront 

purchase decision, but might be helpful further down the line. 

ARIEL FOX JOHNSON: I guess I would just add that in many instances, especially when we're 

talking about mobile, the parents aren't really purchasing a game. They're clicking Download, 

and handing over their phone to their kids. And I think one of the things that we've heard from 

parents at Common Sense, is that they do want to know if in-app purchases is a rare, or 
occasional, or even cosmetic, we heard today that maybe that doesn't have as big an effect as we 

thought. Or is it something where it's like, every time your child wants to advance to the next 

level, like the Thomas the Tank game, they're going to have to spend money, because a parent 

might make a very different choice, even though both could be listed as having in-app purchases. 

KEITH WHYTE: And just to echo that, as Rene and others have said, I mean loot boxes, this 

covers just a massive amount of territory. And at some point, yes, providing information on 

better-- to help everybody better understand the risks. I think there's much more risks in some 

types of loot boxes, and much less risk in others. It's still risk, but helping weight that is 

important. But yeah, really hard. 

And I don't know that we really know. And I think, again, it's a call, as Ariel said, and as David 

said, and others, it's a call for more research to help us understand features that are more harmful 

versus less harmful, help us understand relative risk, and then communicating that relative risk, 

because risk is not seen equally amongst users. And there's-- yes, it's a great question. 

When and where, where's the most effective way to provide that information and to whom. In 

some cases, it's the users. In some cases, it's the broad public. In other cases-- and sometimes it 

may need to be tailored very specifically to specific groups. And I think we're only just the 

beginning stages of understanding this as not a technology problem or a game problem, but as a 

public health issue. 

And if you look at it through a public health lens, I think that points to, again, broader 

prevention, educational initiatives, more of a public conversation around this. And then layered 

with lots of different tools for lots of different audiences, delivered at lots of different times. 

Point of sale being one, but not the only, and probably not even the primary. Just as you wouldn't 

expect your first lesson about alcohol or driving drunk to be delivered when you walk into the 

bar, when you walk in the hotel in Vegas. 

That starts really early and continues throughout your life, because the risk changes as people 

change and mature. And we have even talked about seniors, but we've talked a little bit about 

cognitive development and neurocognitive development. But there's a lot of evidence to suggest 

that seniors may be at higher risk for negative consequences in some of these things as well, and 

it's not just kids. 

WILLIAM DUCKLOW: So I think the final two questions we have today are more broad. First 

off, simply put, can the concerns that we've discussed today regarding loot boxes and other types 

of in-game transactions, can those be addressed effectively through industry self-regulation, or is 

some type of legislative action required? And then beyond that, what might cause the calculus of 

that answer to change in the future? 

KEITH WHYTE: So from our perspective from 50 years of working in the gambling industry, 

self-regulation alone, no. It cannot be effective. Especially, when there's so much profit involved, 

and there's so little understanding of both risk and rewards across this global ecosystem. 
So yeah, we were comfortable with the three legged stool on the gambling side. Industry selfregulation plays a really, really important role, and we partner with a number of gambling 

companies directly. But we also partner with regulators in this space, like the FTC. And as a 

objective, independent, nonprofit advocacy organization, we're the third leg of that stool. We 

play a big role in helping keep the industry honest. 

It's you know trust, but verify. And my counterparts, the left and right probably will also play a 

role. And so, that's the approach that we would suggest based on, again, our experience with the 

gambling industry. 

ANNA LAITIN: I'll echo that. I think there's a lot that industry self-regulation can do. And I 

think ESRB has taken some really important steps. But the chance of this being solved entirely 

by self-regulation, given the broad nature of the industry, given the size of the problem, and 

given the concerns that have been express today, I think there'll be a need for more than just that. 

PATRICIA VANCE: Obviously, I think self-regulation has worked very well, and the Federal 

Trade Commission has looked at our industry over the years, and has written in their reports to 

Congress that we have the strongest self-regulatory code. In high compliance with that code, I 

think we've proven ourselves over the last 25 years that we can do an effective job self-regulating 

and addressing particularly parents' concerns. 

But I would just add one more thing. This is an incredibly fast paced industry. We move really 

fast, and our self-regulatory system moves very fast along with it. 

When we need to make changes to the rating system, we do. When we make changes to the 

marketing guidelines, we do. We are continually adapting and evolving as the industry evolves. 

I don't think regulators can keep up with the industry, and I really fear that should regulations 

come to pass, by the time they're passed they'll be obsolete, or they'll be completely impractical. 

This is a really complicated and very fast moving industry. 

ARIEL FOX JOHNSON: And I guess I would just say that, I don't think that self-regulation can 

keep up with the entire industry either. So I think everyone has to play a role. 

MARY JOHNSON: So last question. Let's say, we're all going to get together again in five years, 

what do you think the key consumer issues would be at that point related to microtransactions? 

Are we still going to be talking about loot boxes at all? So what are your what are your 

predictions? 

KEITH WHYTE: No. 

ARIEL FOX JOHNSON: No. 

[LAUGHTER] 
KEITH WHYTE: Yeah, well if David and I are right, then we're going to see a spike in gambling 

addiction. And so, yeah, I think we'll still be talking about them, unless we really all team up to 

take aggressive action. And that-- I don't know. I wouldn't give it odds, but I think there's some 

good-- there's some good bones there. So there's some framework that we could build on if

everybody really wanted to come together. 

ANNA LAITIN: I think we'll still be talking about transparency, consumer confusion, parental 

misunderstanding of how things work, kids being ahead of their parents. I don't know if we'll be 

talking about loot boxes, in particular. But as I started my presentation, this is part of-- this is not 

something that's specific to games. 

The marketplace is getting more complicated. Pricing is getting less transparent. Purchasing is 

becoming more frictionless, leading to various different problems and some fabulous solutions. 

So we'll be talking about similar concepts, but I have no idea what it'll look like. 

ARIEL FOX JOHNSON: Yeah, I agree. I think the technology might have shifted a little bit, but 

a lot of the concerns will probably remain the same. 

KEITH WHYTE: That's true. 

MARY JOHNSON: Well I want to thank all of you for participating on this panel, and everyone 

who has been here today. I'm going to turn the podium over to Mary Angle to give some closing 

remarks. Thanks so much. 

PATRICIA VANCE: Thank you. 

[APPLAUSE] 

MARY ENGLE: OK, good afternoon, everybody. I'm Mary Engel. I'm the associate director for 

advertising practices here at the FTC. Thank you all for hanging out to the bitter end, and I 

promise I'll be brief. 

First of all, I would like to thank all of the panellists who appeared today for the time they took 

preparing for their presentations, and for presenting their research, and the insights they provided 

today. I've found it really interesting and very helpful. I'm just going to try to kind of quickly 

provide an overview of what we heard today. 

I think, first of all, we heard that loot boxes are just one type of in-game transactions, and that 

there are many different flavors and varieties of loot boxes. Loot boxes do have a number of 

benefits. They enhance gameplay. They make it possible to play games for free. 

They help to keep game prices low, pretty much the same price over time despite inflation. But 

despite these benefits, we've also heard concerns about them, about their potential for addictive 

behavior and the evidence of correlation with problem gambling behavior. We heard concerns 

expressed about whether game companies are engaging in predatory behavior by using 
knowledge of individuals particular game play to maximize the likelihood that consumers will 

buy a loot box, and whether this increases the likelihood of addiction. 

We heard concerns about dark patterns being used with games. Things like grinding, 

appointment dynamics, and pay-to-win, and other techniques that might increase the likelihood 

that consumers will actually buy loot boxes instead of just playing without buying them. We 

heard concerns about it is difficult for people, both adults and children, to know actually how 

much money they're spending because of the way the pricing is presented in the games. And we 

heard that problem gamblers spend a lot of money on loot boxes, and that holds true, even more 

so, for adolescents. 

There were some analogies to baseball cards, and whether this is a relevant analogy was debated. 

It was pointed out that perhaps not because loot box purchases are more frictionless, and are not 

correlated with problem gambling-- that baseball cards are not. We heard about research showing 

that people buy loot boxes for their functional value, but that that is not inconsistent with they're 

also being linked to problem gambling. 

We've heard that the gaming community is very diverse. And actually that 65% or so of 

Americans do play video games. But the gaming community has mixed feelings about loot 

boxes. They mostly feel like that they are like gambling, and are concerned about how easy it is 

for kids to spend money, and how hard it is for parents to control this. 

Gamers also have mixed feelings about government regulation, because they don't trust the 

government to get it right, and are concerned about the impact any such regulation would have 

on jobs in the industry. So they suggest instead perhaps guidance and best practices being 

provided. We heard a lot of concerns about loot boxes, for children in particular, and a 

discussion of research showing that parents have very limited interaction with their kids 

regarding their mobile devices, concerns about internet gaming disorder, where gaming 

interferes with a person's daily functioning in terms of work school or interaction with 

relationships, not just in terms of time spent playing games. 

And we heard about the various parental controls that the industry does provide for consumer--

for parents to control how much time or how much money their children spend on games, and as 

well as the educational information that the ESRB provides. And yet, we also heard that is very 

challenging for parents to navigate all of these controls, and all the different ways, and platforms 

that children may play games, and that there's more parental education is needed. More guidance 

could be provided on these issues. 

There was also a suggestion that there needs to be more research and public health evidence to 

understand the extent of any problem in this area with respect to gambling or addictive behavior, 

and to help develop solutions. We heard ESA announce a new initiative to disclose the relative 

rarity and odds of winning virtual items. That would be put into effect for new games and game 

updates. This news was generally welcome, but considered as a good first step and not a panacea 

to the problem. 
There were concerns for people with gambling problems, that odds aren't going to really matter 

to them, and certainly whether children would understand that or care about them. There was 

some discussion about also that the odds of winning would need to be independently verified, 

because the companies have incentive to lie about what the odds of winning are. They'll make 

more money if they do that. 

There was also some discussion about how meaningful that disclosure is really, and whether-- at 

what point the disclosure would be most helpful to consumers. That point of sale is good, but 

how about down the road, when people are long into gameplay. And also a discussion of really 

disclosures to parents or other adult players, versus disclosures to children, and those being two 

different kinds of things. 

Finally, there was some discussion about whether self-regulation would be enough here, or 

whether legislation would be needed. And we heard a variety of speakers here, both yes and yes 

on both of those. So the FTC is going to be taking this all in. We heard a lot of really valuable 

viewpoints today. 

People are also submitting comments online and we'll be reviewing those as well. So I know we 

have a lot to look forward to. And I want to thank everyone, again, for coming today and, again, 

to our panels for participating. 

[APPLAUSE]