Who makes the Objectively Best French Fries?
By: Adam Brownell
Tags: Food, Analytics, Data Science, Foodies, Fast Food

Who makes the Objectively Best French Fries?

French Fries: The Final Frontier of Data Science

In February, our Data Science team had an argument about which restaurant we went to made the best French Fry.

We decided to make it a competition throughout Feb/March and find out who truly was King of the French Fry. Move over March Madness, Hello Starch Madness

Starch Madness: A significantly less athletic tournament than March Madness (but WAY MORE delicious) — Photo by Emmy Smith on Unsplash

I typed up our findings into the first academic paper about French Fries and submitted it to arXiv.

It was promptly rejected. :(

Nevertheless, here is the story of The Objectively Best French Fries, or as I titled it:

An Objective Analysis of the Subjective Quality of French Fries in the Downtown Santa Monica Area

Warning: This article will very likely make you hungry for French Fries

Abstract

It all started with a team lunch. We always ordered fries for the table. After several lunches, we began to ask the question: What are the best French fries near our Santa Monica office? How do you even determine “best”? As a Data Science company, we knew this couldn’t be determined simply by eating fries at various restaurants — we had to structure our analysis around scientific rigor and specified constraints, so that we could be confident in our findings… If the results weren’t reproducible through peer review, what was the point? And thus, Starch Madness was born; a contest wherein a bunch of data nerds objectively decided which restaurant served the subjectively best French fries in Santa Monica.

Introduction

In 1802, Thomas Jefferson served “potatoes served in the French manner” at a White House dinner. From there, the French fry was born. Potatoes cut into slices and deep fried became a staple in American cuisine.

Despite being most associated with burgers and fast food, French fries are served in most restaurants. With their ubiquity, the question must be asked: “Who serves the best French fry?” The simplicity of the food recipe makes this question especially interesting, as there are only a few variables a chef can play with to craft their ideal French fry.

As a California Native, In-n-Out Fries hold a special place in my heart — Photo by In-N-Out Inc.

In addition, some institutions produce mass quantities of fries per day — does this mass production reduce the quality of the fry? Does the fact that they must focus a large portion of their business on producing enjoyable fries create a better quality than a restaurant known for higher end cuisine?

We would like to take this moment to address the research elephant in the room — if this research is even important in the first place. We propose that in fact:

Yes it is.

The pursuit of knowledge is admirable regardless of scale, and there has yet to be any studies conducted on the French fry quality of the downtown Santa Monica area. We may not have millions of dollars in French fry funding (yet) at our disposal, but we have the tenacity and drive (and palettes) to see this project through. It is our pleasure to add a tiny morsel of new knowledge into the human corpus.

1. Method

1.1 Location

We had to first choose the fries that would be entered into the contest. The simplest sampling method was to select fries that could be purchased at a location that was less than 10min walk away from our office. Luckily, our office was located in the heart of downtown Santa Monica and there was no shortage of fry options; in fact there were far too many. To narrow down the locations, we simply typed “best French fries” into Yelp and chose the top 12 results. Several of the locations were eventually replaced due to fry style (see section 1.2) but this methodology yielded a strong crop of contenders.

To avoid bias from judges based on where the French fries were from, each location was randomly assigned a false identity based on popular T.V shows. Figure 1 is the key mapping the real restaurants to their false identity. Henceforth all restaurants will be referred to by their false identity names.

Figure 1: Table of False Identities given to each contestant to remove location bias

1.2 Style & Procurement

There are many types of French fries, and unfortunately comparing different styles of French fries is like comparing apples and oranges, and thus we had to standardize our French fry types. We settled on the most common style — salted potato fries. This excluded popular variants such as waffle fries and sweet potato fries, but still allowed creativity and diversity amongst the participants.

We kept style preference/bias to a minimum by only judging classic potato fries — Photo by Emmy Smith on Unsplash

Judging one batch of fries at a time was not a feasible option, as that would have required quadruple the amount of time the current experiment was allotted as well as made tournament play (1.3.2) impossible. This increase in time would have inevitably resulted in judging fatigue. We instead attempted to gather as many fries as required per round at one time without degrading the integrity of each fry batch.

Despite our earnest efforts to procure the fry batches as close to simultaneously as possible, our retrieval was not without flaws and some may argue the distances a fry batch travelled correlates inversely with the batch’s score. This conjecture is exhaustively reviewed in our discussion section 3.2.

Individuals in charge of picking up the French fries were instructed to order the fries by saying, “can I please have an order of fries.” Whatever style the restaurant produced from this order was considered their typical fry style.

1.3 Experimental Design

Similar to the FIFA World Cup tournament design, Starch Madness was structured with two sections: group play (1.3.1) and tournament play (1.3.2). Adequate performance in the group play stage qualified you for tournament play.

Figure 2: Group Play Divisions. “Classic” movie distinction is subjective (but correct)

1.3.1 Group Play Stage

Each of the 12 participating French fries were randomly assigned into 4 groups of 3 contestants, called divisions, to be rated during the group play stage. The scoring used was the F.A.T System, defined in 1.4. Judges were instructed to use the F.A.T system (1.4) instead of comparing fries from within a division to reach a score.

This is an important distinction as producing an objective score for each fry is a stronger indicator of fry strength than a comparative one. In addition, if a particular division has three excellent (or terrible) batches, they should be admitted into tournament play on their strength alone, rather than their division’s overall strength. Figure 2 Outlines the divisions.

1.3.2 Tournament Play

Figure 2: Group Play Divisions. “Classic” movie distinction is subjective (but correct)

While not the most scientifically rigorous way to compare French fries, it certainly is entertaining. During this stage the top 8 highest rated French fry batches were seeded and placed in a single elimination bracket. Each head-to-head match-up was a blind vote of which fry the judge preferred. Through the tournament play, a winner is crowned.

1.4 The F.A.T System

One of the first obstacles set forth when judging French fries is how to standardize scoring. And to do this, one must answer a deep philosophical question of, “what makes a good French fry?” Our team of fry-experts narrowed the quality of a French fry down into 3 categories:

Flavor.

Appearance.

Texture.

This 3-part scoring rubric was brilliantly named the “F.A.T System.” Each of these represents a core attribute of a good French fry. By rating batches based on these metrics, we can decide with confidence which fry is best. Each of these metrics is scored on a 5-point scale, with 1 being the lowest and 5 being the highest. Averaging these three numbers produces a fry’s overall quality and helps the judges make important voting decisions during tournament play.

1.5 Judges

The French fries were judged by the capable RETINA.ai team, whose love of fries is only surpassed by their love of data collection and analysis. The merit of these judges is excellently defended in 3.5.2

2. Results

After the group play stage, and each contestant had been graded on the F.A.T System, we produce the following data table below:

Those highlighted in green received acceptably high enough F.A.T scores to qualify for the tournament round. Those not highlighted fell below the McDonald’s line, named for the fact that not only was McDonald’s the lowest scored fry that qualified, but also with an average score of 2.43/5 McDonald’s can be considered an average, mediocre fry.

It should also be noted that Little Ruby had an absolutely incredible score, scoring nearly a half point higher than any other contender and a near perfect score of 4.44/5. An additional interesting result was that Little Ruby and Burger Lounge scored higher than Great American Fries, a food truck that exclusively serves French fries.

All contestants above the McDonald’s line were entered the seeded tournament play resulting in the tournament structure seen in Figure 3. False identities are used because this is the same bracket judges saw during the contest.

Figure 3: Qualifying Fries seeded for Tournament Play Bracket

After the first round, Los Pollos Hermanos & Mos Eisley won as projected, with Los Pollos Hermanos sweeping JJ’s Diner 7–0. However, in the bottom two matchups the both fry-offs were upsets. Krusty Krab won decisively with a 6–1, while Bob’s Burgers squeaked out a victory 4–3 in a hotly debated heat. Krusty Krab knocking out Leaky Cauldron, and doing so effectively, is very surprising because as stated in the previous section, Leaky Cauldron (Great American Fries) only sells fries — and yet they couldn’t win a single game in the playoffs.

The Final Four went as expected, with #1 Los Pollo Hermanos beating #4 Mos Eisley Cantina and #6 Krusty Krab dominating #7 Bob’s Burger 7–0. The Championship game was an instant ESPN classic. Seed #6 Krusty Krab, with only 1 vote against it the entire tournament, took on Seed #1 Los Pollos Hermanos, whose group play score was significantly higher than anyone else — and 1.44 points above their championship opponent. But in the end, after intense debate and judging, the underdog took the championship 4–3.

Hi Ho Burger (Krusty Krab) was crowned victor of Starch Madness!

In an amazing underdog story, Krusty Krab (HiHo Burger) wins Starch Madness despite being 6th seed!

3. Discussion

Now that we have objective scores for each fry, we are given the opportunity to see the impact of certain attributes on French fry quality, derived from F.A.T score averages

3.1 Does Price Imply Quality?

Figure 4: The Relationship between Price and Fry Quality

When looking at French fry price vs quality, there is one thing that sticks out quite clearly: the most expensive French fry in the contest, The Misfit, was also voted the worst. In fact, the Misfit is such an outlier that inclusion of it as a data point actually changes the relationship between fry price and quality from positive to negative.

The correlation between price and quality without the Misfit is moderately positive (0.40 Pearson Correlation Coefficient) and the correlation with the Misfit is none to negative (-0.08 Pearson). To sum this up nicely:

Price does imply quality, but only up to $10

The Misfit (and perhaps any/all fries that are above $10) are a rip off. You are better off going to McDonald’s

Speaking of McDonalds — the cheapest option is actually the best option for the price you get. While absolute quality does increase as price increases, it does not increase to the point that you increase the quality per dollar spent. In fact, a dollar spent at McDonald’s produces 38% more fry quality than the next restaurant. If you’re looking for the best “bang for your buck,” McDonald’s is the best option.

3.2 Does Distance imply Quality?

A major discussion amongst judges during scoring was whether the distance from our office implied if the fries would be better or worse. The reasoning behind this is simply because fries are best when they are fresh and hot, and if we had to walk them back to the office for several minutes, they could lose more heat than a batch procured closer to our office.

There are strong arguments for both sides. In support of this theory, Umami burger is one of the farthest and lowest rated fries, Little Ruby, Plan Check, and HiHo all made the playoffs and are very close to our office. However, Misfit, the closest restaurant, is the lowest score, and Great American Fries is far but still very high quality. Clearly, the only way to solve this is with

Statistics!

Location proximity and its Relationship to Fry Quality. Quadrants named moving clockwise from top-left

We charted where each restaurant was and found the Euclidean distance from each to our office, marked on the map. Firstly, even by removing The Misfit from the data, only 10% of variance can be explained through a linear or exponential relationship (0.1 R2 value), so the hypothesis that distance degrades quality is rejected. What the distance-quality chart does produce is an interesting bucketing strategy for these restaurants. By bisecting the data by both the McDonald’s line and the average distance, we can create 4 quadrants/segments.

Quadrant I can be labelled “Easy Bets,” where each restaurant is close and of high quality.

Quadrant II can be labelled “Worth the Journey” since they are farther away, but still high quality.

Quadrant III can be labelled “Not Worth the Journey” for having below-McDonalds’ quality at above average distance.

Finally Quadrant IV can be labelled “Deceptively Bad” because while they are close to the office, they are not even the short trip. The Quadrant IV name also stems from the fact that we frequent the Misfit often since it is so close, and we have grown to believe that their fries are quite good. Our studies show this is incorrect, and our frequent visits have simply biased us

3.3 Did External Factors influence our Voting?

3.3.1 Comedic False Identity

The false identities were added to the restaurants so that people wouldn’t be biased based on their prior experiences with that restaurant when voting. However, in our attempt to mask their identities, we may have introduced new bias through our associations with the cinematic universes that the fictional restaurants are from. Specifically, what if a restaurant was given a comedic identity, and because of our positive association with that movie/show, we graded it higher? Or conversely, what if a restaurant was given a very serious movie/show identity, and because of this, the judges scrutinized the fry more closely? Would people treat associations to Silence of the Lambs more negatively than Dumb and Dumber?

The answer is no. Splitting fries by comedy, we get a t-test p value of 0.52. The comedic value of the false identity did not influence our voting.

3.3.2 Weather

We conducted these experiments during February and March — the heart of winter in Los Angles. It reached a chilling 58 degrees during some of our rounds — did this influence our votes? Did the cold weather make the warm fries taste even better, and therefore result in higher F.A.T Scores? Or did the overcast weather make us somber, and therefore more critical of the fries we ate?

The answer is no. Splitting fries by if the weather as above 60 degrees of not, we get a t-test p value of 0.74. The weather outside did not influence our voting

3.3.3 Day of the Week

Does judging fries on Fryday impact our votes? No, a p value of 0.55 rejects this hypothesis.

3.4 Can Yelp Ratings indicate F.A.T Score?

Yelp is a service that prides itself on crowd-sourced ratings for restaurants. Many often check Yelp first before deciding if they should eat a particular location. Can we use the same strategy when selecting a location to dine on the best French fries? One would hope that as the overall Yelp score increases, so does the quality of the French fry.

Figure 8: The Relationship between Yelp Scores and Fry Quality

Unfortunately, this is not the case. As seen in Figure 8, highly rated Yelp eaters fail to produce on average fries that are as highly rated as their restaurant. Even when we cut out the Misfit outlier, good Yelp restaurants fall below the Yelp-FAT Line, which simply charts a 1:1 relationship between Yelp score and F.A.T score — a restaurant above the Yelp-FAT line (such as 3.0 Yelp score) produces fries of a higher caliper that the Yelp score would suggest. Our explanation for the 4.0+ restaurants failing to produce 4.0+ fries is that the Yelp score includes not only other foods, but also the ambiance of a restaurant. High Yelp score restaurants tend to have a better dining experience than lower scoring locations, despite their fries being of an equal or lesser quality.

3.5 Addressing Potential Confounding Variables

3.5.1 Playoff Rankings

Everyone loves an underdog story, and so it is possible that despite having blinded each restaurant during group and tournament play, knowing the rankings could have compelled voters to choose the underdog when a contest was close. Fry quality in tournament play was higher than in group play, and so this is entirely possible. How to control for this in the future is to avoid telling rankings, but also changing the false identities several times throughout the contest, thereby removing any bias judges have procured during previous tastings.

3.5.2 Judge Acumen

To address the concern that the judges themselves were bias and not indicative of scores of those not involved in the research, this is also possible. However, each judge has passed the French Fry Judge Certification Program (Figure 9) and therefore their expertise should be considered adequately accurate for French fry quality.

Figure 9: Logo of Certification Program all Judges received

4. Conclusion

Ranking French Fries is a relevant subject to all readers, and we hope to continue our research beyond the Santa Monica area. While our resources confined us to Santa Monica and single elimination, next steps could be an expansion of both region and trials. We are all interested if these results hold up across all restaurant locations, or if there are locations that claim to have the best French Fry that wish to enter the challenge.

For those unconvinced that the research topic of Fry Quality of Santa Monica French Fries was worthy of publication, we would strongly argue that the pursuit of knowledge should always be encouraged, no matter how small the impact or specialized the application.

We hope that all those who read it feel empowered to conduct their own French fry quality assessment as a form of peer review, and publish any interesting results found. The research area of French Fry Quality is young and ripe for more publications.

The Quest for the Ultimate French Fry is still afoot, and Data Scientists have a major role to play in the road ahead.

Visualization of humanity’s quest for The Perfect French Fry — Photo by Matthew Kalapuch on Unsplash

Hope you enjoyed,

Adam “Fry” Brownell