Finding Your Ideal Chocolate Chip Cookie: A Web Application for Parameterized Recipe Generation via k-NN Regression
Tags: octograd2020
Published: 2020-05-20 05:27:35

## Introduction

For my senior capstone, I set out to create a web application for parameterized baking recipe generation -- a project that would require me to employ a wide-swath of software development practices across both frontend and backend disciplines. Baking is an activity that has long-held significant appeal to me, as it is rather like chemistry; small perturbations in inputs and process can yield very different results, but, controlling for outside variables, these outcomes are broadly deterministic. The structure formed by these inputs, processes, and outputs point to something that may be possible to explore using the application of the correct algorithms. The humble chocolate chip cookie provided the subject of my application, owing to the high availability of recipes, the relatively low-variation in ingredients from one recipe to another, the widely divergent outcomes from one recipe to another, and the general universal appeal across most of my hypothetical target audience. 

Recipe websites are a funny genre of website -- notorious for their SEO-driven pages-long screeds that visitors must scroll past before they get to the recipe, the inability to easily switch to a user's preferred system of weights and measurements, the inability to adjust general purpose recipes to suit one's dietary restrictions or culinary preferences, and the inability to scale recipes up or down based on the user's intended yield. To that end, this project has two main goals: 1) To build an interesting dataset of chocolate chip cookie recipes from which I am able to meaningfully manipulate -- forming our model, and 2) To build a web application that alleviates some of my personal peeves with recipe websites. The former goal was to be fulfilled using the manifested dataset, and presenting it and our model in a clean, concise manner. 

Creating the dataset and model would consist of several steps: collecting recipes (our *input* and *process* data), generating companion data describing the results of these recipes by both qualitative and quantitative means (our *output* data), cleaning and transforming data, and creating our model to manipulate the aggregate data to generate recipes from given parameters. I may have taken one of two approaches to the collection of data: either bulk scrape hundreds of recipes from as many sites I can find, or utilize a small, curated number of recipes from perceived high-quality web and print sources. I chose to go with the latter approach, and for two reasons -- I wouldn't have to build a web scraper to do it, and it saved me a lot of dataset cleaning down the line. However, my approach to storing the data does allow for a high degree of extensibility -- this approach will be explained anon. To generate the companion data, I chose to classify the cookies from each recipe based on a small set of quantitative and qualitative characteristics. In order to do this, I baked each of the recipes in the dataset, and had some friends rank them with me in a single-blinded manner. I chose to use a *k*-NN classifier to form the basis of my model, as this allows easy extraction of a recipe given parameters that lie on a scale, which our dataset does.

The web application that I built to present my dataset consists of two parts, an Angular 9 web application frontend -- hosted on Microsoft Azure, and an Azure Function/Azure Storage backend. I chose these technologies in particular as they mirror those that are utilized where I will be employed beginning next month. Frankly, using these cloud technologies for such a small project is probably a little bit of overkill, but implementing them provided some good experience and some interesting challenges. Another concern about using a cloud platform such as Azure is cost, but Microsoft is pretty generous with the education credits.

## Dataset

The twenty-five recipes I selected are all derived from websites and books under the spirit of fair use. I attempted to gather recipes that represent the extremes of the chewy-cakey, crispy-soft, and thin-thick spectra. Allowing for the ability to adjust for dietary restrictions later on, all of the initially-selected recipes are more-or-less ``standard'' recipes following familiar ingredients. Some allowances for more-esoteric ingredients such as browned butter or oats were made due to their suspected significant impact to final recipes. Each of the recipes' ingredients, amounts of ingredients, and steps were recorded into a standardized json format to later be used in our cleaning step. Similar steps from write-ups were unified into a single key within our *steps.json* file, with a single independently-written version that allows for text interpolation where necessary. Ingredients were also recorded separately in our *ingredients.json* file, and combined with other information such as category, standard measure (volume, mass, or discrete), density/mass, as well as flour-type protein content, fat content, and allergen substitutions where appropriate.

When the friends that I enlisted to help and I finally found the time to proceed with the generation of our output data (which wasn't done until late February, thankfully we got it done before the lockdown), we took an entire day baking single-blinded batches of dough. We rated the recipes on a 0-5 scale (6-point) based on softness/crispiness, cakiness/chewiness, and lightness/richness. We also incorporated thickness and width as a single parameter *spread* (the ratio of width to thickness). Comments about the recipes were also recorded, but are not included in our dataset at this time. After many pounds of flour, sugar, chocolate, eggs, and butter expended, and far too many calories consumed -- I had the second half my recipe dataset.

From there, it was necessary to clean the recipe ingredient data to prepare it for ingestion into our model. I needed to convert plain-text amounts such as "2.5 cups" of an ingredient in the raw recipe to its mass via the ingredient's density value in the *ingredients.json* file. Once all ingredient amounts were converted to mass in grams, I standardized each recipe dough amount to the same mass for ease of comparison between them. The recipes (now *baked*), ingredients, steps, and an index of recipe names to recipe metadata (to facilitate recipe retrieval) were all uploaded to a *blob* storage container on Microsoft Azure.

## Model

With our qualitative and quantitative data, I am able to place each of our recipes at a point within a four-dimensional space, composed of the spread ratio between cookie thickness and diameter, and our three qualitative dimensions (soft-crisp, cakey-chewy, and light-rich) – our shorthand for this space is, jocularly, "cookie space." I am then able to perform an inverse square distance-weighted k-nearest neighbors (*k*-NN) regression for any point within this space to generate our model’s best guess of a recipe that might fit that point (*k* = 4). Each recipe's weight on the final output is calculated with a function that calculates the inverse weighted distance for each recipe. I generate the recipe by summing all ingredient values per each of the four recipes, multiplied by each recipe's respective weight (all weights are normalized s.t. they sum to 1).. Any ingredients below a certain threshold are substituted with a near replacement from the same *category*, and the recipe is re-normalized to our standard mass.

![Illustrating a potential distribution of recipes on cookie space. The 4th dimension is shown by the size of the circles.](https://dev-to-uploads.s3.amazonaws.com/i/ajgh8qp7b3flqmyrvrnt.png)

![Weights are calculated based on inverse-squared distances.](https://dev-to-uploads.s3.amazonaws.com/i/3spzxsak7e6qm5acjagz.png)

This model is not without its limitations. As it is subject to a low-number of samples of subjective ratings, the output itself should be treated with some suspicion. Additionally, as the outliers in the recipe dataset may not represent the true bounds of *cookie space*, this model is unable to generate anything beyond the most extreme recipe in the dataset. This likely has led to a narrower band of possible recipes than may have been generated by attempting to fit the recipes to some sort of straightforward function that would have expanded the bounds (but how much fun would that have been?).

## Backend

The backend of our web application consists of an Azure Function written in ASP.NET Core and deployed to the Azure Cloud. This function implements a REST API that receives requests over HTTP from our frontend to one of the API's endpoints, and then responds with appropriate data to fulfill those requests. REST APIs offer a straightforward method to serve data over HTTP – via one of four operations on the endpoint with an optional *payload* – these are **GET**, **PUT**, **POST**, and **DELETE**. As our app's data-flow is currently unidirectional, from our backend to our frontend, the application currently only uses **GET** operations to communicate between the frontend and backend. The currently offered API endpoints are:

* **/api/recipes/** --- GETs an index of our dataset's recipes. Previously generated recipes are stored on a separate index that is not exposed by the API.

* **/api/{name}** --- GETs a named resource file, such as *recipes.json* or *ingredients.json*.

* **/api/recipe/{id}** --- GETs a recipe from the dataset or a previously-generated recipe by its 32-bit base-58 unique ID (determined by a *FNV-1* hash of the .json file's contents).

* **/api/recipe/gen/{parameters}** --- GETs a generated recipe given the parameters, which are deterministically encoded into a 25-bit string in base-58. These parameters also serve as the recipe's *ID* in storage. If the recipe already has been generated, it will return that, otherwise it will calculate, store, and return a newly-generated recipe.

All data served and utilized by the backend is stored in a blob storage container on Azure. This allows a set of files, stored as raw bytes, to be retrieved and stored by the API as necessary.

## Frontend

My frontend is a web application created with Angular 9, and using NgRx state management. Angular is a TypeScript-based, open-source web application framework. Angular applications are structured as a hierarchy of *modules*, each containing components that reference HTML templates -- allowing dynamic, asynchronous loading of components within the application. NgRx is an implementation of the Redux pattern of immutable shared state across an entire application. As it is not possible to directly modify state, developers need to take care to understand the data flow from one component to another. Components receive data from components higher in their hierarchy via property binding in the higher component's template. Components may mutate state or cause side-effects only via *actions* that are *bubbled-up* from components to components and modules higher in the hierarchy, before those *actions* are *reduced* into state changes by *reducers*, or used to return other *actions* containing data from side-effect creating services that may then be put in to the store or used to dispatch more *actions*.

![General structure of an Angular application utilizing NgRx state management and smart/dumb components.](https://dev-to-uploads.s3.amazonaws.com/i/ugabf4ry6q8cea6gfcpg.png)

When the frontend's components are first loaded, it issues an action to request the *recipesIndex*, *ingredients*, and *steps* from the API, and then saves them in the store. The user is then presented with a blank tab with the ability to load a recipe from the dataset -- showing a drop-down of recipes from the dataset, generate a recipe from parameters, or input an id (with or without toolbar parameters). Depending on the user's input, the frontend will issue the appropriate actions, which will trigger the appropriate effects and follow-on actions to request and then save the recipe. The tab then detects that the state has changed, and it updates to show the recipe. Users may then modify how the recipe is displayed, or even adjust for dietary restrictions within that recipe tab's toolbar. *Pipes* injected into the module afford the ability to make these changes to data structures and text in-line in the template, an extremely useful capability. The toolbar reads from and issues events to modify the recipe's options parameter, which then effects how the recipe is displayed to the user. Users may add additional tabs to load more recipes or delete locally-stored recipes.

![The Coookies web application. Showing the initial state.](https://dev-to-uploads.s3.amazonaws.com/i/kbs8hjfbjruloahsdwpl.png)

![The Coookies web application. Showing selecting a recipe from the dataset.](https://dev-to-uploads.s3.amazonaws.com/i/9i8uvta3atwy2m9ncqf2.png)

![The Coookies web application. Showing the initial state, showing a generated recipe.](https://dev-to-uploads.s3.amazonaws.com/i/hv7b5mnymhw8m2xi1u8g.png)

## Conclusion & future work

Overall, I would say that I achieved much of what I initially set out to do. I have created a dataset and model for manipulating that dataset. To display this dataset and provide access to the model's ability to generate new recipes, I created a web application. But I haven't completed *all* I set out to do. Due to time constraints, future work will include:

* Deploying the backend and frontend to Azure, then updating DNS records to point my domain name for the application **coookies.dev** to the frontend. I will be sure to let you know when this happens!

* Refining the model to provide more satisfactory results. There is often a bit of a muddled relationship between the recipe generated and the parameters, but this may improve with tweaking parameter weights within the model. I also would like to eventually add further *input* and *output* data to the dataset.

* Improving error handling and adding unit testing to both the frontend and the backend. This is an area that I do not excel at. In order to get to a minimum-viable-product, I neglected a lot of checking for edge cases. Unit tests do not exist outside of those automatically-generated by the Angular command line interface.

* Improving the look and feel of the web application. Adding custom icons to toolbars and moving away from a default Angular theme. Adding the ability for users to print and share recipes would also be useful.

* Even further-forward, possible additions to the web app's functionality may include adding the ability for users to submit recipes to the dataset, and also add comments, ratings, and pictures to the recipes.