How I hacked the way I do groceries
Tags: showdev, go, productivity, datascience
Published: 2020-01-21 12:24:44

Since I and my family moved to a more remote area last year, we have been doing groceries online. At first we failed a few times, bouncing between different providers until we got a good experience.

For us, a family of four, doing groceries online has few benefits:

#### 1. We save time

It doesn't take us as much time to do groceries since we can just login on a website, pick the previous order, add everything to the shopping cart and tweak some items before ordering again, all from the comfort of your sofa.

#### 2. Increased productivity

We can plan our day better since we know exactly when the food is coming home. We get to do more, we can focus on things that are more important while someone else is picking up the items for you. Leveraging time is powerful.

#### 3. Convenient

With no groceries so-nearby (the closest is like 500m away) two small kids and no car, it's definitely more practical to just get the things delivered. We could walk or just take the bike for few items, but to do groceries for the whole week requires more than hands than we have.

One interesting thing about these groceries delivery services is that they offer recurrent deliveries, i.e. order once, get it delivered every week, every two, three or four weeks. So I started wondering, how could I tap into that feature and get the most out of the service?

## ⚙️ Optimizing groceries

We usually do groceries once a week for the whole week. After few months of doing this online, I started noticing some patterns. There were items that we were ordering every week, while others not so often. For instance cleaning products were usually being ordered every four to five weeks, while others like juice or broccoli every week.

So considering I had enough data, I got the idea to input all that into a program and see the purchase frequency for every item we have ordered.

The idea was to create few **recurring orders with different time intervals**, only with the products we order just that often.

Instead of spending ~20min every week tweaking the shopping list, we can just take that time for ourselves while the groceries were on autopilot.

## 📚 The theory

There are two key metrics we want to measure on this project, for a given product: average units purchased and how often it is purchased.  

For the first calculation, according to [The Financial Dictionary](https://blog.smile.io/how-to-calculate-purchase-frequency), Purchase Frequency (`PF`) is defined as: 

> the number of times an average customer buys a good or service from a single seller in a given period.

<img src="https://latex.codecogs.com/png.latex?PF&space;=&space;\frac{\sum&space;orders}{\sum&space;product&space;purchases}" title="PF = \frac{\sum orders}{\sum product purchases}" />

`PF` is a metric that calculates the average number of times a customer makes a purchase within a given timeframe. I have been buying food from the same store for the last 6 months, so in my case, the timeframe was about 180 days (the exact number is calculated by the code, you will see in a moment how).

Then, Time Between Purchases (`TBP`) calculates the time before a purchase is repeated a customer.

<img src="https://latex.codecogs.com/png.latex?TBP&space;=&space;\frac{Timeframe}{PF}" title="TBP = \frac{Timeframe}{PF}" />

## 🏗️ Preparing the data

There was no way to export the purchase history from the store I buy from, so I just copied the list of items from the confirmation email I receive each time I order. 

I created a cvs file per order, named after the purchase date with format `YYYY-MM-dd`, containing the groceries in the following format

> 2019-11-04,2,Broccoli

So first value contains the purchase date (to make it easier to compute the `TBP`), second is the units purchased and third the item itself.

Also, it was needed to normalize certain values. For example, After I ran the program first time, I noticed there was an item called `Yoghurt Mild Natural` and another one `Mild Yoghurt Natural`. Also, things like `Pasta Fusilli` and `Pasta Penne`, which at the end of the day they are just `Pasta` for me (sorry Italian friends for butchering your food).

This was done manually since there were just few items to adjust.

## ⌨️ The Code

With the data ready, it was time to code. I chose to do this in Golang for a reason, I had learned the basics few months ago and wanted to put it into practice with a real use case. Could have done it in Kotlin, Dart or Java, but wanted to put myself out of my comfort zone.

What the program does is basically this:

- Loop through each order (file)
- For each entry in the order, calculate product units and `PF`
- Calculate timeframe for all the purchases, in days
- Export the result as cvs


```go
type ProductOrder struct {
	Date    string
	Amount  string
	Product string
}
``` 

`ProductOrder` describes an object for mapping each product purchase, i.e. each row of the cvs file.

To read the cvs files:
```go
// readCsv accepts a file and returns its content as a multi-dimentional type
// with lines and each column. Only parses to string type.
func readCsv(filename string) ([][]string, error) {
	file, err := os.Open(filename)
	if err != nil {
		return [][]string{}, err
	}
	defer file.Close()

	lines, err := csv.NewReader(file).ReadAll()
	if err != nil {
		return [][]string{}, err
	}

	return lines, nil
}
```

To calculate the timeframe: 
```go
// timeDiff takes a two dates as string and returns the time difference in days
func timeDiff(initDate, endDate string) int {
	a, erra := time.Parse("2006-01-02", initDate)
	b, errb := time.Parse("2006-01-02", endDate)

	if erra != nil || errb != nil {
		log.Fatal("Couldn't parse times")
	}

	duration := b.Sub(a)
	return int(duration.Hours() / 24)
}

```

To write the output:

```go
// writeCsv accepts a map and generates a csv file from it.
func writeCsv(productPurchaseFrequency map[string]float64, productAmounts map[string]float64, timeIntervalInDays int) {
	fmt.Println("Creating output...")

	output := [][]string{}

	output = append(output, []string{"Product", "Purchasing Frequency (days)", "Avg. Amount"})

	for product, frequency := range productPurchaseFrequency {
		purchasingFrequency := fmt.Sprintf("%f", math.Round(float64(timeIntervalInDays)/frequency))
		averageAmount := fmt.Sprintf("%f", math.Round(productAmounts[product]/frequency))

		output = append(output, []string{product, purchasingFrequency, averageAmount})
	}

	file, err := os.Create("product_purchase_frequency.csv")
	if err != nil {
		log.Fatal(err)
	}
	defer file.Close()

	writer := csv.NewWriter(file)
	defer writer.Flush()

	for _, value := range output {
		err := writer.Write(value)
		if err != nil {
			log.Fatal(err)
		}
	}

	fmt.Println("Output created!")
}
```

After running the code, you get a file `product_purchase_frequency.csv` in the root go workspace, looking like this:

> Product,Purchasing Frequency (days),Avg. Amount
Broccoli,9.000000,2.000000
Granola,35.000000,1.000000
Pasta,53.000000,3.000000
Äpple Royal Gala,10.000000,10.000000

which then I imported into Google Sheets to get a better visualization of the data in a graph like this one:

<img src="https://i.imgur.com/cYnIpfn.png" />

Where the `PF` corresponds to the Y-left axis while the average amount (units) the Y-right axis.

Get the full code:

{% github javierjmc/groceries-purchase-frequency %}

## What's next?

The chart above confirms the patterns I was noticing when ordering new deliveries, and you can clearly see groups of items that are ordered with the same frequency.

Next time I buy food online I will look at this stats and start setting up recurring deliveries so that I get groceries on autopilot.

## FAQ

> What if you want to eat something specific you don't usually buy? 

Sure, that sometimes happens, like the time we made sushi at home. In that case we just go to the store and buy those 5-10 items we will need.

> Isn't it more expensive to buy groceries online?

For us, buying online not only has saved us time but money. It's very easy to buy things you don't need nor want but crave when you physically go to the store.

> Don't you get expired goods or rotten vegetables?

We wouldn't be buying online if that was the case. In Sweden, all the grocery stores providing home delivery guarantee you get fresh veggies, as well as products with at least 7 days from the expiration date, so it's totally fine!.

> Is this software accurate?

As any statistical analysis tool, it works best when you input enough data. The theory and formulas are correct, but the more data you have, the better. Don't expect accurate results if you only input a couple of grocery lists. I actually intend to run this program myself regularly so I get better results and tweak my recurring shopping lists as needed.
 
# Your take

Did you find this useful? Please let me know if you plan to use this in one way or another, I am really curious. Thanks for your comments and your feedback!

--- 

*Thanks for reading! If you like this post please give me some* ❤️ *and share this.*