Humans Can Code Too: Idempotency
Tags: beginners, webdev, programming, architecture
Published: 2019-06-19 02:32:49

---
title: Humans Can Code Too: Idempotency
published: true
tags: beginners, webdev, programming, architecture
canonical_url: https://zafulabs.com/2019/06/19/humans-can-code-too-idempotency/
cover_image: https://zafulabs.files.wordpress.com/2019/06/idempotency.png?w=680
---

Humans Can Code Too is a content series aimed at anyone who is not coming from a traditional computer science background and is trying to gain a better understanding of software development. This includes those who are just getting starting in a programming career, as well as business people who just want to understand what the hell developers are talking about.

## It’s a scary term

Idempotency is a very intimidating word to have someone drop on you in a Pull Request or code review. It sounds super sophisticated and scientific, mathematical even.

The word “idempotency” does come from higher math. However, what a software developer means when they use the word is nothing super complicated.

I’ll give it my own definition

> Idempotency – n
> 
> When a piece of code can be run multiple times with the same input data without causing undesirable consequences.
> 
> <cite>Joe Cannatti — ME</cite>

In practice, this has to do with what’s going to happen if we find ourselves in a situation where we don’t know if a unit of work was run or not. If the code is idempotent, we can simply run it again. If it’s the second (or 10000th) time it’s been run, that’s fine. It doesn’t cause any problems.

This most often comes up in message-passing based systems and background jobs because those systems involve queues of work. Those queues can result in running code multiple times for a bunch of different reasons. That said, it can be an important feature in almost any situation.

## Conceptual Example

At a conceptual level, here’s an example of a non-idempotent way to think about a unit of work followed by an idempotent version.

> Pay $100 to the user whose ID is 123
> 
> <cite>not idempotent</cite>

> Payout the money for transaction with ID 456
> 
> <cite>idempotent! Great job</cite>

### So, what’s the difference between those two?

In the first case, there’s no way to know if the job is a duplicate or not, so if we run that job 3 times, we’re going to send them $300. Businesses usually don’t want to give away extra money

In the second case, on the first run of the code it can say, “Yep, we haven’t sent that money yet, I’ll do it now”, and then on every subsequent run it can say, “Hey, I already did that, I’ll just do nothing”. So you can run that job 1 to infinite times and get the same result.

![](https://zafulabs.files.wordpress.com/2019/06/idempotency.png?w=680)

## Example: Pre-calculating Review Count

```ruby
def increment_review_count(user:)
  user.review_count += 1 
  user.save!
end
```

That’s not Idempotent! If you run that code 100 times you’re going to increase the review code by 100!  
  
Let’s try a safer example

```ruby
def calculate_review_count(user:) 
  user.review_count = user.reviews.count 
  user.save!
end
```

Ah, sweet relief. That one is way better. If you run that 100 times you’re still going to get the right result. The drawback in this example is that it’s also going to be much slower since it needs to read the reviews from the database.

## Example: Payouts

```ruby
def payout(user:, amount:)
  Xtripe.send_payment(id: user.xtripe_id, amount: amount)
end
```

As we discussed earlier, this is no-good because there’s no way for this code to make sure this money hasn’t already been sent. We can use a transaction record to make this idempotent.

```ruby
def payout(transaction:) 
  if transaction.should_be_payed? 
    Xtripe.send_payment(id: transaction.user.xtripe_id, 
                        amount: transaction.amount) 
    transaction.mark_as_paid! 
  end
end
```

That little `transaction.should_be_payed?` makes all the difference. That method can check if the transaction has already been paid out, as well as check for other conditions.

## Example: Backup to S3 (cloud based file system)

```ruby
def backup_to_s3 
  User.all.each do |user|
    S3::File.write(name: user.id, content: user.to_json) 
  end
end
```

The problem with this one is that if your job dies part way through, there’s no way to know where we left off without checking in S3.

One way to fix it is to record a timestamp

```ruby
def backup_to_s3
  User.where('last_backed_up < ?', 1.week.ago).each do |user| 
    S3::File.write(name: user.id, content: user.to_json) 
    user.last_backed_up = Time.now 
    user.save! 
  end
end
```

We could run this job over and over again and it’s never going to do anything unexpected! That’s what idempotency is all about!