Serendipity Engineering - part 1
Tags: balena, iot
Published: 2022-02-15 09:13:14

Back in the summer of 2021 I spent 5 weeks making a Raspberry Pi into a box of puzzles, for no other reason than for fun. It's one of the most valuable things I have ever done at [balena](http://balena.io). Here's why:

##Context
Since balena is a remote-first company we put a lot of thought into making sure we try to connect with each other and have fun. In the past one of the ways we did this was to spend a week, once or twice a year, to physically collect together and spend time with each other. We call these sessions Summit, and there will usually be some swag there, such as devices and SD cards and sensors, t-shirts and hoodies and snacks. However, for obvious reasons, in the last couple of years we have been unable to fly the team around the world and spend time together in the same building, so our summits have had to move to a virtual event. The first time this happened, in 2020, a group of people at balena decided to build a project (the [inkyshot](https://github.com/balenalabs/inkyshot) ) and send one to everyone on the team as part of the swag box. The idea was that it was connecting to all be receiving the same message, initially about summit itself, but latterly to share quotes and insights.

In 2021, about 6 weeks before summit, one of the team rang me up and asked if I had any ideas for a device we could build and send in that year's swag. Of course I did. I rarely have zero ideas. :) However 2021 had an additional constraint: [a boat got stuck](https://en.wikipedia.org/wiki/2021_Suez_Canal_obstruction) and suddenly buying hardware components became almost impossible. With only 6 weeks to think of an idea, design it, build a proof of concept, buy ~100 sets of hardware, build them and send to the four corners of the earth... it was never going to happen.
Then the idea struck me - what if we relied on the hardware the team already had, and just gave them some software? Some fun software....

## Why?
So before we get into the details of the idea, let's address the crux of the post: why did I spend 5 weeks (it took me a whole week to think about it) on a puzzle for the team? Worse-still, I recruited a bunch of other balenistas to help me. So why did I spend 5 weeks of several people's time, making a puzzle?

![definition of serendipity](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v57mvvo684eupg82vdxk.png)

When [Yahoo banned](https://allthingsd.com/20130222/physically-together-heres-the-internal-yahoo-no-work-from-home-memo-which-extends-beyond-remote-workers/) it's employees from working at home, it stated:

"Some of the best decisions and insights come from hallway and cafeteria discussions, meeting new people, and impromptu team meetings."

Whilst I don't agree with the decision Yahoo made, I do agree that good things happen when people hang out together and hack on stuff. So how do we do that at balena, a remote-first company, when we don't have a coffee machine to linger around? We enable everyone to self-organise into projects to build something together, and have fun doing it. 

This puzzle project I had conceived, quickly snowballed from an idea which raised a slight smirk when I told a select few people (the project was kept secret until summit), to one erupting with lessons learned, product feedback and possible future use-cases. It might even turn into another official balena product.

## The idea in a nutshell
The idea was to provide the balenistas with nothing more than an SD card and a postcard printed with a QR code. The code would lead them to an [open fleet](https://www.balena.io/blog/the-road-to-multi-app-transitioning-balenacloud-applications-to-fleets/) on [balenaHub](http://hub.balena.io) which they could then join by flashing the SD card and popping it into a device they already had. The instructions on the fleet card in hub told them to open `http://cattlegrid.local` on a browser, which opened a web server now running on their RaspberryPi device. Here's how it looked:

![Quorum](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k5rsprazs8i5myqimqar.png)
 
Quorum is the minimum number of participants in a system to make it work, and at balena we use the term to mean the number of people who've joined a meeting before we kick off. This was a hark back to a previous teammate, Shaun, who used to say it all the time at the start of the weekly team meeting. I wanted this puzzlebox to tell some of the folklore to new balenistas and stir memories to those that have been here a while.

The "Devices in the herd" line would lead teammates to add more devices to the open fleet, until quorum was reached (60/100) upon which the site would redirect to a page with this image:


![Raspberry pi board](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ga6lkpus5ingfu2n7nl1.png)

The page also had a textbox and a submit button.

The idea was that the user had to try and find more puzzles to solve, by inspecting the device, and each puzzle would provide a code. By entering that code, a centralised API (running on an edge device in my house, obvs) would register that the puzzle had been solved, and everyone's fleet would show the updated count of puzzles solved. This fleet was a **team** effort to find the puzzles and solve them to get the codes. :)

The picture of the pi was actually another hark back to some balena folklore. Alex (CEO) had sat with Chris (hairy labs geezer) looking at a pi together, trying to spot what every chip and port on it was, and whether they could think of a labs project for each one. Here I wanted a puzzle for each.
Additionally, I wanted all of the team to have a physical device in their hands (or on the desk) which they then needed to interact with. It occurred to Chris and I during this project that not everyone on the team has had this experience of physical computing. Actually making the device do something, like light an LED, or produce a noise. Connecting GPIO pins together with wires, making stuff happen! We thought it was super valuable to give the whole team the experience of interacting with real hardware.

Which brings me onto an example of a puzzle. There were puzzles hidden as WiFi access points, bluetooth pairing challenges, digital steganography delivered through the HDMI port, audio clues though the 3.5mm jack, and more. But the first one I made was using the GPIO pins. When the user clicked the pins on the picture of the pi, they got redirected to this page:


![Pi GPIO Puzzle](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5sn3wawf538iie675d6x.png)

This puzzle needed them to earth the pins in a particular sequence, to unlock the code. And any wrong pin earthed, took them back to the beginning of the sequence. :D
I loved this one, because it made people use wires and pins and interact with the device in a way a lot of the team probably hadn't before.

A counter on this page told the team how many puzzles could be found and solved on the device, and when they solved all the puzzles....they were sent to a website (running on a NUC in my house) where there were MORE puzzles to solve:


![Cattlegrid API](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j8wmks5x0jgxlidtvnrc.png)

Actually this is an old screengrab, we ended up with 33 puzzles for the team on this page. Each one uses a [Deploy With Balena](https://www.balena.io/docs/learn/deploy/deploy-with-balena-button/) button, which puts the puzzle onto the users device. They then have to find and solve the puzzle, and put the resulting code into the main website to tick it off. The range of puzzles we had here was amazing:

* turning the pi into an FM transmitter with a GPIO wire antenna, and using text-to-speech to broadcast the code
* having to learn and navigate the [balenaOS image partitions](https://www.balena.io/os/docs/architecture/#Image-Partition-Layout) to find otherwise hidden files
* using the [UART GPIO pins](https://www.balena.io/docs/learn/develop/hardware/i2c-and-spi/#using-uart-or-serial-on-raspberry-pi-3) to form a loopback
* cryptography puzzles
* balena trivia quiz puzzles
* morse code LED puzzles
.....etc

And we could also make puzzles that relied on additional hardware, since that just caused the team to work together to find someone with that hardware. I made a version of the [Nokia Snake game](https://www.itsnicethat.com/features/taneli-armanto-the-history-of-snake-design-legacies-230221) that used a [SenseHAT](https://www.raspberrypi.com/products/sense-hat/), which necessitated someone getting a high enough score to make it display the code. Not everyone has a SenseHAT, but I knew SOMEONE on the team would. :)

## Next part
In the next post I'll explain how I managed to deliver to build an open fleet, an API and puzzle frontend, 33 deploy with balena puzzles, 7 open fleet puzzles, all tested and proven working inside 5 weeks WITH NO DEDICATED RESOURCES. No orders, no hierarchy, just pure [gameB](https://www.game-b.org/) ways of working with others. :)