Zero to Scavenger Hunt: An Interactive Experience at the California Academy of Sciences
Tags: typescript, node, react, webdev
Published: 2022-09-12 15:32:36

> 
How the MADE used interactive web documents and a museum's adult nightlife event to prototype interactive location-based exhibits.

A few weeks ago the the [Museum of Digital Art and Entertainment](https://medium.com/r/?url=http%3A%2F%2Fwww.themade.org) came out to the [California Academy of Sciences](https://medium.com/r/?url=https%3A%2F%2Fwww.calacademy.org) to show off some video game history and give visitors a peek into elements of how video games are created.  We are a non-profit 501c(3)  video game museum in Oakland, CA at 921 Washington Street (please come visit!). We work to inspire the next generation of digital creators through preservation, restoration, and education. 
Video games are an interactive medium: We strive to have a completely playable collection. So when it comes to a scavenger hunt, we wanted to push ourselves to make it an interactive, fun, and, most importantly, educational activity. Through this exercise, we learned a lot about what works and what doesn't - we hope digital creators involved in this space learn a lot from our execution and we start seeing more interactive, spatial experiences like this one!

## Ideation
Our approach to our interactive exhibit actually started off with some internal ideation about publicly accessible interactive exhibits. It's not a stretch to assume that smart phones with touch screens and an Internet browser are ubiquitous these days. If we begin with this assumption, we can assume that this phone can be treated as a peripheral to a physical exhibit that is placed behind glass, or that it can be a minimal setup that requires nothing more than a screen.

![An exhibit behind glass displays a QR code screen encouraging users to scan it.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lj2x00shwjsmrqsl6swt.png)

This phone, in turn, can be used as a game controller to one of our game console exhibits. Our exhibits can be kept on at all times of the day, allowing folks to engage in video game history even when we're closed.

![Once scanned, the QR code sends users to a website that is a controller for the exhibit.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cf9updpbrt3x0wb7zq1o.png)

We submitted this as a proposal for a few grants, but didn't get any takers. Without any funding, we've had to put this on the backburner while we prioritized reopening. When the California Academy of Sciences invited us to be a part of Nightlife, we were thrilled to spin this idea in a different direction.

## The Project
![General flow of our user story](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ut2wy87fvdc0cyf1acpl.png)
To start, we needed an overarching user story to make sure we have a beginning and end to the user experience. It also gives everyone working on the project a working starting point. Using this document, we started divvying up each portion of the experience to different folks. Our amazing designer, Steven, would create the signage for the event, I worked on the general flow, and two of our technical volunteers helped create the three constituent experiences.
Instead of requiring any secondary screens, we decided to make the user's mobile device the core of the educational experience instead of it being just a peripheral. The user would scan QR codes to solve puzzles directly on their phone, and their progress would be stored within the browser's local storage.

As we started working, we focused on three core principals:

1. Develop a quick iteration loop. We deploy to static pages using nodejs, Typescript, and React, and use the provided static-page server setup to validate changes to the site.
2. Run to the finish line, and then fill in the gaps. Deliver something as complete as possible, and use user testing to focus effort on the most important aspects.
3. Focus on human parallelization. Using React to create modular widgets allowed each technical volunteer to work separately on their own individual section.

## Design
Once we had the overarching user flow in mind, it was time to develop each of the individual educational experiences. Once again, we drafted out design documents to get everyone on the same page while simultaneously outlining the user experience.

### Geometry
![Geometry user flow](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4goq03dpq1y10xx2lf3i.png)

### Shaders
![Shader user flow](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/iy1nttuob55fntotzl8l.png)

### State Machines
![State Machines User Flow](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/evgbad105fl4d0zdpd1y.png)

## Implementation
With the rough design work done, it was time to work on implementation. We had four weeks to finish this with only volunteer time, so we had to focus on efficiency. We would program as little as possible, using libraries that have great cross-device support and treat content as much like a document as possible.
As a result, we ended up using three.js for 3D experiences and processing for 2D experiences. React widgets came to the rescue, enabling highly parallel development, since our widget logic would generally not collide when merging code. Finally, instead of using a database to keep track of user sessions, we used the localStorage browser API instead. By scoping out the boundary of the experience, we did as little engineering as possible to get what we needed done.
As we started to build each experience, we integrated user testing as our primary form of testing. To find bugs, we sent links to each challenge to our internal volunteer Slack channel. If you choose this workflow, it's critically important that you work quickly. Your testers need to see their input reflected, or you'll quickly lose momentum among them. And, without that momentum, you will lose out on valuable feedback. Used correctly, feedback will drastically reduce your workload by showing you where you should work to most effectively refine your experience.
Yes, we know, we could automate more testing! However, we were able to sidestep many of these issues by aiming  to build as small of a program as possible to prevent needing tests to begin with. Once we solidify our work into a system, we'll absolutely look into integrating unit tests into our most reused code.

## Deployment
Over the past two years, we've been using the GitHub actions workflow for existing static site deployment, which connects to Azure as our host. We continued to use this process for Nightlife. When one of our volunteers finished their work on a portion of the experience, they would open a PR. If everyone was cool with the changes, then we merged the PR triggering our Github actions, which pushed our new static react pages up to our CDN where they are served.
For our metrics, we used Google analytics. Their live view allowed us to see engagement throughout the night, which ended up being really useful. In the future, we could even look into running async Javascript HTTP requests at dummy URLs to identify key milestones in intermediary stages of experiences. This would allow us to pinpoint any blockers people may be hitting and how prevalent they are in real-time.
Finally, with this being a scavenger hunt, we needed a hook that would tie the physical world to the virtual one. To do this, we needed to create physical QR codes. These were designed by our amazing volunteer designer, Steven, printed by the folks at the California Academy of Sciences, and then mounted in the locations corresponding to the scavenger hunt map.

## Results
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zyifqeu9zhnhegyuzm9h.jpg)
All of this would mean nothing without showing you the results. Here they are! 
Scavenger Hunt Map: [https://www.themade.org/nightlife/exhibit_map](https://www.themade.org/nightlife/exhibit_map)
Geometry Exercise: [(https://www.themade.org/nightlife/v4bwb8uest)](https://www.themade.org/nightlife/v4bwb8uest)
Shader Exercise: https://www.themade.org/nightlife/o3eou2nssd
State Machine Exercise: https://www.themade.org/nightlife/mb761gb8wz
Reset Exhibits: https://www.themade.org/nightlife/reset_exhibit

## Practices that Worked
With any interactive experience, it's crucial that developers create debugging workflow as soon as possible. Things like creating a webpage to reset local storage state enabled us to quickly debug the overarching scavenger hunt completion flow. Having QR website codes on-hand while developing the experience let us quickly jump between exercises without having to copy/paste all the time.
Leveraging existing libraries paid off a ton. React, three.js, and p5.js allowed us to focus on the experience creation instead of plumbing. And the local storage browser API let us simulate a persistent user session without the need for a database running on a server.

## Areas of Improvement
As mentioned before, we'd like more automated tests, particularly for components that are system related. By going back to the beginning and writing tests, we could standardize things like local storage of session data, unifying our code and making it much more maintainable.
We also need to get more testers, and have them test early. Even with four weeks, with only volunteer time, we didn't have nearly enough iteration loops for the quality that we wanted to deliver.
Finally, we think it would be a good idea to really spend time thinking about what core pieces of code we need for interactive elements on our website. Having even somewhat relevant code is better than none - with a bit more effort solidifying our foundations, we can ensure that educational interactive exhibits get better and better.

## Closing Thoughts
If you got this far, thanks for reading! As I mentioned before, we're a 501c(3) that is run almost entirely by volunteer efforts. We believe that access to playable, interactive digital history should not be the domain of only rich collectors, and that an academic institution that seeks to preserve and expand access to this work can go a long way in bridging the digital divide. We have a physical space at 921 Washington Street in Oakland, CA where you can come play video games and relive the past of digital entertainment. If you're interested in being a volunteer, shoot an email off to [info@themade.org](mailto:info@themade.org). And finally if you want to donate to support the work we do, you can do so here: [https://www.themade.org/donations](https://www.themade.org/donations).