We need to serve the Internet offline
By: R.J. Steinert Iii
Tags: Decentralization, Decentralized Web, Pouchdb, Couchdb, Offline First

We need to serve the Internet offline

Digitizing critical data collection for disconnected communities

Software that brings economic efficiencies increasingly depends on an Internet connection. Half of the world is “offline” [source] and the average person in a low income country pays a significant percentage of their yearly income for one gigabyte of mobile data [source]. Globally, Internet adoption growth rates are less than a decade ago because some parts of the world are reaching saturation levels [source]. It’s no longer enough to lower the cost of access to the Internet — we have to fundamentally change the Internet so that it works for people living in offline areas of the world.

Tangerine’s case for combining the Sneakernet and the Internet, presented at Offline Camp Oregon 2019

My story with developing apps

In 2011, I observed a paper-based data collection that assessed math and reading capabilities of students in a dozen schools near Accra, Ghana. In a single day, I saw an entire truck bed filled with paper. I later found out that the data took weeks to compile by hand. This was an area where digitizing data collections would reduce the costs and time involved.

In 2019, I’m one of many building an Open Source digital survey tool, called Tangerine at RTI. Tangerine has been used in over one million assessments and surveys in over 60 countries and in 100 languages. When Tangerine was first developed in 2011 by developer Mike McKay, it was an early offline-first app for Android tablets that used using Cordova and CouchDB on device with an online CouchDB server to sync to. Today, CouchDB on device has been replaced by PouchDB, which still syncs with an online CouchDB. It still uses Cordova to publish content to an installable APK file for Android, and now users also have the option to publish as a Progressive Web App (PWA).

During the lifetime of Tangerine, the type of project using the tool has evolved from one-off data collections to longitudinal studies that need to support the flow of downstream app/content updates and upstream flow of user-generated data from devices that may never be online.

Long-term projects need a way to update apps offline

Surveys that span multiple years allow project researchers to learn from the results, which can lead to new survey questions. For example, a long-term project using Tangerine is the USAID-funded Tusome project. The project employs Teacher Coaches who visit schools in Kenya monthly to provide in-classroom coaching for teachers. The Teacher Coaches use Tangerine forms to track their observations and the app suggests feedback that the Coach can give the Teacher during the visit. In 2017, the Tusome project employed 1,180 Teacher Coaches who visited around 16,000 of the 24,000 primary schools in Kenya.

About twice a year, new questions are added to the Tangerine forms, algorithms for feedback are refined, and the Teacher Coaches receive training on the changes. To download the app updates on their tablet, the Coaches tap the “Update App” option in the main menu and the app downloads and installs the relevant updates over the Internet. Usually about 10% of Coaches are successful updating the app on their own before the training, while the remaining 90% update the app during the training.

Because training facilities in Kenya often lack adequate Internet connection to update the tablets, project staff find rooms in nearby hotels with a strong enough mobile data connection to conduct the update. Meanwhile, the 25–30 Coaches at the training learn about the updates with only a handful of tablets. If Tangerine were able to update offline, the tablets could be updated at the training facilities without delays and without needing additional IT staff to facilitate an offsite update.

Long-term projects need a way to sync data with offline devices

Some long-term projects involve facilities with devices that will always be offline yet still need the right data, at the right time, in the right place. In the upcoming Sante Mali project, funded by the Gates Foundation, medical professionals at offline facilities in Mali will use Tangerine to track participant data in a two-year drug trial.

Sante Mali is “a double blind randomized controlled trial assessing the efficacy of oral azithromycin administered to pregnant women and/or infants during routine care in preventing stillbirths and mortality through 6–12 months of age in Mali, West Africa, where rates of infant and under five mortality are among the highest in the world.” [source]

The study will track 50,000 participants across 80 facilities, with an average of three Android tablets with Tangerine installed per facility. Only 30% of facilities will have an adequate mobile data connection to be able to sync data with the online server. Tangerine’s current client-server sync model is a challenge for the program’s needs.

Centrally located data managers need to continually review the data to maintain the efficacy of the study. In some long-term projects with always-offline devices, project staff collect the devices and bring them to a central office with a strong internet connection. But in the Sante Mali project, the tablets with the participant data need to be at the facility at all times, ready for when a participant arrives for an additional visit. Another challenge is that a facility may need another facility’s data because pregnant women in Mali typically move to another town during their pregnancy. If a participant arrives at a facility that doesn’t have their records, staff can’t use valuable contextual information to aid in administering the program. This requires us to find a solution to sync data between devices to avoid interruption of data availability.

Looking ahead — a world where there is no “offline” and “online”

Traditional offline-first approaches supported by current web standards, such as Progressive Web Apps and in-browser databases like PouchDB, can be transformational in their ability to allow offline data collection in an occasionally connected environment. Tangerine has used these features to their full advantage and seen vast improvements in the efficiency of data collection for the projects we support. However, it’s important to note that these tools require a device to be online for first page load or app download, as well as when syncing updates to a central database shared by other devices. This means that users in fully disconnected environments still aren’t adequately served.

Decentralized and peer-to-peer (P2P) Web technologies are beginning to address this gap. Technologies such as IPFS, Dat/Beaker Browser, and Scuttlebutt show the world how the Internet can move past the binary labels of offline/online to a world where you are simply connected to more or fewer devices directly, rather than each device needing to connect to a centralized resource. Someday P2P technologies will enable tablets at a training facility for Teacher Coaches to share the app update without requiring each tablet to download the update from an online server. Using P2P technologies at an offline medical facility, a traveling project employee with a single tablet could sync data between facilities without any of the tablets ever being “online.”

The Internet is evolving and Tangerine will evolve with it, thanks to contributions from donors (such as USAID, the Gates Foundation, and Google.org) and to the RTI International staff who use Tangerine every day. For more details on how the Tangerine team and other Offline Camp attendees are approaching the offline/online problem, stay tuned for my summary of the “How would you build an offline chat app?” session from Offline Camp Oregon 2019.

—

Thanks for reading! Follow me on Twitter at @rjsteinert and the Tangerine project at @tangerinetool.