‘Contextualization Engines’ can fight misinformation without censorship
By: Aviv Ovadya
Tags: Social Media, Messaging, Misinformation, Censorship, Machine Learning

What a contextualization engine might look like in practice

Imagine that you were forwarded a terrifying message in a group chat. Or saw a post shared on Facebook which made you furious at some news organization. But something seems a tiny bit fishy…

Option A: Without a contextualization engine

While you would like to know if the claims are really true — and you may “want” to look it up…you just don’t have time for that sort of thing. It’s easier to just go with the flow. It’s also a giant pain to copy and paste things or type out many search terms trying to figure out if someone else is just confused — especially on a phone. So you don’t check.

It remains in your memory, as something perhaps true — but you may forget the ‘perhaps’ with time. If you see enough similar messages, maybe you start to instinctively believe them — and then you may start sharing those messages also.

Option B: With a (very basic) contextualization engine

You see something that looks fishy — and tap a button to ‘contextify’ it…

The contextualization engine compares the content being shared with that from authoritative sources and provides articles or other media results that are sufficiently related. This might be in a search result style interface, though a chatbot, or a hybrid. If it finds no close enough matches, it warns the user and potentially identifies the most likely relevant keywords that the user can run a more traditional search with if they would like (with another tap). It adds the media object to a triage queue for relevant organizations to potentially evaluate (e.g. fact-checkers).

The ‘contextify button’ is a drop-in for WhatsApp’s magnifying glass feature — but the results are very different. WhatsApp creates a keyword search for a traditional search engine, which can backfire badly due to data voids.

This magnifying glass feature on WhatsApp was a valuable step forward. But it doesn’t currently work in practice in many cases. It makes it easier to look up messages on Google, but keyword search doesn’t work for long messages, images, videos, audio, with data voids, etc. We need more tools designed for contextualization.

Why even the basic contextualization engine helps

Key Insights: Unlike a Google keyword search, ‘contextifying’ does several crucial things:

Analyzes complete ‘media objects’ — to see how likely they are to be related to one another; e.g. the entire chain message, entire fact-check articles.

Focuses on authoritative sources — likely initially using whitelist certification through recognized 3rd parties such as the International Fact-Checking Network (IFCN), First Draft, News Guard, standards organizations, etc.

Warns about data voids — lets the user know if the system can’t find good information on the topic.

Supports the people doing deeper investigations — provides the human fact-checkers and other organizations with information about what is important to explore — and potentially revenue from web traffic in ways that are directly aligned with the users’ goals.

These all support the “Find better coverage” component of the SIFT media literacy method — an approach developed by Michael Caulfield, inspired by research at Stanford, and taught by many educational and civic organizations, from the University of Washington in the US, to Civix across Canada.

Contextualization systems can be even more helpful

This is just the beginning of the potential for contextualization engines and interfaces. A contextualization system might also support the remainder of the SIFT method:

S top ( S IFT): The contextualization engine flow can provide educational support for executing other aspects of media literacy. For example, it can help remind users to pause and notice their emotional reactions to the content. It might even provide tips on how to bring up the potential misinformation in a delicate way in a group chat or comment thread.

top ( IFT): The contextualization engine flow can provide educational support for executing other aspects of media literacy. For example, it can help remind users to pause and notice their emotional reactions to the content. It might even provide tips on how to bring up the potential misinformation in a delicate way in a group chat or comment thread. I nvestigate the source (S I FT): If the contextualization system already has information on why a source might be considered authoritative, it can provide that information to the user — showing why they might trust it (e.g. this source is certified by IFCN).

nvestigate the source (S FT): If the contextualization system already has information on why a source might be considered authoritative, it can provide that information to the user — showing why they might trust it (e.g. this source is certified by IFCN). F ind better coverage (SI F T): Building on the ‘analyze’ component described earlier, a more fully featured contextualization engine would not only auto-generate audio and video transcripts from media, but also automatically interpret any imagery and captions in order to better understand the content and find contextually relevant sources.

ind better coverage (SI T): Building on the ‘analyze’ component described earlier, a more fully featured contextualization engine would not only auto-generate audio and video transcripts from media, but also automatically interpret any imagery and captions in order to better understand the content and find contextually relevant sources. Trace claims, quotes, and media to the original context (SIFT): Finally, the contextualization engine can do the tracing for the user. It can essentially scour the web for the original context of any content.

None of this requires any new technology — this isn’t science fiction — though it is only recently that this sort of analysis has become effective and practically feasible. Some aspects of these suggestions have started being integrated in small ways into existing platforms, for example by excerpting contextual snippets from Wikipedia, but contextualization still does not appear to be their focus.

The potential — and risks — of artificial intelligence advances

While recent advances in artificial intelligence make a ‘contextify button’ possible, imminent advances will also make contextualization systems critically important to address threats to democracy and financial systems. Deepfake videos, incredibly effective AI-optimized phishing attacks, and automated troll armies may become pervasive — and indistinguishable from the real thing by an ordinary person.

Thankfully, the same technology that is creating these threats — powerful new language understanding and generation systems — can also help support contextualization to counter them. These AI advances will enable software to directly answer those key questions for users: “What does this mean? How does it relate to the things I know about and care about?” These powerful language systems can be used to help translate jargon — e.g. from scientific papers and legal documents — into writing and images that everyday people can understand and apply. AI advances are even enabling the creation of systems that could automatically integrate content from multiple authoritative sources to generate helpful mini-essays and engaging animated videos (this could be technically possible within the year — with very significant investment).

Such systems will need to navigate a challenging terrain of bias, information quality, misuse, and privacy, especially as they extend beyond the domains of authoritative sources. We must fund research and responsibility infrastructure to ensure that this revolutionary potential is applied wisely — while maintaining a bias for action given the clear negative impacts of moving too slowly.

How can we make this happen?

A “contextify button” to push media to a contextualization engine could be built into everything — just a normal and expected part of the interfaces for viewing and sharing content.

But such systems do not quite exist yet — they face a chicken-and-egg problem where it is challenging to get traction unless existing platforms buy into them, but platforms will not adopt them until the contextualization systems have traction. Funders and investors know this, and so it is difficult to raise the funds to hire the necessary talent. This has left us many years behind where we need to be given current and emerging threats.

Recent developments, such as Meedan’s work with WhatsApp to develop chatbots for fact-checking and contextualization are a valuable step in the right direction — but platform integration and funding for such work pales in comparison with platform integration and funding for systems that (often unintentionally) facilitate deception. To accelerate the development of contextualization systems, policymakers may need to create usage mandates and provide rapidly-deployable public sector funding — ideally including dedicated funding for responsible deployment.

The first two decades of the millennium were dominated by search and recommendation engines, bringing Google, Facebook, and Amazon to prominence. We now have a chance to innovate — building the contextualization engines that could define this third decade of the millennium — and perhaps help address the harms of the blundering tech giants.