Exploring Augmented Reality to fight distractions and improve productivity
By: Sahil Faraz
Tags: Eye Tracking, Innovation, Virtual Reality, Augmented Reality, University Of Toronto

Does that imply technology is harmful?

No. While excessive use of technology has a drug-like effect, let’s not forget the contributions of technology in our daily lives ever since the internet became widespread. Dropping a clichéd quote here as a reminder :

Here are a few workplace statistics that strengthened my decision to look for a solution using technology given the nature and criticality of the issue :

Humans need tools. Humans need and want augmentation. And as the saying goes ‘First we make our tools, then our tools form us.’

- Paul Jones

Real problems faced by real people

I reached out to many people on my social media who regularly work with computers, inquiring about the kind of productivity-related challenges they face; and these were the problems they talked about :

Factors affecting productivity

Why address these problems with AR?

AR is thriving in the education sector and expected to grow even faster in the next five years. With a growing library of 3D objects, AR is already becoming the next big thing in gamifying experiences to impart tacit knowledge even outside of studies.

There is no denying that we will only see increasing use case of AR glasses at work and in schools. It makes complete sense to incorporate the concept of digital well-being early on to tackle these issues.

Defining the scope

I have been following the developments in this space for a few years now. However, I still had to spend countless hours skimming through recent research papers concerning productivity, distractions & digital well-being. I paid particular attention to understanding why AR is the best solution for the implementation I will be discussing further in this article.

I did the generative research by discussing all these productivity-related problems with AR experts, designers & engineers at length. They suggested some cutting-edge technologies that can be leveraged to build AR experiences, which are otherwise not possible in the real world.

Based on all the knowledge I gained, I started exploring the solutions to these themes using augmented reality :

Themes I wish to look into with Augmented Reality

Focussing on the Solution

1 in 3 Americans have taken steps to improve their digital wellbeing in the past year, and more than 80% of them said this had a positive impact on their overall sense of wellbeing.

With that in mind, we shall explore some ‘global’ well-being solutions based on AR, which doesn’t just help us with technological distractions but can boost productivity in our daily lives as well :

(Unless stated otherwise, the AR implementations discussed further are based on AR glasses or HMDs and not smartphones)

The job-to-be-done by this design would be to ‘allow the user to focus on the primary task.’

1. InfoBoxes instead of unnecessary tabs

“Media multitasking is becoming more prevalent in our lives today, and there is increasing concern about its impacts on our cognition and social-emotional well-being”

Seeing this screen while studying made me realize that something is going wrong with the way I am consuming content

As per my primary research, even though having these tabs might expose you to more knowledge, the likelihood of losing focus on the immediate task increases with each new tab. This results in delays and stress due to the information overload while trying to multitask.

The Implementation

Imagine this hover UI working literally everywhere you see! Of course, you can take control

I absolutely love how Wikipedia has this hover interaction which gives the users a bite-sized ‘infobox’ instead of compelling them to swim in the sea of URLs — leading to this :

Even Google mocks us for having 100 tabs in our backlog

TThe challenge is; not every website has that infobox implementation. But, something else can be used that would work globally and is not confined to just Wikipedia or even a screen for that matter.

Using text & image recognition techniques, we can show a “knowledge panel”(as Google likes to call it) for any perceivable object around us. One can probably relate how this UI has prevented us from going through multiple links countless times.

Google has over 500 billion facts about five billion entities for these knowledge panels

Even though this can not be currently implemented outside of the Google system. But, if Google can do that — it’s possible! This modular design can be made by transcluding an infobox into an existing document, in this case, into the real world! This can be done by specifying some or all of the attribute-value pairs associated with that infobox, known as parameterization.

An example of how easy it is to make a Wikipedia infobox :

{{Infobox prepared food

| name = Crostata

| image = Crostata limone e zenzero 3.jpg

| imagesize =

| caption = Crostata with lemon ginger filling

| alternate_name =

| country = [[Italy]]

| region = [[Lombardia]]

| creator =

| course = [[Dessert]]

| type = [[Tart]]

| served =

| main_ingredient = Pastry crust, [[jam]] or [[ricotta]], fruit

| variations = ''Crostata di frutta'', ''crostata di ricotta'', many other sweet or savoury variations

| calories =

| other =

}}

Note that values are to the right of the equals sign (=), and that the parameter names are the same as those in the specification for the infobox template. The values are in wiki markup: entries enclosed in square brackets (e.g. [[Tart]]) will be rendered as a link to the respective Wikipedia article (e.g. Tart), and the linked file will be transcluded into the article in place of its markup.

I am not being paid to advertise, but please donate to Wikipedia to support them in doing what they do best :D

Let’s go back to the interaction and see how to fit this interface intuitively in AR Glasses/HMD.

By implementing Microsoft’s ‘hand rays’ interaction, the user will interact with items such as texts, real-world objects as long as they are clearly identified by image recognition.

The pointer identifies the item to be searched and shows a spinner while it fetches the result

The infobox will be displayed as an overlay till the time the user holds their hand in front of the sensors placed on the AR Glasses/HMD. As soon as they put their hands back, the infobox would disappear. This interaction is perfectly suited for short periods.

Keep in mind the infobox image is formed on the glass and not on the monitor display, so it needs to be ensured that it’s big enough in size with configurable transparency & appearance to avoid accidents.

A decent-sized overlay is displayed for as long as the sensor on the AR Glass/HMD detects a hand ray gesture

In case the user wants to revisit the information in more detail on another device such as the PC or smartphone, they can choose to bookmark the item to the linked account using a voice command or a different hand gesture.

In case of overlap with some higher priority interaction, the hand ray gesture could only be enabled after performing a different hand gesture, which would then use it to display infobox overlays only.

The most tricky part of the implementation would be hand gestures. But considering the support for this tech on multiple AR glasses & HMDs, this is relatively feasible and will gain more support in the next few years. The downside of this tech is that the users will consume bite-sized information more conveniently than ever. But that can be fixed by implementing digital well-being concepts by allowing them to set daily learning limits and reminders from the companion app settings.