ConVRsation : An Exploration in Immersive Communication Design
By: Hugues Vincey
Tags: Design Process, Virtual Reality, Hackathons, Conversational UI, Unity3d

In early december 2020 I participated in a particular type of hackathon : a designathon

Called Interhackt, organized by https://twitter.com/azlenelza & other designers, the objective during a week was to explore and flesh out new design concepts around the theme “Making & Understanding Together”

Interested in working on this theme through the prism of VR with Unity, I was lucky to find another Unity dev, https://twitter.com/lukasdonkers, to jam with.

In summary, this was a very rewarding VR project that ended in us winning the first prize in the hackathon among 30+ other interesting projects.

Personally I learned and progressed a lot, notably through:

Organizing brainstorming & collaboration with the Miro whiteboard software

Implementing a VR player in Unity using the XR Interactions Toolkit SDK with an Oculus Quest headset

Learning the Normecore Networking SDK and implementing multiplayer code in C#

Writing an in depth presentation for the project submission to communicate design ideas & vision

Here’s the full story of what we worked through during that week and what came out of it.

Initial brainstorming & tests

Through an initial brainstorming in Miro, we quickly decided on creating a collaborative VR Mind Mapping tool & to find a use case from there.

By creating and connecting nodes in a 3D space, you’d have great freedom over the layout and structure.

We found our specific graph use case by plugging in the Normcore networking solution and meeting in VR as friendly avatars.

https://normcore.io/documentation/assets/media/working-avatar.3d41d6dd.mp4

By simply having a conversation with embodied 3D characters, we realized that VR as a conversational medium was an excellent thread to follow, hence “ConVRsation”.

One of the most powerful aspects of interacting and collaborating in VR is the Sense of Presence.

Just being able to see where someone is looking, and seeing their body language and gestures, you can :

Engage better with each other

Communicate more fluidly & naturally

See the focus of each participant & collaborate better

Making in-person conversations tangible

We then decided to explore how the flow & items of conversation could also be a physical presence between the two participants.

We imagined that throughout the conversations, nodes representing ideas & thoughts would emerge, connected through a thread.

They would have the ability to revisit that same conversation in the same way you would revisit a physical place.

You could even find ways to branch out and explore alternate paths from that conversation.

Tools for making the mapping process seamless

In VR, using a keyboard ends up being very cumbersome. For that reason, we decided to have all text input through voice transcription. We implemented Google Speech to Text which worked perfectly.

Next we decided to give to the users a palette of core sentence types :

Subject : To set the focus of the conversation, working as a visual anchor in the scene.

: To set the focus of the conversation, working as a visual anchor in the scene. Idea : To communicate ideas in the most general sense

: To communicate ideas in the most general sense Question : To prompt a response in the next connected node

: To prompt a response in the next connected node Pros : To show arguments in favor of a point

: To show arguments in favor of a point Cons: To show opposition to a point

These basic building blocks can be used to categorize all elements of conversation. The overall result was easy to navigate at a glance through color coding.

To reduce friction in adding content, we made the nodes available at the waist level, so you could grab and place them easily. This quick process allowed users to maintain a pace close to normal spoken conversation.