About

## Introduction

Mocap Fusion [VR] is an immersive roomscale mocap sandbox for artists and animators who wish to create and export motion capture animations, or create live content, using conventional VR hardware.

With as little as a single VR HMD and two controllers users may create mocap on their own avatars. Advanced users may create more detailed motion capture including full body tracking and can be combined with other sensors (eg. Apple IPhone Truedepth sensor and Oculus Quest 2 optical finger tracking) to connect avatars to simultaneous inputs.

This fusion of multiple sensor inputs can combine many layers of motion capture in a single take; including: full body tracking, face capture, lipsync, gaze tracking, optical finger tracking.

## Features

* Export mocap and create scenes in Blender and Unity instantly.
* Exporting .anim files for Unity.
* Live-Link plugin for Unreal Engine 4.26, 4.27 and 5 live production and live recording (includes face capture sync).
* Live-Link plugin for Unity Editor live production (includes face capture sync).
* Live-Link plugin for Blender live recording (includes face capture sync).
* HTC™ Vive Trackers full body tracking.
* Optional trackers supported: foot L, foot R, knee L, knee R, hip, chest, clavicle L, clavicle R, elbow L, elbow R, hand L, hand R and head trackers.
* Ability to record, playback, pause, slo-mo, scrub mocap in VR.
* Customizable IK profiles and avatar parameters.
* SteamVR Knuckles support for individual finger articulation.
* Quest 2 optical finger tracking app for individual finger articulation and finger separation.
* Vive Pro Eye blink and gaze tracking support.
* Sidekick IOS Face capture app (Truedepth markerless AR facial tracking).
* User customizable Worlds, Avatar and Props may be built for mocap using the APS\_SDK.
* Compatible with existing Unity3D™ avatars and environments.
* Supports custom shaders on mocap avatars.
* DynamicBone support for adding hair, clothing and body physics simulation to avatars.
* Respiration simulation and upper chest animation.
* Add/Record/Export VR Cameras for realistic camera mocap (eg. VR Cameraman effect).
* Place "streaming" cameras for livestreaming avatars to OBS or as desktop overlays.
* Use USB gamepad to control and position scene cameras.
* Microphone audio recording with lip-sync visemes and recordable jaw bone rotation.
* Storyboard mode, save mocap experiences as pages for replaying or editing later.
* Animatic video player, display stories and scripts, choreograph movement.
* Dual-handed weapon IK solvers for natural handling of carbines.
* Recordable VTOL platform for animating helicopter flight simulation (eg. news choppers).
* VR Camcorders and VR selfie cams may be rigidly linked to trackers.
* VR props and firearms may be rigidly linked to trackers.
* Ghost curves for visualizing the future locations of multiple avatars in a scene.

## Target Audience

* Creators, Artists and Animators
* Game developers
* Live streamers
* Educators, Presenters, Architectural visualizations (arch-viz)
* Many others

## Compatible Headsets (VR HMDs)

* Valve Index
* HTC Vive (and Vive Pro Eye).
* Oculus Quest (1 and 2).

## Optional Tracking Hardware

* SteamVR Vive trackers
* IPhone Truedepth sensor (facecap and eye tracking)
* Oculus Quest 2 (full optical finger tracking)

## Gameplay

The experience varies depending on the user's PC and the tracking hardware used. The recommended SteamVR headsets are the Valve Index or the HTC Vive. A Quest HMD may also produce reasonable results.

It is possible to use the software without an HMD (eg. when livestreaming). Full body tracking is only available when using feet and hip trackers (and optional elbows, knees, chest).

Users may achieve more realistic tracking results when using body trackers. Body trackers are optional and standing mocap is supported. Further realism my be achieved on compatible avatars by also enabling face capture or using a Vive Pro Eye for gaze and blink tracking.

## Desktop and VR mode

**Mocap Fusion VR** operates in two modes, VR and Desktop. If you have a headset you can use both modes at the same time, meaning you can operate the software thru your computer screen as well as in VR (If you don't mind taking off you headset from time to time. If you don't have a headset and use only trackers for mocap, then you can only use Desktop mode, also called Non-VR mode.

Most of the features are available in both modes, but a few are only available in either modes.

## VR and Non-VR mode

When launching the program and adding the command line argument -vrmode none you should see the HMD track button is grayed out in the landing screen, also SteamVR dashboard would indicate it's standing by and not show any game is running. But the vive trackers and controllers should still be visible in the game window.