Beta - Help - MF General notes

## What is APS Luxor?

APS Luxor is a SteamVR game that can be used for live production or to record avatars and export mocap as blender friendly files, in the generated export folder there is a sceneloader.blend file that can be used to imports the mocap compellation into a Blender scene. But there are also export options for Unity, and live link features for Unit, Unreal and Blender..

It's a tool for animators to record full body motion capture w/ facecap using consumer VR hardware and was initially designed to allow quickly exporting mocap and avatars to Blender for final render. However there has been a lot of community feedback and many new features have been added. I'm not sure how to summarize exactly what Luxor is, it's a customizable multi-purpose motion capture tool focused around VR, a vtuber tool, a live-production tool, a VR training simulator tool, a choreography and review tool, a storyboarding tool. I'm also hoping to someday add online multiplayer and a minigames sandbox to improve the interactive features.

## First steps notes

APS is designed to record mocap data at 60fps using SteamVR as input and is capable of exporting mocap, prop and scene data to Blender friendly files. And by using a sceneloader.blend (the sceneloader.blend is auto generated by APS whenever exporting by the Blend builder) you can import the scene compilation into a Blender scene using one click Python automation script.

Here is a Youtube video showing some basic mocap recording using a HTC Vive and some controllers. And how to build Blender scenes automatically: https://youtu.be/ykJ7O0Bs8oQ?t=1351

## Ive been wondering, what are the filetypes for export using this tool?

Mocap data is exported as standard .bvh for each character in the scene to a destination folder, in that folder contains a sceneloader.blend.

The sceneloader.blend is a blender scene which contains an automation script that loads the compilation of mocap data and characters models into a single Blender scene, from there the animation data can be touched up by hand and then exported from Blender to any of its supported formats including .fbx format.

Here's a short video which demonstrates recording mocap with APS, then exporting and loading it into Blender (via the sceneloader.blend), and from Blender to a single .fbx https://www.youtube.com/watch?v=SyFvBqTX6RA

## Exporting a scene

The entire mocap skeleton is exported as a single .bvh and the sceneloader copies the keyframes to the .fbx (or blender model). From there you could extract the hands and head keyframes and discard the rest. But I can't really think of a way to export only tracker data as individual bones or empties in the current build. I'll give it some thought.

## HAX, where is avatar configuration stored?

Can we also get a feature where we can save our edits to the blendshapes menu into a file and upload those settings to other characters? So far I've had to do a few fixes of my avatar, but every time i rexport i have to redo all of it's profile and blendshape and offset settings in Luxor and it's driving me a little crazy.

I think you could make a copy of the character.cfg that gets created in avatar folder in VR\_MocapAssets.

"I did but the blendshapes don't seem to have copied" In facecap.csg file

## More on recordings

Check out the "Recordings" tab where you can quick-save the current avatar/mocap for adding it back to the scene. If any quick-save "Recordings" exist in the list try clicking "Add" to place the quick-saved avatar into the scene.

Once a quick-saved avatar(s) has been added the player avatar can interact with them in a couple of ways:
- By aiming the UI laser at any quick-save avatar will display a overhead menu options ("interact physics" or "ghost curves").
- The main avatar can grab the hand of any quick-save avatar.
- If a quick-save avatar is shot by a firearm prop it will enable ragdoll physics.. Also check out "Storyboards" which can be used for saving entire scenes (multiple avatars and props) for later review, export or editing, or creating story sequences..

After creating a new story folder and saving the scene as a page click the numbered "page" icon which should load the saved scene. Pressing stop while a story is playing will end the storyboard but, the animations and avatars should remain in the scene allowing adding/editing and saving edits as a new page.

## About error logs

I managed to add a new error log for the APS live-link server and hopefully can help with debugging connection issue. You should see a new \Error\_Logs folder located in the LUXOR game data folder.

The APS live-link server generates the new log file (LOG\_APS\_UE\_LIVE\_LINK.txt) in that directory whenever it initializes and should give a full picture of what the thread was doing at the time the client connected or disconnected, and logs any exceptions that might have been thrown.

You can find the new log file located in the folder: %USERPROFILE%\AppData\LocalLow\Animation Prep Studios\LUXOR\Error\_Logs\ Also I have added a new \Configs folder where a .JSON file lives that allows users to change some of the (low level) game parameters such as the live-link port number. Possibly by trying different port numbers may resolve some connection issues.

Hope this will be helpful!

## Recording MF windows

"I wanna make a tutorial for how I use LUXOR, but I don't know how to record the program. Is there a way to mirror the headset view in MF so you can see what the headset sees instead of just the default view?"

Use FPV cam in Luxor or display VR view from SteamVR dashboard.

## About MF world origin

How do you make 100% sure your animation starts in the middle? is the only method to roomscale and stand right in the middle?

Yep, everything is relative to the playspace origin.

## Some more general concepts

"Is there any tutorial that covers the absolute basics on putting the files from Luxor into blender and changing the avatar to a custom one. Say if I recorded the motion capture and wanted to become a king or something."

The inspiration behind APS was to design a tool that allows using custom avatars so that the avatar is used throughout the entire workflow, this avoids the need to do any retargeting in the final scene (which produces much cleaner results and practically no retargeting offsets).

But you can still retgarget if you'd like. However it is recommended to build a custom avatar (or base character) for recording the motion capture in APS, then you can export the scene from APS and the SceneLoader does most of the work importing and building the exported scene completions into Blender in a single click.

If you already have a custom model you'd like to use, a king for example, you can build the king as a custom avatar by using a .fbx of the model, just drop the .fbx into a Unity project, add the APS\_SDK, and follow tutorial here: (add tutorial)

Or if all you have is a mesh of a king you could convert the model to a Mixamo rig using mixamo's online tool and then build the result as a custom mocap avatar for APS.

It is recommended to use a mocap avatar with the same skeleton as the final, the default avatars were added only for demonstration and basic getting started but were not intended to be used for retargeting in the final scene; though it is completely possible and retargeting does work, it is just way extra work and produces anomalous results just as would any retargeting.

Best example I can make is if you have a custom scene with a light switch that you want the avatar to approach and flick with its finger, then if you use one of the default avatars to record and retarget a king to the resulting mocap, the king's finger might have slight offset making it appear not to make contact with the actual switch, but if you use the king avatar in APS to record, then the result in Blender will be exactly what you see in VR down to float 32 precision (and no need for retargeting).

Also here's a basic getting started tutorial that shows how to record and export mocap to Blender and build the scene: https://youtu.be/XdlFtt\_F-3Y - however it shows the old holographic UI which has changed in recent versions, but is still relevant.

## Clearing MF local files

"Weird question, do you like to clear your locallow out between versions of APS or do you just leave it? ive been leaving it thusfar myself but wonder if that might cause issue"

It's okay to delete the entire contents of **%USERPROFILE%\AppData\LocalLow\Animation Prep Studios\LUXOR\** APS will just regenerate it when launched, however be sure to backup any custom avatars in your VR\_MocapAssets or any export data in the Blender\_Scenes folder before.

## Files created when exporting scene

When APS exports a scene and mocap it generates a folder which has a sceneloader.blende and all the other mocap scene data. Check out this video at (2:18) when the scene is exported a new folder appears, then open sceneloader.blend and run the script to import the mocap and customavatar: https://youtu.be/us5zREdEADg?t=138

## Lip syncing and Unreal

"Does anyone know if there Is there a way to do lip syncing from an audio file using an exported animation from luxor within unreal?"

If the avatar has lipsync blendshapes and a VisemesLinker was added when building the avatar, then it should be possible to record mocap in APS with the lipsync enabled and then export the animation to Blender, run the sceneloader to import the skeletal animation and lipsync together on the same character. Then export the character from Blender as a .fbx and somehow use it in Unreal as an animation, which could be synced with the original or post recorded audio. Not sure but even try live recording in UE using the APS live-link plugin. The plugin is still in beta but it can be used to live record mocap, just can't remember if it also records the blendshapes.