Patent ID: 11948260
Assignee: LUCASFILM ENTERTAINMENT COMPANY LTD
Field: Computer technology (Electrical engineering)
Classification: CPC G  A  H | IPC A  G  H

Claim 0:
1. A method comprising:
receiving a motion or position of a first real-world performer in a first real-world environment;
moving or positioning a virtual representation of the first real-world performer in a 3-D virtual environment based on the motion or position of the first real-world performer;
rendering a 2-D video stream of the virtual representation of the first real-world performer in the 3-D virtual environment;
adding or altering a virtual asset in a 3-D virtual environment in response to the motion or position of the first real-world performer;
determining a mathematical 3-D transform to be applied to the virtual asset based on identifying the motion or position;
preparing a stream of real-time communication packets, wherein each communication packet in the stream of real-time communication packets comprises a field configured to represent the mathematical 3-D transform to be applied to the virtual asset in the 3-D virtual environment; and
transmitting the stream of real-time communication packets to a plurality of display devices distributed within a second real-world environment; and
causing a real-time video stream to be visible on a display device of a handheld or wearable device in the plurality of display devices within the second real-world environment, wherein:
the handheld or wearable device captures the real-time view of a second real-world environment comprising view of a second real-world performer;
the real-time video stream comprises the 2-D video stream of the virtual representation of the first real-world performer composited with the real-time view of the second real-world environment comprising a view of the second real-world performer; and
the motion or position of the first real-world performer in the first real-world environment moves or positions the virtual representation of the first real-world performer in the 3-D virtual environment in real time as the 2-D video stream is composited with the real-time view of the second real-world environment such that the second real-world performer and the virtual representation of the first real-world performer appear to interact in real time with each other when viewed on the display device of the handheld or wearable device.