Patent Description:
The present disclosure relates to a data processing method, and a computer readable storage medium.

When content creators are shooting contents for virtual production (e.g., the production of virtual reality contents) with cameras, it is crucial to accurately track the camera and/or other to-be-tracked objects. Conventionally, the content creators and/or the related staff need to spend hours on complicated installations of the tracking devices for the cameras, which slows down the progress and increases the difficulty of creating the virtual productions. Attention is drawn to <CIT> describing a 3D camera tracking and live compositing system which includes software and hardware integration and allows users to create, in conjunction with existing programs, live composite video. A video camera, a tracking sensor, encoder, a composite monitor, and a software engine and plugin receive video and data from and integrate it with existing programs to generate real time composite video. The composite feed can be viewed and manipulated by users while filming. Features include 3D masking, depth layering, teleporting, axis locking, motion scaling, and freeze tracking. A storyboarding archive can be used to quickly load scenes with the location, lighting setups, lens profiles and other settings associated with a saved a photo. The video camera's movements can be recorded with video to be later applied to other 3D digital assets in post-production. The system also allows users to load scenes based on a 3D data set created with LIDAR.

Further embodiments of the invention are described in the dependent claims.

Accordingly, the disclosure is directed to a data processing system according to claim <NUM>, a data processing method according to claim <NUM>, and a computer readable storage medium according to claim <NUM>, which may be used to solve the above technical problems.

The aspects of the disclosure provide a data processing system, including a signal processing device. The signal processing device performs: receiving a positioning data from a positioning device, wherein the positioning data corresponds to a camera device; receiving a lens parameter of the camera device from a lens encoder; encapsulating the positioning data and the lens parameter in at least one data packet; and sending the at least one data packet to a data processing device by a network interface and the data processing device, configured to perform: receiving the at least one data packet from the signal processing device and retrieving the positioning data and the lens parameter from the at least one data packet; determining a camera pose of the camera device based on the positioning data; receiving a reference clock signal and accordingly determining a timestamp corresponding to the camera pose; encapsulating the camera pose, the timestamp, and the lens parameter in a data signal; and sending the data signal; wherein the reference clock signal comprises a plurality of clock pulses, and the data processing device performs: determining the camera pose and the timestamp in response to one of the clock pulses, wherein the timestamp corresponds to a timing point of the one of the clock pulses.

The aspects of the disclosure provide a data processing method, including: receiving, by a signal processing device, a positioning data from a positioning device, wherein the positioning data corresponds to a camera device; receiving, by the signal processing device, a lens parameter of the camera device from a lens encoder; encapsulating, by the signal processing device, the positioning data and the lens parameter in at least one data packet; and sending, by the signal processing device, the at least one data packet to a data processing device by a network interface and the data processing device, configured to perform: receiving the at least one data packet from the signal processing device and retrieving the positioning data and the lens parameter from the at least one data packet; determining a camera pose of the camera device based on the positioning data; receiving a reference clock signal and accordingly determining a timestamp corresponding to the camera pose; encapsulating the camera pose, the timestamp, and the lens parameter in a data signal; and sending the data signal; wherein the reference clock signal comprises a plurality of clock pulses, and the data processing device performs: determining the camera pose and the timestamp in response to one of the clock pulses, wherein the timestamp corresponds to a timing point of the one of the clock pulses.

The aspects of the disclosure provide a computer readable storage medium, the computer readable storage medium recording an executable computer program, the executable computer program being loaded by a data processing system to perform steps of: receiving a positioning data from a positioning device, wherein the positioning data corresponds to a camera device; receiving a lens parameter of the camera device from a lens encoder; encapsulating the positioning data and the lens parameter in at least one data packet; and sending the at least one data packet to a data processing device by a network interface and the data processing device, configured to perform: receiving the at least one data packet from the signal processing device and retrieving the positioning data and the lens parameter from the at least one data packet; determining a camera pose of the camera device based on the positioning data; receiving a reference clock signal and accordingly determining a timestamp corresponding to the camera pose; encapsulating the camera pose, the timestamp, and the lens parameter in a data signal; and sending the data signal; wherein the reference clock signal comprises a plurality of clock pulses, and the data processing device performs: determining the camera pose and the timestamp in response to one of the clock pulses, wherein the timestamp corresponds to a timing point of the one of the clock pulses.

The drawings illustrate aspects of the invention and, together with the description, serve to explain the principles of the disclosure.

Reference will now be made in detail to the present preferred aspects of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers are used in the drawings and the description to refer to the same or like parts, In the present description and drawings, any examples and technical descriptions of apparatuses, products and/or methods which are not covered by the claims should be taken as background art or examples useful for understanding the invention.

See <FIG>, which shows a schematic diagram of a data processing system according to an aspect of the disclosure. In <FIG>, the data processing system <NUM> may include a signal processing device <NUM>, a positioning device <NUM>, a lens encoder <NUM>, a camera device <NUM>, a data processing device <NUM>, a terminal device <NUM>, and a clock source <NUM>.

In the aspects of the disclosure, the camera device <NUM> can be used for shooting contents of virtual production. For example, the camera device <NUM> can be disposed in a studio, wherein the studio can be disposed with, for example, a green screen and/or other equipment for shooting (e.g., light sources). In one aspect, actors/actresses can stand in front of the green screen, and the camera device <NUM> can be used to shoot images of the actors/actresses in front of the green screen. Afterwards, the image regions corresponding to the actors/actresses can be extracted and combined with virtual backgrounds as the visual contents of virtual production, but the disclosure is not limited thereto.

In one aspect, the positioning device <NUM> can be (fixedly) mounted on the camera device <NUM> via, for examples, screws or other adapting structures. In one aspect, the positioning device <NUM> can be used to detect the positioning data P1 of the positioning device <NUM>. Since the positioning device <NUM> is (fixedly) mounted on the camera device <NUM>, the positioning device <NUM> can be moved in response to the movement of the camera device <NUM>. In this case, the positioning data P1 of the positioning device <NUM> can be regarded as the positioning data P1 of the camera device <NUM>.

In some aspects, the positioning data P1 of the camera device <NUM> may include raw measurement data of the translation components and/or rotation components (e.g., the <NUM> degree of freedom) of the camera device <NUM>, but the disclosure is not limited thereto. In one aspect, the positioning device <NUM> can be a tracker attached to the camera device <NUM> and may detect the positioning data P1 of the positioning device <NUM> and/or the camera device <NUM> via, for example, outside-in tracking mechanism or inside-out mechanism. In outside-in tracking case, the environment can be disposed with several base stations emitting beacons for the tracker to accordingly detect the positioning data P1 of the positioning device <NUM> and/or the camera device <NUM>. In other aspects, the positioning device <NUM> can use any existing positioning mechanism to determine the positioning data P1 of the positioning device <NUM> and/or the camera device <NUM>.

In one aspect, the lens encoder <NUM> can be disposed on the camera device <NUM> and connected to (the lens of) the camera device <NUM> for measuring/reading/detecting the lens parameter L1 of the lens of the camera device <NUM>.

In one aspect, the lens parameter L1 obtained by the lens encoder <NUM> can include at least one counting value corresponding to at least one of an iris, a focus, a focal length of the lens of the camera device <NUM>. In one aspect, the counting value can be mapped by, for example, the terminal device <NUM> to the actual setting values of the iris, the focus, and/or the focal length of the lens of the camera device <NUM>, but the disclosure is not limited thereto.

In <FIG>, the signal processing device <NUM> may include a first data transmission port <NUM>, a second data transmission port <NUM>, a first processor <NUM>, and a network interface <NUM>. The first data transmission port <NUM> is coupled to the positioning device <NUM> and receiving the positioning data P1 from the positioning device <NUM>. The second data transmission port <NUM> is coupled to the lens encoder <NUM> and receives the lens parameter L1 of the camera device <NUM> from the lens encoder <NUM>.

In other aspects, the signal processing device <NUM> can include more data transmission ports for connecting with other positioning devices and/or lens encoders. For example, in one aspect, the signal processing device <NUM> can include two more data transmission ports for connecting two additional lens encoders and receiving the corresponding lens parameter measured by the two additional lens encoders, but the disclosure is not limited thereto.

In some aspects, the first data transmission port <NUM>, the second data transmission port <NUM> (and other data transmission ports) can be plug and play ports/interfaces, such as universal serial bus (USB) ports/interfaces, but the disclosure is not limited thereto.

The first processor <NUM> is coupled to the first data transmission port <NUM> and the second data transmission port <NUM>. In various aspects, the first processor <NUM> may be, for example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like.

In one aspect, the first processor <NUM> encapsulates the positioning data P1 and the lens parameter L1 in at least one data packet. In one aspect, the at least one data packet may include a first data packet PA1 containing the lens parameter L1 and a second data packet PA2 containing the positioning data P1, but the disclosure is not limited thereto.

In <FIG>, the network interface <NUM> is coupled to the first processor <NUM> and send the at least one data packet to the data processing device <NUM> for further processing. In the aspects of the disclosure, the network interface <NUM> (and other network interfaces considered in the disclosure) can be, for example, Ethernet interfaces, but the disclosure is not limited thereto.

In one aspect, the first data packet PA1 can be a User Datagram Protocol (UDP) packet, and the first processor <NUM> can control the network interface <NUM> to send the first data packet PA1 via a UDP protocol. In addition, the second data packet PA2 can be a Transmission Control Protocol / Internet Protocol (TCP/IP) packet, and the first processor <NUM> can control the network interface <NUM> to send the second data packet PA2 via a Universal Serial Bus / IP (USB/IP) protocol, but the disclosure is not limited thereto.

In other aspects, the first data packet PA1 and the second data packet PA2 can be implemented as the packets corresponding to the desired protocol of the designer.

In <FIG>, the data processing device <NUM> includes a first network interface <NUM>, a second network interface <NUM>, and a second processor <NUM>.

In one aspect, the second processor <NUM> is coupled to the first network interface <NUM> and the second network interface <NUM>, and the possible implementations of the second processor <NUM> may be referred to the descriptions of the first processor <NUM>, but the disclosure is not limited thereto.

In one aspect, the first network interface <NUM> is coupled to the network interface <NUM> of the signal processing device <NUM> and receives the at least one data packet (e.g., the first data packet PA1 and the second data packet PA2) from the signal processing device <NUM>. In one aspect, the second processor <NUM> may retrieve the positioning data P1 and the lens parameter L1 from the at least one data packet. For example, the second processor <NUM> may retrieve the lens parameter L1 from the first data packet PA1 and retrieve the positioning data P1 from the second data packet PA2, but the disclosure is not limited thereto.

In one aspect, the second processor <NUM> determines a camera pose PO of the camera device <NUM> based on the positioning data P1. In the aspect where the positioning data P1 includes the raw measurement data of the translation components and/or rotation components of the camera device <NUM>, the second processor <NUM> may analyse/process the positioning data P1 and accordingly determine the actual translation components and/or rotation components of the camera device <NUM> as the camera pose PO. From another perspective, the signal processing device <NUM> forwards the raw measurement data (e.g., the positioning data P1 detected by the positioning device <NUM>) of the camera pose PO to the data processing device <NUM>, and the data processing device <NUM> accordingly determine the camera pose PO by analysing the raw measurement data, but the disclosure is not limited thereto.

In one aspect, the second processor <NUM> receives a reference clock signal CS from the clock source <NUM> coupled to the camera device <NUM>, the data processing device <NUM>, and the terminal device <NUM>. In one aspect, the reference clock signal CS can be a reference signal for the camera device <NUM>, the data processing device <NUM>, and the terminal device <NUM> to perform synchronization. In one aspect, the reference clock signal CS may be a pulse train including a plurality of clock pulses, wherein the clock pulses may be provided under a specific frequency. In this case, the duration between adjacent clock pulses can be a reciprocal of the specific frequency.

In one aspect, the second processor <NUM> may determine a timestamp TS corresponding to the camera pose PO. For example, the second processor <NUM> can determine camera pose PO and the timestamp TS in response to one of the clock pulses, wherein the timestamp TS corresponds to a timing point of the one of the clock pulses. In one aspect, the second processor <NUM> can use the timing point of the one of the clock pulses as the timestamp TS corresponding to the camera pose PO. That is, the second processor <NUM> follows the timing of the reference clock signal CS to determine the camera pose PO and the corresponding timestamp TS.

In one aspect, the second processor <NUM> encapsulate the camera pose PO, the timestamp TS, and the lens parameter L1 in a data signal S1 and sends the data signal S1 via the second network interface <NUM>.

In <FIG>, the terminal device <NUM> coupled to the data processing device <NUM> can be any smart devices and/or computer devices used for rendering visual contents of the virtual production. In one aspect, the terminal device <NUM> can be the computer used by the content creator for editing the visual content of the virtual production, but the disclosure is not limited thereto.

In one aspect, the terminal device <NUM> may receive the data signal S1 from the data processing device <NUM>. Accordingly, the terminal device <NUM> could know the lens parameter L1 and the status of the camera pose PO at the timestamp TS from the data signal S1. In addition, each image captured by camera device <NUM> can be labelled, by the camera device <NUM>, with corresponding timestamp based on the reference clock signal CS, and the captured images with the corresponding timestamp would be provided to the terminal device <NUM>. In this case, the terminal device <NUM> can know the camera poses of the camera device <NUM> when shooting the images based on the timestamp of each image and each camera pose.

For example, when the terminal device <NUM> determines that a specific image among the images captured by the camera device <NUM> has a timestamp same as the timestamp TS, the terminal device <NUM> can determine the camera pose PO is the camera pose of the camera device <NUM> when shooting the specific image.

In one aspect, the terminal device <NUM> can run/render a virtual scene (e.g., a VR scene). In this case, when the camera device <NUM> is used to shoot some real objects (e.g., actor/actress) as the specific image, the terminal device <NUM> can crop/segment the image regions corresponding to the real objects from the specific image and combine the cropped image regions with the virtual scene to produce a virtual production video. In the aspect, the virtual scene includes a virtual camera corresponding to the camera device <NUM>, and the pose of the virtual camera in the virtual scene needs to be referred to the camera pose PO (e.g., rotation and/or translation) of the camera device <NUM> when shooting the specific image. Since the solution provided by the aspects of the disclosure can obtain accurate camera pose PO of the camera device <NUM>, the cropped image region can be accurately combined with the virtual scene, which improves the quality of the virtual production.

In one aspect, the signal processing device <NUM> can further receive a lens controlling signal LC and accordingly control the lens encoder <NUM> to adjust a lens setting of the camera device <NUM>. In one aspect, the lens setting of the camera device <NUM> may include, for example, the iris, focus, and/or focal length of the lens of the camera device <NUM>, and the user can determine the lens setting via, for example, operating a control interface (e.g. software/hardware) of the terminal device <NUM>. In this case, the terminal device <NUM> can control the data processing device <NUM> to send the lens controlling signal LC to the signal processing device <NUM> according to the setting of the user. Accordingly, the signal processing device <NUM> can control the lens encoder <NUM> to adjust the lens setting of the camera device <NUM> to be corresponding to the requirement of the user, but the disclosure is not limited thereto.

See <FIG>, which shows a flow chart of the data processing method according to an aspect of the disclosure. The method of this aspect may be executed by the signal processing device <NUM> in <FIG>, and the details of each step in <FIG> will be described below with the components shown in <FIG>.

In step S210, the signal processing device <NUM> receives the positioning data P1 from the positioning device <NUM>. In step S220, the signal processing device <NUM> receives the lens parameter L1 of the camera device <NUM> from the lens encoder <NUM>. In step S230, the signal processing device <NUM> encapsulates the positioning data P1 and the lens parameter L1 in the at least one data packet (e.g., the first data packet PA1 and the second data packet PA2). In step S240, the signal processing device <NUM> sends the at least one data packet. Details of the steps in <FIG> can be referred to the descriptions in the above, which would not be repeated herein.

See <FIG>, which shows a flow chart of the data processing method according to an aspect of the disclosure. The method of this aspect may be executed by the data processing device <NUM> in <FIG>, and the details of each step in <FIG> will be described below with the components shown in <FIG>.

In step S310, the data processing device <NUM> receives the at least one data packet from the signal processing device <NUM> and retrieves the positioning data P1 and the lens parameter L1 from the at least one data packet. In step S320, the data processing device <NUM> determines the camera pose PO of the camera device <NUM> based on the positioning data P <NUM>. In step S330, the data processing device <NUM> receives the reference clock signal CS and accordingly determine the timestamp TS corresponding to the camera pose PO. In step S340, the data processing device <NUM> encapsulates the camera pose PO, the timestamp TS, and the lens parameter L1 in the data signal S1. In step S350, the data processing device <NUM> sends the data signal S1. Details of the steps in <FIG> can be referred to the descriptions in the above, which would not be repeated herein.

As can be understood in the above, the aspects of the disclosure provide an easy, effective solution to accurately track the camera pose of the camera device when shooting contents. In the aspects of the disclosure, the user (e.g., content creators) only needs to install the positioning device <NUM> and the lens encoder <NUM> onto the camera device <NUM>, and the signal processing device <NUM> and the data processing device <NUM> can cooperate to facilitate the terminal device <NUM> to synchronize the captured images and the camera poses of the camera device <NUM>. Accordingly, the efficiency of creating virtual production can be improved.

In addition, the aspects of the disclosure further provide a method for determining coordinates, which can be carried out by the data processing system of the disclosure, and the details thereof would be introduced in the following.

See <FIG>, which shows the data processing system according to <FIG> of the disclosure. In <FIG>, the data processing system <NUM> may include signal processing devices 11a, 11b, positioning devices 12a, 12b, and the data processing device <NUM>. In the aspect, each of the signal processing devices 11a and 11b can be the same kind of device as the signal processing device <NUM> in <FIG>. Each of the positioning devices 12a and 12b can be the same kind of device as the positioning device <NUM> in <FIG>, and the data processing device <NUM> in <FIG> can be the same device as the data processing device <NUM> in <FIG>. In addition, the data processing system <NUM> can further include other devices in <FIG>, such as the lens encoder <NUM>, the terminal device <NUM>, the clock source <NUM>, but the disclosure is not limited thereto.

In the aspects of the disclosure, similar to the situation of the signal processing device <NUM> and the positioning device <NUM> being mounted on the camera device <NUM> in <FIG>, the signal processing device 11a and the positioning device 12a can be mounted on the corresponding camera device (not shown) or other to be tracked object/locations. Likewise, the signal processing device 11b and the positioning device 12b can be mounted on the corresponding camera device (not shown) or other to be tracked object/locations.

In addition, although only two combinations of the signal processing device and the positioning device (each combination at least includes one signal processing device and one positioning device) is shown in <FIG>, the data processing system <NUM> may include more combinations of the signal processing device and the positioning device, but the disclosure is not limited thereto. In addition, for connecting with the signal processing devices 11a, 11b (and other additional signal processing devices), the data processing device <NUM> can be disposed with corresponding amount of first network interfaces <NUM>. For example, if the data processing device <NUM> needs to be able to be connected with up to N (N is a positive integer) signal processing devices, the data processing device <NUM> can be designed with N first network interfaces <NUM>, but the disclosure is not limited thereto.

In some aspects, the data processing device <NUM> may determine the camera poses of the camera devices corresponding to the positioning device 12a, 12b based on the positioning data from the positioning devices 12a, 12b. The data processing device <NUM> can send the determined camera poses of the camera devices to the terminal device <NUM> for the terminal device <NUM> to produce the corresponding virtual production (e.g., combining cropped image regions corresponding to real objects with the rendered virtual scene), but the disclosure is not limited thereto.

See <FIG>, which shows a flow chart of the method for determining coordinates according to an aspect of the disclosure. The method of this aspect may be executed by the data processing device <NUM> in <FIG>, and the details of each step in <FIG> will be described below with the components shown in <FIG>.

In step S510, the data processing device <NUM> receives a plurality of positioning data, wherein the plurality of positioning data correspond to device positions of a plurality of positioning devices in a real world, and the positioning devices comprise a first positioning device and a second positioning device.

For better explaining the concept of the disclosure, the positioning device 12a and 12b would be respectively assumed to be the first positioning device and the second positioning device, but the disclosure is not limited thereto.

In <FIG>, the plurality of positioning data received by data processing device <NUM> may include a first positioning data P11 corresponding to the positioning device 12a and a second positioning data P12 corresponding to the positioning device 12b. In the aspect, the data processing device <NUM> may receive the first positioning data P11 from the signal processing device 11a connected to the positioning device 12a and receive the second positioning data P12 from the signal processing device 11b connected to the positioning device 12b. Specifically, the signal processing device 11a may receive the positioning data P11 from the positioning device 12a and send the positioning data P11 to the data processing device <NUM>, and the signal processing device 11b may receive the positioning data P12 from the positioning device 12b and send the positioning data P12 to the data processing device <NUM>.

In other aspects, the data processing device <NUM> can receive the positioning data P11 and P12 directly from each of the positioning data 12a and 12b, but the disclosure is not limited thereto.

As described in the aspects of <FIG>, the positioning data P11 may correspond to a first camera pose of a first camera device on which the signal processing device 11a and the positioning device 12a are mounted, and the positioning data P12 may correspond to a second camera pose of a second camera device on which the signal processing device 11b and the positioning device 12b are mounted, but the disclosure is not limited thereto.

In one aspect, the positioning devices 12a and 12b may obtain the positioning data P11 and P12 via the outside-in tracking mechanism or inside-out tracking mechanism (e.g. SLAM). In outside-in tracking case, the positioning data P11 and P12 can be understood as characterizing the positions of the positioning devices 12a and 12b relative to, for example, the base stations, but the disclosure is not limited thereto.

In one aspect, the data processing device <NUM> can determine the device positions of the positioning devices 12a and 12b in the real world. In an aspect where the positioning data P11 includes the raw measurement data of the translation components and/or rotation components of the positioning device 12a, the data processing device <NUM> may analyse/process the positioning data P11 and accordingly determine the actual translation components and/or rotation components of the positioning device 12a, which can be used to determine the device position of the positioning device 12a in the real world. Similarly, in an aspect where the positioning data P12 includes the raw measurement data of the translation components and/or rotation components of the positioning device 12b, the data processing device <NUM> may analyse/process the positioning data P12 and accordingly determine the actual translation components and/or rotation components of the positioning device 12b, which can be used to determine the device position of the positioning device 12b in the real world.

In step S520, in response to determining that the first positioning device (e.g., the positioning device 12a) is selected as a reference point of a coordinate system of a virtual world, the data processing device <NUM> determines a coordinate of the second positioning device (e.g., the positioning device 12b) in the coordinate system of the virtual world based on a relative position between the device positions of the first positioning device and the second positioning device.

For better explaining the concept of the disclosure, the reference point of the coordinate system of the virtual world would be assumed to be the origin of the coordinate system. In other aspects, the reference point can be any other desired point in the coordinate system.

For example, it is assumed that a default reference point is the position of one of the base stations (i.e., the position of the base station regarded as the reference point may be (<NUM>, <NUM>, <NUM>)), and the device positions of the positioning device 12a and 12b are respectively (x1, y1, z1) and (x2, y2, z2) in the real world. In this case, when the positioning device 12a is selected as the origin of the coordinate system of the virtual world, the coordinate of the positioning device 12a in the coordinate system of the virtual world can be, for example, (<NUM>,<NUM>,<NUM>). Based on the relative position between the device positions of the positioning device 12a and 12b, the data processing device <NUM> may determine the coordinate of the positioning device 12b in the coordinate system of the virtual world to be, for example, (x2-x1, y2-y1, z2-z1), but the disclosure is not limited thereto.

Accordingly, the data processing device <NUM> can easily use the relative positions between each of the positioning devices 12a, 12b and the base stations in the real world to determine the coordinates of the positioning devices 12a, 12b in the virtual world, which increases the convenience of creating virtual production.

In one aspect, the data processing device <NUM> can further provide the coordinates of the positioning devices 12a, 12b to the terminal device <NUM> in <FIG>, such that the terminal device <NUM> can use these information to create a visual content with better quality, but the disclosure is not limited thereto.

In addition, by using the solution introduced in the disclosure, the coordinates of the positioning devices 12a, 12b in the virtual world can be easily determined even if the base stations are moved. To be specific, the user may change the locations of the base stations in the studio based on particular requirements. In this case, the user can simply determine the positioning device 12a as the reference point, and the coordinates of the positioning devices 12a and 12b in the virtual world can be properly determined even if the base stations have been moved to new locations.

For example, after the base stations used for outside-in tracking are moved, the devices positions of the positioning devices 12a and 12b may respectively become (x1', y1', z1') and (x2', y2', z2') relative to the base stations. However, when the positioning device 12a is selected as the reference point again, the coordinates of the positioning devices 12a and 12b in the virtual world would be respectively determined to be (<NUM>, <NUM>, <NUM>) and (x2'-x1', y2'-y1', z2'-z1'), wherein (x2'-x1', y2'-y1', z2'-z1') would be the same as (x2-x1, y2-y1, z2-z1). Accordingly, the efficiency of creating virtual production can be improved.

In one aspect, in response to determining that the second positioning device (e.g., the positioning device 12b) is selected as the reference point of the coordinate system of the virtual world, the data processing device <NUM> can further determine a coordinate of the first positioning device (e.g., the positioning device 12a) in the coordinate system of the virtual world based on the relative position between the device positions of the first positioning device and the second positioning device.

In this case, when the positioning device 12b is selected as the origin of the coordinate system of the virtual world, the coordinate of the positioning device 12b in the coordinate system of the virtual world can be, for example, (<NUM>,<NUM>,<NUM>). Based on the relative position between the device positions of the positioning device 12a and 12b, the data processing device <NUM> may determine the coordinate of the positioning device 12b in the coordinate system of the virtual world to be, for example, (x1-x2, y1-y2, z1-z2), but the disclosure is not limited thereto.

That is, after the positioning device 12b is used as the origin, the data processing device <NUM> can change to use the positioning device 12b as the origin of the coordinate system of the virtual world, and the coordinates of the positioning devices 12a, 12b would be accordingly updated.

In the aspect where the data processing system <NUM> includes other positioning devices, the data processing device <NUM> can determine a coordinate of each of other positioning devices in the coordinate system of the virtual world based on a relative position between the device positions of the positioning device selected as the reference point and each of the other positioning devices, but the disclosure is not limited thereto.

See <FIG>, which shows a user interface provided by the data processing device <NUM> according to an aspect of the disclosure. In <FIG>, the data processing device <NUM> may provide a user interface <NUM>, wherein the user interface <NUM> shows a re-center function <NUM>. In the aspect, the re-center function <NUM> can be used by the user to select a desired one of the positioning devices as the reference point of the coordinate system of the virtual world.

In one aspect, in response to determining that the re-center function <NUM> is triggered, the data processing device <NUM> can provide a plurality of detected positioning devices in the user interface <NUM>. In one aspect, the data processing device <NUM> can detect the positioning devices by checking the positioning device information in the package data/pose data received from the positioning device(s), but the disclosure is not limited thereto. In one aspect, the user interface <NUM> can only show the positioning devices currently detected by the data processing device <NUM>.

In <FIG>, assuming that the icons 614a-614c respectively corresponds to the positioning devices 12a, 12b, and another positioning device detected by the data processing device <NUM>. In this case, the user can select the desired positioning device as the reference point of the coordinate system of the virtual world.

In one aspect, assuming that the icon 614a corresponding to the positioning device 12a is selected by the user, the data processing device <NUM> can accordingly perform step S520 based on the descriptions in the above, but the disclosure is not limited thereto.

The disclosure further provides computer readable storage mediums for executing the data processing method and/or the method for determining coordinates. The computer readable storage medium is composed of a plurality of program instructions (for example, a setting program instruction and a deployment program instruction) embodied therein. These program instructions can be loaded into the data processing device and the signal processing device and executed by the same to execute the data processing method and/or the method for determining coordinates and the functions of the data processing device and the signal processing device described above.

In summary, the aspects of the disclosure provide an easy, effective solution to accurately track the camera pose of the camera device when shooting contents. In the aspects of the disclosure, the user (e.g., content creators) only needs to install the positioning device and/or the lens encoder onto the camera device, and the signal processing device and the data processing device can cooperate to facilitate the terminal device to synchronize the captured images and the camera poses of the camera device. Accordingly, the terminal device can combine the cropped image region (e.g., the image region corresponding to real objects such as actor/actress) with the rendered virtual scene more accurately based on the camera pose of the camera device, which improves the quality and the efficiency of creating virtual production.

Claim 1:
A data processing system (<NUM>), comprising:
a signal processing device (<NUM>), configured to perform:
receiving a positioning data (P1) from a positioning device (<NUM>), wherein the positioning data corresponds to a camera device (<NUM>);
receiving a lens parameter (L1) of the camera device from a lens encoder (<NUM>);
encapsulating the positioning data and the lens parameter in at least one data packet (PA1, PA2); and
sending the at least one data packet to a data processing device (<NUM>) by a network interface (<NUM>) and
the data processing device, configured to perform:
receiving the at least one data packet from the signal processing device and retrieving the positioning data and the lens parameter from the at least one data packet;
determining a camera pose of the camera device based on the positioning data;
receiving a reference clock signal (CS) and accordingly determining a timestamp corresponding to the camera pose;
encapsulating the camera pose, the timestamp, and the lens parameter in a data signal; and
sending the data signal;
wherein the reference clock signal comprises a plurality of clock pulses, and the data processing device performs:
determining the camera pose and the timestamp in response to one of the clock pulses, wherein the timestamp corresponds to a timing point of the one of the clock pulses.