Patent AU-2023202859-B2 (AU)
Title: Method And Apparatus For Constructing Real-Geographic-Space Scene In Real Time
Inventor: MAO SHANJUN
IPC: G06T19/20
Filed: 2023-05-08
Granted: 2024-01-04
Abstract: The present disclosure provides a method for constructing a real-geographic-space scene in 
real time based on a panoramic-video technique. By using a measuring robot and the attitude 
sensors, accurately determining the geographic coordinates and the attitudes of the cameras, 
wherein the cameras may be installed in a fixed or stringing manner, wherein in the fixing type a 
plurality of neighboring videos at the same moment undergo orthographic correction and splicing, 
and in the stringing type the cameras are installed to a guiding device and may locally, 
independently and quickly move and shoot, and the videos of the neighboring cameras are spliced 
in real time; and fusing the videos, the geographic coordinates and the environment sounds that 
satisfy the delay time, to form a scene video streaming.
1/4 
by a high-performance computing device, receiving image data collected by a plurality of 
panoramic cameras, and combining the image data into an image-data sequence, wherein 
the plurality of panoramic cameras are controlled by using a remote-control instruction, and 101 
the plurality of panoramic cameras are installed in the real-geographic-space scene in a 
fixing manner or a stringing manner, and provide marking points in the real-geographic
space scene 
by the high-performance computing device, receiving a three-dimensional geographic 
coordinate of each of the plurality of panoramic cameras in the scene, wherein the three- 102 
dimensional geographic coordinate is, by a measuring robot, acquired by using a prism 
installed at each of the plurality of panoramic cameras and subsequently sent 
by the high-performance computing device, receiving external-azimuth parameters from 
attitude sensors, wherein each of the plurality of panoramic cameras is installed with one of 
the attitude sensors, and each of the attitude sensors is configured for determining in real 
time an azimuth angle, a pitch angle and a roll angle of the panoramic camera where the 103 
attitude sensor is located, and providing the external-azimuth parameters in 
photogrammetry, to facilitate subsequent orthographic geometric correction, splicing and 
optimization of a panoramic image 
by the high-performance computing device, receiving a scene environment sound, wherein 
the scene environment sound is acquired and sent by a plurality of audio collecting devices 104 
in the real-geographic-space scene 
by the high-performance computing device, according to different installation modes, the 
image-data sequence, the marking points, the external-azimuth parameters and the three
dimensional geographic coordinates, performing splicing-stitching and the orthographic 
geometric correction or distortion correction to the image-data sequence, to obtain an 
orthographic image or a panoramic-image sequence, wherein the different installation 
modes use different methods of splicing optimization, the orthographic image is obtained 105 
by splicing-stitching and orthographic geometric correction to an image sequence obtained 
in a fixing-type installation mode, and the panoramic-image sequence is obtained by 
splicing-stitching and distortion correction to an image sequence obtained in a stringing
type installation mode 
by the high-performance computing device, performing augmented-reality optimization to 
the synthesized orthographic image or panoramic-image sequence with the scene 
environment sound and the three-dimensional geographic coordinates, to obtain a real- 106 
geographic-space-scene video streaming having augmented reality, space audios and 
geographic coordinates located in a geographic-coordinate frame 
by the high-performance computing device, sending the real-geographic-space-scene video 
streaming to an output controlling device, so that the output controlling device displays the 
real-geographic-space scene in real time, and an operator sends the remote-control 107 
instruction by using the output controlling device, to remotely in real time control all or part 
of the panoramic cameras to shoot or move 
FIG. 1