Patent ID: 11908344
Assignee: CENTRAL CHINA NORMAL UNIVERSITY
Field: Computer technology (Electrical engineering)
Classification: CPC G  Y | IPC G

Claim 7:
8. The method of claim 5, wherein step (S3) comprises:
(S3-1) setting a field mode of the RGB-D camera to a narrow view, with a central field area front of the electronic sand table; accurately acquiring background environment of the teaching activity area and texture images and depth point cloud sequence data of the teacher during the teaching process; and transmitting the background environment, the texture images and the depth point cloud sequence data to the acquisition and processing module on the electronic sand table through Bluetooth, WIFI or USB 3.0;
(S3-2) sending, by the acquisition and processing module, the depth point cloud sequence data captured by the RGB-D camera to the edge computing server according to time sequence; extracting a two-dimensional (2D) confidence map and an association field of a transformed image using the Mask R-CNN neural network technology, and solving connection between the 2D confidence map and the association field using bipartite matching in graph theory; extracting the skeleton data of the teacher in the point cloud sequence data; and extracting the changes in the skeletal joints of the teacher during the teaching process according to changes in RGB and depth data of points in the point cloud sequence data though edge computing;
(S3-3) receiving the skeletal data of the teacher transmitted back from the edge computing server; extracting 3D coordinates of each joint point using the segmentation strategy; calculating distances between adjacent joint points and horizontal components thereof; and connecting and merging key nodes of the skeleton of the teacher using a Hungarian algorithm based on a vectorial nature of the skeleton of the teacher.